Fixed broken sdcard.py library (previously impossible to have external SPI uSD card with Pycom boards)



  • I've had many problems with Pycom boards, mostly LTE-radio-related. I recently solved one issue which I share here for the benefit of others

    No serious application would employ the overwrought, power-consumptive Pycom expansion boards, which contain SDIO-based SD card modules. An external SPI-based SD card module would be used in any serious application for low-cost, power-efficient persistent storage.

    This is currently impossible, since Pycom has:

    1. Failed to correct the SPI bug noted here: https://forum.pycom.io/topic/1260/spi-bug/2
    2. Failed to update sdcard.py to comport with Pycom
    3. Failed to provide any resolution in the following thread which would enable someone to actually get an SD card working: https://forum.pycom.io/topic/2064/external-sd-card

    I can't believe I'm the first one to post a solution to these issues. I like the direction of the Pycom products and would love to keep using them, but there are lot of basic problems like this which need to be solved.

    The attached sdcard.py is edited to run on Pycom boards (most of that was noted in the thread above) AND contains a proxy SPI "read" function that actually obeys the write=argument. That Pycom's default SPI library is still broken in November 2018 is beyond my comprehension. (See this thread: https://forum.pycom.io/post/23519)

    0_1542239004189_sdcard.py

    
    """
    EDITED November 2018 by Paul Maravelias
    The original library fails for multiple reasons:
    1. Incompatible syntax (low()/high() instead of value(0/1) and positional argument errors)
    2. Pycom SPI bug (spi.read() IGNORES the write= argument)
    I overcame the second issue by reimplementind spi.read() using only spi.write_readinto
    Tested working on GPy + SparkFun microSD breakout module on SPI(0)
    """
    """
    MicroPython driver for SD cards using SPI bus.
    
    Requires an SPI bus and a CS pin.  Provides readblocks and writeblocks
    methods so the device can be mounted as a filesystem.
    
    Example usage on xxPy:
    
        import machine, sdcard, os
        sd = sdcard.SDCard(machine.SPI(0), machine.Pin("P9"))
        os.mount(sd, '/sd2')
        os.listdir('/sd2')
    
    
    """
    from micropython import const
    import time
    
    
    _CMD_TIMEOUT = const(100)
    
    _R1_IDLE_STATE = const(1 << 0)
    #R1_ERASE_RESET = const(1 << 1)
    _R1_ILLEGAL_COMMAND = const(1 << 2)
    #R1_COM_CRC_ERROR = const(1 << 3)
    #R1_ERASE_SEQUENCE_ERROR = const(1 << 4)
    #R1_ADDRESS_ERROR = const(1 << 5)
    #R1_PARAMETER_ERROR = const(1 << 6)
    _TOKEN_CMD25 = const(0xfc)
    _TOKEN_STOP_TRAN = const(0xfd)
    _TOKEN_DATA = const(0xfe)
    
    class SDCard:
        def __init__(self, spi, cs):
            self.spi = spi
            self.cs = cs
    
            self.cmdbuf = bytearray(6)
            self.dummybuf = bytearray(512)
            for i in range(512):
                self.dummybuf[i] = 0xff
            self.dummybuf_memoryview = memoryview(self.dummybuf)
    
            # initialise the card
            self.init_card()
    
        def spiread(self, nbytes, write=0x00):
            wb = bytearray()
            wb.extend(bytearray([write]*nbytes))
            rb = bytearray(len(wb))
            self.spi.write_readinto(wb,rb)
            return rb
    
        def init_spi(self, baudrate):
            master = self.spi.MASTER
            self.spi.init(master, baudrate=baudrate, phase=0, polarity=0)
    
        def init_card(self):
            # init CS pin
            #self.cs.init(self.cs.OUT, value=0)
    
            # init SPI bus; use low data rate for initialisation
            #self.init_spi(100000)
    
            # clock card at least 100 cycles with cs high
            for i in range(16):
                self.spi.write(b'\xff')
    
            # CMD0: init card; should return _R1_IDLE_STATE (allow 5 attempts)
    
            for _ in range(5):
                if self.cmd(0, 0, 0x95) == _R1_IDLE_STATE:
                    break
            '''
            else:
                raise OSError("no SD card")
            '''
    
            # CMD8: determine card version
            r = self.cmd(8, 0x01aa, 0x87, 4)
            if r == _R1_IDLE_STATE:
                self.init_card_v2()
            elif r == (_R1_IDLE_STATE | _R1_ILLEGAL_COMMAND):
                self.init_card_v1()
            else:
                raise OSError("couldn't determine SD card version")
    
            # get the number of sectors
            # CMD9: response R2 (R1 byte + 16-byte block read)
            if self.cmd(9, 0, 0, 0, False) != 0:
                raise OSError("no response from SD card")
            csd = bytearray(16)
            self.readinto(csd)
            if csd[0] & 0xc0 != 0x40:
                raise OSError("SD card CSD format not supported")
            self.sectors = ((csd[8] << 8 | csd[9]) + 1) * 2014
            #print('sectors', self.sectors)
    
            # CMD16: set block length to 512 bytes
            if self.cmd(16, 512, 0) != 0:
                raise OSError("can't set 512 block size")
    
            # set to high data rate now that it's initialised
            #self.init_spi(100000)
            #self.init_spi(40000000)
    
        def init_card_v1(self):
            for i in range(_CMD_TIMEOUT):
                self.cmd(55, 0, 0)
                if self.cmd(41, 0, 0) == 0:
                    self.cdv = 512
                    #print("[SDCard] v1 card")
                    return
            raise OSError("timeout waiting for v1 card")
    
        def init_card_v2(self):
            for i in range(_CMD_TIMEOUT):
                time.sleep_ms(50)
                self.cmd(58, 0, 0, 4)
                self.cmd(55, 0, 0)
                if self.cmd(41, 0x40000000, 0) == 0:
                    self.cmd(58, 0, 0, 4)
                    self.cdv = 1
                    #print("[SDCard] v2 card")
                    return
            raise OSError("timeout waiting for v2 card")
    
        def cmd(self, cmd, arg, crc, final=0, release=True):
            self.cs(0)
    
            # create and send the command
            buf = self.cmdbuf
            buf[0] = 0x40 | cmd
            buf[1] = arg >> 24
            buf[2] = arg >> 16
            buf[3] = arg >> 8
            buf[4] = arg
            buf[5] = crc
            self.spi.write(buf)
    
            # wait for the response (response[7] == 0)
            for i in range(_CMD_TIMEOUT):
                #response = self.spi.read(1, write=0xff)[0]
                response = self.spiread(1, write=0xff)[0]
    
                if not (response & 0x80):
                    # this could be a big-endian integer that we are getting here
                    for j in range(final):
                        self.spi.write(b'\xff')
                    if release:
                        self.cs(1)
                        self.spi.write(b'\xff')
                    return response
    
            # timeout
            self.cs(1)
            self.spi.write(b'\xff')
            return -1
    
        def cmd_nodata(self, cmd):
            self.spi.write(cmd)
            #self.spi.read(1, write=0xff) # ignore stuff byte
            self.spiread(1, write=0xff) # ignore stuff byte
            for _ in range(_CMD_TIMEOUT):
                #if self.spi.read(1, write=0xff)[0] == 0xff:
                if self.spiread(1, write=0xff)[0] == 0xff:
                    self.cs(1)
                    self.spi.write(b'\xff')
                    return 0    # OK
            self.cs(1)
            self.spi.write(b'\xff')
            return 1 # timeout
    
        def readinto(self, buf):
            self.cs(0)
    
            # read until start byte (0xff)
            #while self.spi.read(1, write=0xff)[0] != 0xfe:
            while self.spiread(1, write=0xff)[0] != 0xfe:
                pass
    
            # read data
            mv = self.dummybuf_memoryview[:len(buf)]
            self.spi.write_readinto(mv, buf)
    
            # read checksum
            self.spi.write(b'\xff')
            self.spi.write(b'\xff')
    
            self.cs(1)
            self.spi.write(b'\xff')
    
        def write(self, token, buf):
            self.cs(0)
    
            # send: start of block, data, checksum
            #self.spi.read(1, write=token)
            self.spiread(1, write=token)
            self.spi.write(buf)
            self.spi.write(b'\xff')
            self.spi.write(b'\xff')
    
            # check the response
            #if (self.spi.read(1, write=0xff)[0] & 0x1f) != 0x05:
            if (self.spiread(1, write=0xff)[0] & 0x1f) != 0x05:
                self.cs(1)
                self.spi.write(b'\xff')
                return
    
            # wait for write to finish
            #while self.spi.read(1, write=0xff)[0] == 0:
            while self.spiread(1, write=0xff)[0] == 0:
                pass
    
            self.cs(1)
            self.spi.write(b'\xff')
    
        def write_token(self, token):
            self.cs(0)
            #self.spi.read(1, write=token)
            self.spiread(1, write=token)
            self.spi.write(b'\xff')
            # wait for write to finish
            #while self.spi.read(1, write=0xff)[0] == 0x00:
            while self.spiread(1, write=0xff)[0] == 0x00:
                pass
    
            self.cs(1)
            self.spi.write(b'\xff')
    
        def count(self):
            return self.sectors
    
        def readblocks(self, block_num, buf):
            nblocks, err = divmod(len(buf), 512)
            assert nblocks and not err, 'Buffer length is invalid'
            if nblocks == 1:
                # CMD17: set read address for single block
                if self.cmd(17, block_num * self.cdv, 0) != 0:
                    return 1
                # receive the data
                self.readinto(buf)
            else:
                # CMD18: set read address for multiple blocks
                if self.cmd(18, block_num * self.cdv, 0) != 0:
                    return 1
                offset = 0
                mv = memoryview(buf)
                while nblocks:
                    self.readinto(mv[offset : offset + 512])
                    offset += 512
                    nblocks -= 1
                return self.cmd_nodata(b'\x0c') # cmd 12
            return 0
    
        def writeblocks(self, block_num, buf):
            nblocks, err = divmod(len(buf), 512)
            assert nblocks and not err, 'Buffer length is invalid'
            if nblocks == 1:
                # CMD24: set write address for single block
                if self.cmd(24, block_num * self.cdv, 0) != 0:
                    return 1
    
                # send the data
                self.write(_TOKEN_DATA, buf)
            else:
                # CMD25: set write address for first block
                if self.cmd(25, block_num * self.cdv, 0) != 0:
                    return 1
                # send the data
                offset = 0
                mv = memoryview(buf)
                while nblocks:
                    self.write(_TOKEN_CMD25, mv[offset : offset + 512])
                    offset += 512
                    nblocks -= 1
                self.write_token(_TOKEN_STOP_TRAN)
            return 0


  • @robert-hh Thanks, I am working on a very small laptop so I didn't see that as it wasn't very obvious :-)



  • @stevo52 For tags: At the bottom line where it say "enter tag", just type the words that you like to tag the post with,



  • @paulm Thanks for the reply. It took a while but I think I have it sorted now, using 'P10', 'P11' ,'P14' and 'P9' for CS. I can on;y get the SPI interface to work at > 10000000 though but that works reliably at least.

    This forum is not the easiest to work with - I am currently trying to post a query about i2c but get tols I need to 'add a tag' but I can't find out how to do that.. I am sure it isn't hard, but again prior knowledge is 'assumed'..



  • @stevo52 Glad I could be of help. I am using the following for pin config on a new GPy. I can understand how getting this working would drive someone crazy, since Pycom has still failed to fix the SPI library:
    spi = SPI(0, mode=SPI.MASTER, baudrate=1000000, polarity=0, phase=0, pins=('P10','P11','P8'))
    sd = sdcard.SDCard(spi,Pin('P9', Pin.OUT))



  • @paulm - very glad to find this because the combination of SD Card, SPI and GPY has had me going around in circles trying to get a workable solution.
    Can you advise what pins you used for the SPI (default SPI)?

    On the GPY/Expansion board these aren't very clear to me (noobie) but seem to be from these:

    P4 = SD CMD = ?
    P8 = SD DAT0 = SPI
    P9 = SDA = MOSI = ?
    P14 = MISO = ?
    P23 = SD CLK = ? = ?

    Getting the right pins is part of my problems in getting an external SD Card to work. I'd really like to clarify the pins - a lot of posts here assume the OP knows all the in's and outs but the Pycom documentation has a few holes in it I think, which is compounded by old, incorrect information

    Stevo



  • @robert-hh Thanks for the background info. All I know is this is necessary for using external SPI SD cards, and it would not work by following solely the instructions in the thread I linked above.



  • @paulm The driver you mentions was just a left-over from the micropython.org repository, which was used as a base for Pycom's variant of MicroPython. There is a LOT of stuff lying around in the repository which is not needed by Pycom and may confuse people, obvious stuff like STM32 or ESP8266 ports, and less obvious things like the driver you adapted.
    I suggested already to clean up the repository, but no-one seems to have or take time for that. Tidying things up seems not to be the most attractive task. But, during tidying this driver would have been removed.



Pycom on Twitter