Using SD Card for the Whole Project.

[hakan kayan]

Hi,

I am trying to implement a machine learning method on Pysense + LopY combo. I need to use Python libraries(e.g. Numpy, Scikit) that uses more a space than the flash memory, so I want to utilize SD card for that. I have several question that might help to achieve my task:

• Can I use SD card as to manage the whole project? Currently upload & download via REPL is only using flash memory. I created a data.txt but I am not able to fetch it via REPL. Is there any ways to achieve these?
• Can I create a lib folder (to upload python ML libraries) in SD card and use that instead of the lib folder that we upload to flash? Just achieving this might be adequate in my case.
  Note: The ML method is Isolation Forest. You are not actually training anything, so the TinyML is out of question, hence I just need more space to implement that.

Cheers,

[livius]

@hakan-kayan
Yes, you can import modules from sd card.

first mount sd card

from machine import SD
import os
import sys
sd = SD()
os.mount(sd, '/sd')

now you must add sd card to search path

sys.path.append('/sd')

and you are ready to import any module from it e.g.: your module is placed in sd card and its name is mysdcardfile.py then you can import it like this

import mysdcardfile

[hakan kayan]

Hi @livius. Thanks for quick answer.
So I used the example given in Pycom documents and created text.txt file in the sd card.

from machine import SD
import os
import sys

#sd = SD()
#os.mount(sd, '/sd')
# check the content
os.listdir('/sd')

I commented os.mount() because it is already mounted. When I run this main.py file via REPL, it does not show anything. However running os.listdir('/sd') from REPL console it outputs 'test.txt'.

The firmware version is 1.20.2.r4. The SD card is 64 GB, FAT32. The document says it supports up to 32 GB, that might be the issue?

Also regarding your answer, my guess is I should manually upload those libraries to SD card. There is no way to install via REPL.

[hakan kayan]

@hakan-kayan

Update: I have tried with 32 GB SD card, the issue persists. The commands from main.py is ignored, but via REPL, they are working.

[Matthew Felgate 0]

@hakan-kayan
Hi Hakan

In code, for example main.py you would have to wrap it in a print statement:
print(os.listdir('/sd'))

Let me know if this works.

Writing files to SD card you can do manually by putting the SD card in your computer.

You could upload files to the board, then use REPL to move them to the SD card with the rename command:
os.rename('test.py','sd/test.py')

Matt

[hakan kayan]

@Matthew-Felgate-0
Thanks Mathew, I don't know how I missed print(os.listdir('/sd'))to be honest. I need some fresh air I guess.

Regarding implementing Isolation Forest, for now it does not seem possible, as the library itself uses huge modules like Numpy. The discussion here also claims that is not possible import libraries like Numpy.

I will be looking for a workaround, even though I am not very hopeful.

[robert-hh]

@Matthew-Felgate-0 said in Using SD Card for the Whole Project.:

os.rename('test.py','sd/test.py')

Sorry no, that is not possible. You cannot move files between volumes by renaming. You have to copy them.

[hakan kayan]

@robert-hh

Hi Robert, could you provide an example please?

[robert-hh]

@hakan-kayan An example of a copy function:

import os

def cp(s, t):
    try: 
        if os.stat(t)[0] & 0x4000:  # is directory
            t = t.rstrip("/") + "/" + s
    except OSError:
        pass
    with open(s, "rb") as s:
        with open(t, "wb") as t:
            while True:
                l = s.read(512)
                if not l: break
                t.write(l)

So can call that with:

from cp import cp
cp("source_name", "target_name")

if target_name is a directory, the target name will be create as target_name/source_name. So you could use:
cp("source_file", "/sd")
If you want to copy more than one file at once, you have to create a loop.