/lib/ folder in /frozen/base
We're working on getting SecureBoot and Encryption working and we have come quite long so far - everything compiles nicely except one core part of our code - the libraries. The internal libraries we have are put in a directory relative to flash,
/flash/lib. They are included in the build under
...esp32/frozen/Base. I'm wondering if this is the right way to put it though, as it doesn't seem to work. Running the compiled code on the device, we get "Module not found" errors.
See the REPL output when connected to an encrypted device:
>>> os.listdir("/flash/lib/")  >>>
This makes me think that there could be some kind of conflict. The building of the firmware indicates that the lib/ folder at least is being built:
MPY frozen/base/lib/filename1.py MPY frozen/base/lib/dirname1/filename2.py
I see there are multiple folders where you can put things, but they are not documented more thoroughly. For example, the
Custom/directory's README talks about "custom user scripts", but what does "custom user scripts" even mean? Therefore, I can't determine where I should put the
lib/folder or what I'm doing wrong.
I'm more involved with the techincal parts so I can fill in on some technical details for @Don-iot.
We have seen this behavior with a batch using 2 out of 10 boards. What happens is the following:
- A building script downloads and compiles firmware for the devices. (SecureFirmware method is used, final image size is 2022224 bytes)
- The firmware gets flashed onto devices.
- On (in our tests) 8/10 units:
The firmware gets flashed correctly and the device starts with no strange errors on first boot
- On (in our tests) 2/10 units:
The firmware gets flashed, but the device fails to read and write to files (we get ENOENT errors). It also fails to import modules (module name not found errors) that other devices can import just fine.
If you re-run the same script, with the exact same code being flashed enough times (in one case 4 flashes, in another case 8), a device that doesn't work (see 4.) boots up just fine with no errors at all.
This makes us think this is a PyCom problem, since the hardware side and setup has been consistent (Don wrote more about it above).
The flashing tool nor building tool reports no errors.
Don-iot last edited by
@robert-hh Working on the same project as @techykermit we have run into problems. We can encrypt devices but in the last batch we had 3 out of 10 devices that were not sucessful, using the same code in all devices but with different encryption keys. Some of these devices did start up after recurring uploads of the code. As we use the same code, it seems like the GPys are not happy, nor sucessful when doing this repeatedly across devices. What kind of work flow would you recommend to get a more stable solution? Currently we have developed our own system, and we are very careful using a good power supply, and a FTDI TTL adapter. The GPys are on our boards, soldered by professionals, on a CE-certified board. So when it comes to hardware and being consistent, I feel that we are in the right ballpark. Yet these 3-4-5 uploads before everything is working is just not going to scale. So any thoughts and ideas are welcome!
@robert-hh Okay! Thanks a lot for that clarification and a fast answer.
robert-hh last edited by
@techykermit Actually there is no fix for that. The set of frozen files is not contained in a file-system-like structure. So you have to name the lib files in a appropriate manner.
@robert-hh Hi, sorry for a little late response. I had to get hold of the test unit :)
Thanks for the command suggestion! Running it, I see that my libraries all show up, but they do show up in the list with
lib/in front of their import name. See below, where
lib3are the internal libraries that have been frozen:
lib/lib1 select uqueue hashlib lib/lib2 socket ure json lib/lib3
All our previous code relies on not using
libin front of the library when importing it (for example,
lib2is imported with the line
import lib2instead of
from lib import lib2in our source code. While the line
import lib.lib2works (so thanks a lot for pointing me to this function and figuring out that the libraries are there!), is there anything we can do to not have to change all our import statements to import from
lib/(is there anything to do so we can import the libraries from
lib/using a prefix like
robert-hh last edited by
@techykermit You can check with help("modules"), whether and under which name the frozen modules were included.
@jcaron I try to load them as you can do "normally" (when not building your custom firmware):
for example we do:
when we want to import the
logmodule that is under lib.
This gives a
Traceback (most recent call last): File "main.py", line 1, in <module> ModuleNotFoundError: No module named 'log'
@techykermit frozen files are not present as actual files in the filesystem, so it’s normal you can’t list them. They should be available for import directly.
How do you import them and what error do you get?
Don-iot last edited by