Protecting your code

[robert-hh]

@Don-iot If this varying data is to be embedded into the firmware, you have to create an unique firmware for each of them. Since only a few frozen scripts are change, that build is rather fast, consisting of mpy-cross for the changed files, a single c-compile and link phase. The latter is the slowest. Uploading the firmware to the target will take longer than this custom build step, so you could to that on the fly.

[jcaron]

@Don-iot I don't know if you can easily embed random files (i.e. something other than a frozen python file) in a firmware.

You can try to include the data in the firmware:

• Create a python file that contains that data (e.g. as strings), and as @robert-hh says, build one firmware version for each device with the relevant file.
• Or check if there's a way to include other files in the firmware, and that the rest is the same.

However that means that every time you update the firmware, you need to generate a version for each device. An alternative would be that the initial firmware actually write the data to the filesystem on first run.

I think a better alternative is to have a standard firmware, but then provision each device. There are many ways to do that, including:

• Use standard FTP to upload the files after flashing.
• Include a provisioning mode in the firmware, which will run the first time (i.e. when the files are not yet present on the filesystem), and somehow receive the relevant data over the serial port. So you would flash, then wait for the provisioning system to start, and send the data. You of course need some integrity checks, retries, etc.
• You can probably use the same methods Pymakr uses to transfer files over the serial port, but I have no idea if those are documented. IIRC it's just sending Python commands to the REPL or something similar?
• Ditto, but have the initial provisioning code retrieve the device-specific data from a server (e.g. over HTTP). Each device must have a unique identifier (e.g. MAC address) which will have the server return the right data. This of course requires the device to somehow be connected to a network.
• Another option could be to use the esptool to flash a custom filesystem image for each, but that seems like a lot of work when there are probably simpler alternatives.

There are probably tons of combinations and alternatives, they depend a lot on the type of device, the environment, and more, but I hope this gives you a few ideas.

[Don-iot]

@jcaron and @robert-hh thanks a lot for your insights. Great ideas and I think we have a few ideas moving forward!

[livius]

@Don-iot
Be aware of revision of the chip as some are still not safe even when you "turn on protection"

https://www.espressif.com/en/news/Security_Advisory_Concerning_Fault_Injection_and_eFuse_Protections

[Don-iot]

Thanks @livius!

We are succesful in writing to the device and so on, but our main problem after trying with the frozen code is where and how to store config files (device unique) and the log files (changes during operation). They should not be compiled with the other files obviously, but the documentation is struggling a bit on where and how to treat these entities.

[Don-iot]

@jcaron @robert-hh or @Gijs -do you know the answer to this question? Would really appreciate your help. So close to putting our device on the market and this is really the last bit of the puzzle.

[livius]

@Don-iot
It depend of config and logs. What do you need to store?
For me SD card is obvious choice, as it is simplest to prepare configs and send to the user. And logs are near unlimited then ;-)
But you can store it in flash too, in nvram.

[Don-iot]

@livius We are storing MQTT payloads between uploads and TLS-certificates for example. We are using our own hardware and board, and it is used on vibrating machines so I rather not use the SD card, even if it is handy. Any more details on storing it in the NVRAM even though the rest of the device is using frozen code?

[kjm]

@Don-iot You can store strings or integers with something like pycom.nvs_set('mqqt_payload', payload). Where the first item in brackets is the register name and the second is the string or integer you wish to store. For bigger data use a typical python file write (eg with open('mqqt_payload', 'w') as f: f.write(payload)

If you're communication with a server you can store config files on the server & download them periodically. This approach has the advantage that you can rejig the config on the server (independent of the pycom module) via a browser.

[Don-iot]

@kjm said in Protecting your code:

For bigger data use a typical python file write (eg with open('mqqt_payload', 'w') as f: f.write(payload)

That is equal to our current approach, however, we do not seem to be able to create new files when we run a device with frozen code, we get OS-related errors. The file are being put at the same location that we are using without any problems on devices running non-frozen code. The files are put in a subpath to /flash/, a directory we call "storage" (so for example a file could be /flash/storage/logs/log.txt). The PyCom documentation is not really clear about where we should put files that we are modifying during operation and that are individual to the device (stored MQTT payloads, device cert files, etc.). So, the question is, essentially, where we should put them so we can proper access them?

[kjm]

@Don-iot beyond my pay grade Don, Robert-hh/jcaron/et al might know?

[Don-iot]

@kjm thanks, let us hope that @robert-hh or @jcaron can help us out.

[robert-hh]

@Don-iot When opening the files for writing & reading, do you specify the full path or do you rely on sys.path settings or the current directory path? I never have seen any differences in file path handling between frozen code and code imported from the file system.

[jcaron]

@Don-iot what error are you getting? Adding frozen code should not change either the structure of the file system or write access to it.

What is possible is that while building your firmware you somehow got different settings configured, but this is most likely not directly related to the frozen code.

Another possibility is that while rebuilding you changed the layout of the flash (position and size of the partitions) to get more space, and the space for the file system no longer points to a valid file system. You would need to initialise it first.

[Don-iot]

@robert-hh We specify the full path like this when we want to access a file called "payload.json" in the "storage" directory:
/flash/storage/payload.json

This is when we call open(), so for example open("/flash/storage/payload.json", "r"). We get an ENOENT error.

[Don-iot]

@jcaron Thanks for clarification, we will have another go!

[kjm]

@Don-iot At the risk of stating the brutally obvious Don, you'll get an ENOENT error for open("/flash/storage/payload.json", "r") if the file doesn't exist, ie you need to have previously created it with something like f=open('/flash/storage/payload.json', 'w'); f.close()

[jcaron]

@kjm and of course the “storage” directory needs to exist beforehand.

[Don-iot]

@jcaron @kjm But it should be ok if the file is present in the build, right?

[jcaron]

@Don-iot I must have a brain freeze, I don’t remember a standard way to include custom files in a build, other than frozen code. Where and how do you include those files? Are you sure they end up where you think? You should probably list files at various levels of your fs hierarchy to check that the files are where you think they are (if they are there at all).

Note that even if there is or was a way to include those files at build time, you would probably need to recreate the fs for those files to appear.