New deployment options

If you have been looking into building your own firmware packages, you've probably come across the script file by now:

[ ["e", "0x1000", "0x40000"], ["e", "0x41000", "0x40000"], ["e", "0x81000", "0x40000"], ["e", "0xC1000", "0x40000"], ["e", "0x101000", "0x40000"], ["e", "0x141000", "0x40000"], ["e", "0x181000", "0x40000"], ["e", "0x1C1000", "0x40000"], ["w", "0x1000", "bootloader.bin"], ["w", "0x8000", "partitions.bin"], ["w", "0x10000", "wipy.bin"] ]

This small json file tells the firmware updater where to write each of the 3 firmware files embedded in the package. While this lets you erase and write any memory region, it is very powerful but not very user friendly, as you need to know the offset for each flash location. So I decided to embed the information from the partition table into the updater so that users can use easy to identify names rather than hex offsets and byte lengths.

These are the partitions defined in the updater:

'secureboot' : ["0x0", "0x8000"] 'bootloader' : ["0x1000", "0x7000"] 'partitions' : ["0x8000", "0x1000"] 'nvs' : ["0x9000", "0x7000"] 'factory' : ["0x10000", "0x180000"] 'otadata' : ["0x190000", "0x1000"] 'ota_0' : ["0x1a0000", "0x180000"] 'fs' : ["0x380000", "0x7F000"] 'config' : ["0x3FF000", "0x1000"] 'fs1' : ["0x400000", "0x400000"] 'all' : ["0x0"]

For the last partition "all" the size is determined based on the flash size.

For additional convenience, the abbreviation ota can be used instead of otadata and cb can be used instead of config as this naming convention is also used in other commands.

Now you can also use these predefined partition definitions when writing the script file for your firmware package. As the size is known for each partition, you do not need to specify it when using a partition in the script file. If you want to force the erasure of the nvs partition, the otadata partition and the file system, your script file could look like this:

[["e", "nvs"], ["e", "ota"], ["e", "fs"]]

For the write ["w"] command, you can now also use the partition labels. Because the size of the partition is known in this case, the image to be written is either truncated to expanded with 0xFF to fit into the available space. If you write the bootloader, partition table or firmware image, you no longer need to erase these regions first before writing to them.

A note of caution here that the updater will try its best to ensure the configuration partition (also known as config block) is not written. This is because this partition contains configuration settings such as the lora mac, Sigfox credentials and other valuable details.

If you want to bypass these measures, such as when restoring a device backup, you can use the command "o" (as in overwrite) instead.

For example: You made a full backup of your device with the command pycom-fwtool-cli copy -p all (assuming you had ESPPORT set to the correct COM port), you will end up with a file like 240AC4027C18-all.bin

In order to "restore" the device via the firmware updater after packing it into an archive with tar -cvzf restore_240AC4027C18.tar.gz 240AC4027C18-all.bin script and flashing it with pycom-fwtool-cli flash -t restore_240AC4027C18.tar.gz

If your file script would have:
[["w", "all", "240AC4027C18-all.bin"]]
It would not overwrite the config partition and it would truncate the image if necessary to fit the available flash size.

If you want to make sure that everything is overwritten, use the following instead:

[["o", "all", "240AC4027C18-all.bin"]]

Beware though that if your binary is bigger than the available flash size, you will get an exception.

A similar thing is true for the file system partition. There are actually two different partitions, fs and fs1. In our 4MB devices, the file system is at offset 0x380000 and 508KB big. In our 8MB devices the file system starts at 0x400000 and has 4MB. The reason that there is not a single file system partition that is dependant on the flash size similar to the "all" partition is so that you can create firmware packages that include the fs for devices with different file system sizes, such as WiPy 2 & 3 or LoPy1 and L01.

If you two file system images fs.bin and fs1.bin, for the two different file system regions, you can specify them in the script file as follows:

[["w", "fs", "fs.bin"], ["w", "fs1", "fs1.bin"]]

On a 4MB device, only the file system fs.bin is flashed, while on an 8MB device, both fs.bin and fs1.bin are flashed. If you target only 8MB devices such as the GPy, FiPy or LoPy4, you can omit the fs partition.

Finally, when flashing firmware images, the new partition table becomes even more beneficial. As the firmware binaries are smaller than their respective partitions, we first erase pretty much everything but the fs, nvs and config partitions. However, as the new "w" command automatically appends 0xFF to fill up the partition which is the same value that is written during erase, the erase is no longer needed. Additionally, the firmware updater is now able to write to either the factory or ota_0 partition, and if a firmware image is written to ota_0, the otadata partition is updated in order to activate this partition. If a firmware is only written into the factory partition, and the otadata partition has not been otherwise erased or written via the script, it is updated to boot from the factory partition. This allows you to distribute two firmwares in the same file, one written into the ota_0 partition and activated by default and the other when using the long safeboot option. We might consider adding a switch in future firmwares to toggle between factory and ota_0 firmware, which makes testing new features a lot less painful as you don't need to downgrade if things go wrong.

A final example putting together all of the above:

[["e", "nvs"], ["w", "bootloader", "bootloader.bin"], ["w", "partitions", "partitions.bin"], ["w", "factory", "wipy.bin"], ["w", "ota_0", "wipy-test.bin"], ["w", "fs", "fs.bin"], ["w", "fs1", "fs1.bin"]]

While I have tested all of these examples many times, please be aware that this is still a development / beta version and it is possible that there are some odd combinations that I haven't thought about that will throw an error or simply don't flash the device properly.

So please make sure to give this a good testing and report any issues you may encounter so we can fix them as quickly as possible in order to release this new updater as a stable version.