How to properly build custom FW image (tar.gz)

[danielm]

I would like to distribute single *.tar.gz file to my customers which will contain specific version of Pycom MicroPython together with frozen application. They will flash their devices using updater.py/esptool or I will make some GUI install wizard for them later.

Now the question is what is the right way how to produce .tar.gz files?

I see that they contain files „bootloader.bin“, „lopy_868.bin“ which can be compiled from source.

Then there is file „partitions.bin“ and two script files which I don’t know how to generate. I could probably take .tar.gz file built by Pycom and replace lopy_868.bin but is this the right way how to make my own .tar.gz file?

[Xykon]

@danielm The way we build the release files is as follows (this assumes the LoPy 868MHz firmware)

cd build/LOPY_868/release/
mkdir -p firmware_package
cd firmware_package/
cp ../bootloader/bootloader.bin .
cp ../appimg.bin .
cp ../lib/partitions.bin .
cp ../../../../boards/LOPY/LOPY_868/script .
cp ../lopy_868.bin .
tar -cvzf LoPy_868-custom.tar.gz appimg.bin bootloader.bin partitions.bin script lopy_868.bin

As for flashing the firmware, you can use our new binary updater tool which is currently being beta-tested for Windows & MacOS (a Linux version will be released shortly). The latest links to the cli version can be found in this forum thread https://forum.pycom.io/topic/2351/beta-new-updater-tool-version-v1-1-5-b3

To use the tool simply execute the CLI version with:

For traditional expansion board with firmware wire:
pycom_fwtool_cli -p <serial port> -s 921600 -t <path_to_tar.gy> flash

For Pytrack / Pysense:

pycom_fwtool_cli -p <serial port> -s 921600 -t <path_to_tar.gy> --pypic flash

I am pretty sure I can now reliably detect the Pytrack & Pysense to no longer need the --pypic option so it shouldn't be required from one of the next releases.

I am also considering implementing custom firmware flashing in the GUI version.

[danielm]

@Xykon
Thanks a lot!
I support your idea of GUI updater to be able to flash custom FW. You would make life of my customers easier :)

[Xykon]

@danielm Just a heads-up I made a small edit as my copy & paste seems to have overlooked the last file in the tar command.

[Xykon]

@danielm Sorry one more edit in the command line for Pytrack / Pysense.

When I wrote the original instructions I didn't have access to a module to verify.

[danielm]

@Xykon
Did you already decide if you will include flashing of custom FW into GUI updater tool?

[Xykon]

@danielm I will most likely do this at some point but I can't give any ETA at this point.

[danielm]

This post is deleted!

[danielm]

@Xykon
Does pycom-fwtool-cli support flash erase?
Any news on GUI tool ability to flash custom tar.gz files? :)

[livius]

@danielm
why you need flash custom tar.gz?
if you have custom firmware do make flash

[danielm]

@livius
Because I want to deliver single file containing both firmware and custom application to my customers. It would be easier for them to use GUI tool to flash it. For now I will ask them to use pycom-fwtool-cli which works well.

[Xykon]

@danielm said in How to properly build custom FW image (tar.gz):

@Xykon
Does pycom-fwtool-cli support flash erase?

It will in the next release. However please be mindful that the flash contains a config block that stores details such as LoRa / Sigfox MAC and Sigfox credentials. These will be restored when using the online firmware flasher as we keep of copy of those details in our database, but if you erase the flash and then install a custom built firmware, these details are gone.
The next cli tool will have functionality to backup and restore the config block as well as nvs and ota partitions.
I am also working on an option to erase just the file system partition (similar to what os.mkfs('/flash') does but for obvious reasons this will need thorough testing before it can be used safely.

Any news on GUI tool ability to flash custom tar.gz files? :)

It's on my to-do list along with many other things... I have some ideas about this but I can't give you any firm timeline when this could be ready.

[this.wiederkehr]

@xykon said in How to properly build custom FW image (tar.gz):

tar -cvzf LoPy_868-custom.tar.gz appimg.bin bootloader.bin partitions.bin script lopy_868.bin

I'm confused. I remember that older fw-images only did contain(e.g. https://software.pycom.io/downloads/LoPy_915-1.7.4.b1.tar.gz):

• bootloader.bin
• partitions.bin
• script
• lopy_868.bin

is the appimg.bin really required in the fw_image?

[seb]

appimg.bin is used for OTA updates

[this.wiederkehr]

@seb thanks for the quick answer. Can you be more specific: What's inside appimg.bin that is not included in lopy_868.bin or vice versa?

[seb]

@this-wiederkehr It includes a hash of the file (at the end I believe) that the bootloader uses to verify if the rest of the firmware image is valid. This is included in the tar.gz just so that customers who wish to update to an official firmware, either via FTP or via the OTA functions we provide, do not have to compile the firmware themselves.

https://github.com/pycom/pycom-micropython-sigfox/blob/5bed5e9a84bb1abfc4d59476f3f62b38920c5eda/esp32/tools/appsign.sh#L25

[this.wiederkehr]

awesome, thanks for the clear answer.

[Mainaleygu11]

This post is deleted!