OTA or not OTA that is the question...

[robert-hh]

@Innocenzo As far as I understand the ftp.c and ftpupdater.c code, the file appimg.bin, when received by ftp, is not written to the file system, but treated separately and written to a dedicated buffer. That's why it does not show up in the file system. Ftp looks out for that special file name "/flash/sys/appimg.bin".
The file system size is 508 k and is way to small to hold the image file.

[BetterAuto]

@robert-hh But @Innocenzo said that the procedure completes without problems when he uploads using FileZilla FTP.

@Innocenzo perhaps do a capture of the data being passed during the working upload to see what may need to be added to your filesystem code.

Working back to my thoughts on a small daughter device, perhaps an ESP8266 can FTP the code over. I had ignored that method since I couldn't get it to work with FileZilla myself.

[BetterAuto]

@Innocenzo when you say the procedure completes without problems, are you saying it actually positively flashes to the newer version? Try going from a version behind to current using FileZilla.

[robert-hh]

@BetterAuto Using his own code, @Innocenzo could not transfer appimg.bin to /flash/sys. using the built-in ftp, he can transfer the file, because it is NOT stored in the file system, but in a separate area in flash, which is large enough to hold the image. The size is 1538 kByte, which changed recently from about 1000 kByte.
For Wipy1 ftp upload is the preferred method of updating. With WiPy 2 and Lopy you could update the application image using ftp, until the file size grew beyond the 1MByte size. I see the growth of the dedicated storage area for the image as an indication of work in progress for OTA. Trying to update my Lopy this way failed. I could upload appimg.bin, but the device did not start the, but only showed the "red RGB led" state.n I had to erase & reflash to make it working again.
Once ftp update works, it can be done directly to the final device. No daughter device needed in between, unless you want to add security, which is not given at the moment.

[Innocenzo]

@robert-hh Ok with firmware 1.7.5.b1 does not work! red RGB led also for me. With previous firmware version, using FileZilla FTP, I was able to flashing my custom firmware!

[robert-hh]

@Innocenzo When you say "previous", which version was it?

[Innocenzo]

@robert-hh I am sorry, but I do not remember precisely. Or the first release of 1.7 or the the latest 1.6 ( I had to change the file bootloader.h for IMG_SIZE field).

[jmarcelino]

@robert-hh
Back in April I flashed a 1.6.9 custom firmware OTA with a the simple partition size patch - change #define IMG_SIZE (1536 * 1024) in the bootloader.h. It was definitely running my OTA image as I changed the version string and added an extra function.

Haven't really tried it since then. I think the code is already all there but as it was suggested maybe the bootloader isn't stable, the partitioning is still in flux or maybe it'll change to be connected in some way to the Pybytes platform.

[robert-hh]

@jmarcelino At the moment it is defined as:
#define IMG_SIZE ((1024 + 512) * 1024)
So I'm not sure what#s going on. I tried to upload an self-compiled appimg.bin. I could try to extract that from the 'official' image and try again.

[BetterAuto]

If the issue is only one of storage space, perhaps a smallish external SPI flash. Cheap.
https://www.digikey.com/product-detail/en/winbond-electronics/W25Q64JVSFIQ/W25Q64JVSFIQ-ND/5803990

[jmarcelino]

@BetterAuto
There is enough storage space, all the kits have 4MB Flash. The partitioning setup allows for two 1536K application images (factory + OTA) and a 508K user area (what you see in as /flash)

[robert-hh]

@BetterAuto At the moment it looks like only a somewhat small piece of software bug has to be changed. Maybe the same that causes the pre-build imagas and self-built images to be incompatible = using a different partitioning of the memory.

[BetterAuto]

I just found this post from back in April with a method from @jmarcelino which may be useful for us. I haven't tried it yet.
https://github.com/pycom/pycom-micropython-sigfox/issues/5

[robert-hh]

@BetterAuto The carnage @jmarcelino ist set by default now. You can do an OTA upload this way, but then the device either does not start any more or the uploaded image is ignored, depending on which state the device was before. There is a discrepancy between the partition as defined in lib/partitions.csv, compiled to partitions.bin, and the address definitions of bootloader.h. According to partitions.bin, the addresses where the image is stored with OTA is 0x10000 or 0x1a0000, according to bootloader.h it's 0x10000 or 0x190000. And some of the flags that are used to control the process are taken from the sixth partition called dummy, but the 4th partition is called ota_data, and may be the right one. The layout of memory and its use is not clear to me, and instead of me guessing, someone from pycom should fix that in the intended manner.
@daniel, what's your opinion?

[Paco88]

Any news about the OTA features?

[robert-hh]

@Paco88 OTA works. I use it for a lolin32 device where the USB/UART chip is not working reliably.

[jcaron]

@Paco88 I can confirm OTA works, at least if the version you're upgrading from is relatively recent. I think something like 1.6.whatever is required.

You'll need to build the image yourself though.

[BetterAuto]

So could someone please share high level details about the procedure?

[jcaron]

@BetterAuto The very high-level procedure is:

1. Make sure the board already has a firmware version that is recent enough to support OTA (I believe it needs to be >=1.7.7b1)
2. Follow the instructions here: https://github.com/pycom/pycom-micropython-sigfox to build the new firmware. IIRC I had to download and install a slightly different version of the Xtensa toolchain than the one linked to there, but YMMV.
3. Upload the file pycom-micropython-sigfox/esp32/build/LOPY_868/release/appimg.bin to /flash/sys/ on your module via FTP.
4. Reset

[robert-hh]

@jcaron Thank @jcaron for the explanation. A few notes to be added:

1. Once you have set-up your own build environment, you can also put you own Python scripts into frozen bytecode, just by placing them into the "frozen" subdirectory before running make. That save RAM and speeds up loading of modules substantially, since the code is executed from flash and no RAM is needed for parsing & compiling.
2. You can also download the image by USB using the command "make flash", or erase flash with "make erase". The latter is required, if you move from the pycom downloader to the your own images. If the LOAR mac is lost, today a fix was made public here:https://forum.pycom.io/topic/1272/lora-mac-ffffffffffffffff/21
3. An observation that may have been fixed meanwhile: If you mix the use of USB downloads and OTA downloads, you might see that a newly loaded image is not used. The reason: there are two reserved areas for the firmware image. OTA uses them alternatively, while the USB loader uses always the same area.