OTA or not OTA that is the question...

[jmarcelino]

@Innocenzo
a FTP client feels a bit wasteful on resources, it's a very verbose protocol with lots of old baggage. Is there a particular reason for it?

[Innocenzo]

@robert-hh There is some guide where I can learn about using pre-compiled files or froze bytecode?

@jmarcelino I want perform the FileSystem Update through micropython code. In other words I want that a function in my code can connect to a Server, download a new version of boot.py and main.py and reset the machine. With previous answers I understand that the only way to do this is through FTP.

[robert-hh]

@Innocenzo Besides that, you can always implement a simple file transfer using sockets with a few dozen lines of code.
open connection to sever
request: name_of_file
response: file_or_sorry
get data until the server closes the connection.

[robert-hh]

@Innocenzo Both mpy-cross and dfrozen bytecode require the buidl environment for WiPy (https://github.com/pycom/pycom-micropython-sigfox). I can make a mpy-cross (the pyhton cross-compiler) for you, if you tell me your OS. Then you would compile you python code on your PC, get a file with the extension .mpy and store it on the wipy, like a python script. Th ecompield file may not be much smaller, but it needs less space when loaded and does not need RAM during loading.

[jmarcelino]

@Innocenzo
It's much better if you run use a FTP client on your Server to connect to the existing FTP server running on the LoPy. (confusing sorry :) The FTP server on the LoPy is more efficient and already done in C.

Alternatively use the light weight sockets approach robert-hh suggested.

[Innocenzo]

@robert-hh Actually I use Windows 10 (if there are problems I have also a VM with Ubuntu). Thx

@jmarcelino For my purposes the Wipy must require the transfer and contact the server. I think that the socket approach is the better solution.

[robert-hh]

@Innocenzo OK. It'll take a few housr before I'm home to prepare the binaries. Linux is easy, for Windows10 I can use my Windows VM.

[robert-hh]

@Innocenzo I have placed Linux and Windows versions here:
https://github.com/robert-hh/Shared-Stuff
The versions for the pycom and micropython.org variants differ in the tag of the generated file, so take the right one.
mpy-cross --help
tells you the usage. On WiPy, the .py files take precedence over the .mpy files. If you want to run the .mpy variant, delete or rename the other version first.

[Innocenzo]

@robert-hh Thx a lot.

[robert-hh]

@Innocenzo I just gave ftplib a try.

• the uncompressed python script does not load
• the cross-compiled python file loads and works well
• a minified python script loads too and works well.

Since the minified version is not really readable too and requires an PC pre-process step, the compiled version is preferred. You could also compile the minified version, which gives the smallest footprint (about 11k). (get minify with pip install minify)
I would still try to use my own little loader, since that gives you full control. The only disadvantage is, that you have to implement both client & server.

[BetterAuto]

I'm trying OTA over FTP, is this still non-functional? I'm able to upload only about 1/4 of appimg.bin and it dies.

This is what I see on the console when it dies. Here, I am going from 1.4.0.b1 to 1.6.0.b1 on a WiPy 2.0.

Also could someone please upvote me as well? Grin

Guru Meditation Error of type IllegalInstruction occurred on core  0. Exception was unhandled.
Register dump:
PC      : 0x40188c04  PS      : 0x00050930  A0      : 0x80096671  A1      : 0x3fff2730
A2      : 0x3ffced9c  A3      : 0x3ffc1dec  A4      : 0x40188603  A5      : 0x00000000
A6      : 0x00000000  A7      : 0x0000012e  A8      : 0x80188600  A9      : 0x3fff2710
A10     : 0x3ffc1dec  A11     : 0x00060923  A12     : 0x00060920  A13     : 0x00000002
A14     : 0x00000fdb  A15     : 0x00060023  SAR     : 0x0000001a  EXCCAUSE: 0x00000000
EXCVADDR: 0x00000000  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0x00000000

Backtrace: 0x40188c04:0x3fff2730 0x40096671:0x3fff2750 0x4009824e:0x3fff2770 0x40099fa2:0x3fff2790

                                                                                                  Rebooting...

[Innocenzo]

@BetterAuto In order to uplod the full appimg.bin you must bulid the environment and edit the bootloader. (Actually the size for the appimg is fixed to 1MB). After that you can upload the entire file appimg.bin using FTP. Anyway, I do some tests and after the upload of the appimg.bin file in the sys folder nothing change. In other words OTA doesn't work also if you upload the full image on sys folder.

[BetterAuto]

Ok but you see the file won't even upload. I'm able to upload only about 1/4 of appimg.bin and it dies. But since you said it's not ready I'll wait.

[jmarcelino]

@BetterAuto
You have to be running an image with the modified boot loader flashed via the normal serial process before you try OTA, otherwise the code will overwrite itself during the FTP and cause a crash.

[BetterAuto]

Could an image be downloaded to the WiPy and copied to an 8266? Then the helper chip could reset the WiPy, hold pin 23, feed the update, and reset it again.

If you capture the bytes from a serial port of a successful flash you could then replay that exact sequence from the helper board and no need to program much logic into it. Simply reset, pin 23, stream the prerecorded flash session, reset.

[robert-hh]

@BetterAuto Besides the fact,that this might only shift the problem to the ESP8266, you could also try to port the relevant portion of esptool.py to the ESP8266. But you maybe not done until OTA is working again with WiPy and it's cousins.

[BetterAuto]

@robert-hh That's why I discussed capturing the bytes from a serial port of a successful flash. By replaying the exact sequence from the helper board there's no need to program much logic into it. No need to port esptool.py. Simply reset, pin 23, stream the prerecorded flash session, reset.

[robert-hh]

@BetterAuto If you look at the flash communication, it's not as simply as just dumping a stream of data to the xxPy. Flashing consists of two parts, the stub loader and the actual image, and then the data is transferred in blocks of 16 kByte each, with a (simple) protocol.
Esptool.py by itself is most likely too big to be ported to an ESP8266, but it could be split in an preparation phase running on the PC, which creates the binary data stream to be sent to the ESP32, which is then transferred to the ESP3266, and a smaller loader, which just takes this prepared image and sends it over to the ESP32. The first part might already be implemented in esptool.py, there is an option called write_image, for the second the protocol can be extracted from esptool.py. That seems doable.
Besides that, you have to organize on how the image is retrieved by the ESP8266. That device has to be managed too, like switching it on only when needed, etc.
But still, I do not think it's worth the effort, since OTA updates are still on Pycom's to-do-list.

[BetterAuto]

@robert-hh said in OTA or not OTA that is the question...:

If you look at the flash communication, it's not as simply as just dumping a stream of data to the xxPy. Flashing consists of two parts, the stub loader and the actual image, and then the data is transferred in blocks of 16 kByte each, with a (simple) protocol.

Understood, but if that entire session of two parts is captured raw the sending device should not need much if any logic. If you capture the raw stream coming from the TX pin (maybe via an intermediary device) it should work fine. Thus the helper chip should not need to understand what it is sending, it should be able to just send exactly the same stream of bytes that the original, with esptool.py, had sent.

Besides that, you have to organize on how the image is retrieved by the ESP8266. That device has to be managed too, like switching it on only when needed, etc.

The device has an enable pin which must be held high to operate. You would have to put a small amount of logic to tell it when to start and what file to send but that should be all.

I do not think it's worth the effort, since OTA updates are still on Pycom's to-do-list.

It depends if their timeline is three weeks, three months, or three years. I'm making a product for market right now and can't wait three years.

Do you know their timeline?

[jmarcelino]

@BetterAuto
At the Eindhoven hackaton yesterday Daniel said they were still waiting for the bootloader section - which can't be updated via OTA - to stabilise before enabling over the air upgrades, he expected it to be possible in 3-4 weeks.