OTA or not OTA that is the question...



  • After reading documentation and forum threads and done some trying I am not able to say if WiPy is (will be) able to manage full OTA for firmware update and for user program.
    I am looking for same easy to use OTA functions I found with Particle but without dependencies to internet connection.
    For OTA firmware update it seams to be not possible for now despite the fact that it was documented. I tried without success... Will it be possible one day or not ?
    For OTA user program update it should be necessary to have a hardware reset (cf. If your board is running code at boot time, you might need to boot it in safe mode.). So no really OTA facility.
    Could we have clear answers on these two points as they are main reasons to use or not Py devices.
    Thanks



  • @BetterAuto The carnage @jmarcelino ist set by default now. You can do an OTA upload this way, but then the device either does not start any more or the uploaded image is ignored, depending on which state the device was before. There is a discrepancy between the partition as defined in lib/partitions.csv, compiled to partitions.bin, and the address definitions of bootloader.h. According to partitions.bin, the addresses where the image is stored with OTA is 0x10000 or 0x1a0000, according to bootloader.h it's 0x10000 or 0x190000. And some of the flags that are used to control the process are taken from the sixth partition called dummy, but the 4th partition is called ota_data, and may be the right one. The layout of memory and its use is not clear to me, and instead of me guessing, someone from pycom should fix that in the intended manner.
    @daniel, what's your opinion?


  • Pybytes Beta

    I just found this post from back in April with a method from @jmarcelino which may be useful for us. I haven't tried it yet.
    https://github.com/pycom/pycom-micropython-sigfox/issues/5



  • @BetterAuto At the moment it looks like only a somewhat small piece of software bug has to be changed. Maybe the same that causes the pre-build imagas and self-built images to be incompatible = using a different partitioning of the memory.



  • @BetterAuto
    There is enough storage space, all the kits have 4MB Flash. The partitioning setup allows for two 1536K application images (factory + OTA) and a 508K user area (what you see in as /flash)


  • Pybytes Beta

    If the issue is only one of storage space, perhaps a smallish external SPI flash. Cheap.
    https://www.digikey.com/product-detail/en/winbond-electronics/W25Q64JVSFIQ/W25Q64JVSFIQ-ND/5803990



  • @jmarcelino At the moment it is defined as:
    #define IMG_SIZE ((1024 + 512) * 1024)
    So I'm not sure what#s going on. I tried to upload an self-compiled appimg.bin. I could try to extract that from the 'official' image and try again.



  • @robert-hh
    Back in April I flashed a 1.6.9 custom firmware OTA with a the simple partition size patch - change #define IMG_SIZE (1536 * 1024) in the bootloader.h. It was definitely running my OTA image as I changed the version string and added an extra function.

    Haven't really tried it since then. I think the code is already all there but as it was suggested maybe the bootloader isn't stable, the partitioning is still in flux or maybe it'll change to be connected in some way to the Pybytes platform.



  • @robert-hh I am sorry, but I do not remember precisely. Or the first release of 1.7 or the the latest 1.6 ( I had to change the file bootloader.h for IMG_SIZE field).



  • @Innocenzo When you say "previous", which version was it?



  • @robert-hh Ok with firmware 1.7.5.b1 does not work! red RGB led also for me. With previous firmware version, using FileZilla FTP, I was able to flashing my custom firmware!



  • @BetterAuto Using his own code, @Innocenzo could not transfer appimg.bin to /flash/sys. using the built-in ftp, he can transfer the file, because it is NOT stored in the file system, but in a separate area in flash, which is large enough to hold the image. The size is 1538 kByte, which changed recently from about 1000 kByte.
    For Wipy1 ftp upload is the preferred method of updating. With WiPy 2 and Lopy you could update the application image using ftp, until the file size grew beyond the 1MByte size. I see the growth of the dedicated storage area for the image as an indication of work in progress for OTA. Trying to update my Lopy this way failed. I could upload appimg.bin, but the device did not start the, but only showed the "red RGB led" state.n I had to erase & reflash to make it working again.
    Once ftp update works, it can be done directly to the final device. No daughter device needed in between, unless you want to add security, which is not given at the moment.


  • Pybytes Beta

    @Innocenzo when you say the procedure completes without problems, are you saying it actually positively flashes to the newer version? Try going from a version behind to current using FileZilla.


  • Pybytes Beta

    @robert-hh But @Innocenzo said that the procedure completes without problems when he uploads using FileZilla FTP.

    @Innocenzo perhaps do a capture of the data being passed during the working upload to see what may need to be added to your filesystem code.

    Working back to my thoughts on a small daughter device, perhaps an ESP8266 can FTP the code over. I had ignored that method since I couldn't get it to work with FileZilla myself.



  • @Innocenzo As far as I understand the ftp.c and ftpupdater.c code, the file appimg.bin, when received by ftp, is not written to the file system, but treated separately and written to a dedicated buffer. That's why it does not show up in the file system. Ftp looks out for that special file name "/flash/sys/appimg.bin".
    The file system size is 508 k and is way to small to hold the image file.



  • The stranger thing is that if I upload my appimg.bin using FileZilla FTP , the procedure complete without problems. But if I use a code like this:

    import socket 
    import uos
    import machine
    from network import WLAN
    import utime
    import gc
    from machine import SD
    
    wlan = WLAN(mode=WLAN.STA)
    wlan.connect('TP-LINK_2.4GHz_E113DF', auth=(WLAN.WPA2,"***********"))
    while not wlan.isconnected():
        machine.idle()
    
    #sd = SD()
    #uos.mount(sd, '/sd')
    
    print("Mem_alloc: {}".format(gc.mem_alloc()))
    print("Mem_free: {}".format(gc.mem_free()))
    uos.chdir("sys")
    s = socket.socket(socket.AF_INET,socket.SOCK_STREAM, socket.IPPROTO_TCP)
    addr = socket.getaddrinfo("**************", 80)[0][-1]
    print(addr)
    s.connect(addr)
    print("connected")
    s.write(b"GET /*************/appimg.bin\r\n")
    s.write(b"\r\n")
    #f = open("/sd/appimg.bin","wb")
    f = open("appimg.bin","w")
    while 1:
        req = s.recv(8192)
        if not req or req == b"\r\n" or req == b'':
            break
        f.write(req)
        gc.collect()
        
    f.close()
    print("DONE")
    

    or like this:

    import socket
    import gc
    import uos
    import machine
    from network import WLAN
    import utime
    import sys
    
    wlan = WLAN(mode=WLAN.STA)
    wlan.connect('TP-LINK_2.4GHz_E113DF', auth=(WLAN.WPA2,"******"))
    while not wlan.isconnected():
        machine.idle()
    
    def getPort(req):
        req = req.split('(')
        print(req)
        req = req[1].split(')')
        print(req)
        req = req[0].split(',')
        port = (int(req[4]) << 8) + int(req[5]) 
        return port
    
    uos.chdir("sys")
    s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
    s.connect(socket.getaddrinfo("*********", 21)[0][-1])
    s.sendall(b'USER *******\r\n')
    req = str(s.recv(8192))
    print(req)
    s.sendall(b'PASS *********\r\n"')
    req = str(s.recv(8192))
    print(req)
    s.sendall(b'PASV\r\n')
    req = str(s.recv(8192))
    print(req)
    s.sendall(b'TYPE I\r\n')
    req = str(s.recv(8192),'utf-8')
    print(req)
    port = getPort(req)
    p = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
    p.connect(socket.getaddrinfo("************", port)[0][-1])
    print("Connected!")
    f = open("appimg.bin",'wb')
    s.sendall(b'RETR appimg.bin\r\n')
    req = str(s.recv(8192))
    print(req)
    print("Start transfer")
    
    while 1:
        req = p.recv(8192)
        if not req or req == b'':
            break
        try:    
            f.write(req)
        except:
            print("NO!")
        gc.collect()
        utime.sleep(0.1)
    f.close()
    print("DONE")
    

    I have an exception on the f.write(req) and I can transfer only 497.664 bytes of the appimg.bin.


  • Pybytes Beta

    I see, thanks. That's the kind of information I was looking for. Since we're so close to a solution I'll wait. I'm producing a product for sale and if a few of the first models cannot OTA it'll have to suffice.

    I'm building OTA for the scripts in at least, which is where I think most potential vulnerabilities would lie anyway.

    The solution for OTA for the scripts is I am downloading any updated scripts as something.py.new and will then reset the device. I will have code in boot.py to look for *.new, if found it will delete any existing something.py and move something.py.new to something.py. Then proceed with the rest of the bootup.

    One could even extend this to back up existing files and provide a fallback option via GPIO but I've decided not to proceed with that.



  • @BetterAuto The protocol is a9 variable in timing and it b) may vary in response, if it allows for a "transmission failed, please repeat las message" response. For a) a worst case timin could be used, but b) cannot be prerecorded.


  • Pybytes Beta

    My kids used to like to watch Dora the Explorer. On that show the cartoon character turns to the audience and asks questions and pretends to respond. Even though the show is prerecorded, the illusion of ask/response is sufficient to satisfy many toddlers.

    What does that have to do with OTA?

    The prerecorded flash session need not be actually interactive. Two identical devices would respond in two identical ways even if one of them is only listening to a prerecorded message. I still think my method could work.

    But if the timeline is somewhere between three weeks and year's end then I'll wait.



  • @robert-hh
    Understood :) My feeling, because of the Pybites platform, the FiPy etc is we'll have it within 3 months.


Log in to reply
 

Looks like your connection to Pycom Forum was lost, please wait while we try to reconnect.