New firmware release, version 1.6.0.b1

daniel

I still have problem of corruption of files stored throught ftp :(

@livius Really? We have been testing a lot during the last 2 weeks with large files and the issue seem resolved. Does it happen when you perform file operations in your script while at the same time the FTP server is busy with a transfer?

livius

@daniel
it is by copy file throught e.g. filezilla and without while true: loop

daniel

@livius what's the size of the file?

daniel

@livius could it be that your file system is corrupted from previous firmware versions? Could you try:

import os
os.mkfs('/flash')

to format the file system and start from scratch...

Cheers,
Daniel

livius

@daniel
3905 bytes

I will test with 1.6.2 and will see if something changed

could it be that your file system is corrupted from previous firmware versions? Could you try:

I can erase all data - but i can back tomorrow with review

JF002

@daniel With 1.6.1, I encountered some instabilities : the lopy hangs when trying to sync the project from Pymakr, and when I call gc.collect(). Are you aware of these issues? Is it possible that they are fixed in 1.6.2?

Thanks

daniel

@livius can you show me an example of the gc.collect() crash?

JF002

@daniel

import gc
gc.collect()

Just freeze my lopy. I noticed it when trying to fix another issue with ds1820 and mqtt : https://forum.pycom.io/topic/670/onewire-wifi-mqtt-crash/7

daniel

Just freeze my lopy. I noticed it when trying to fix another issue with ds1820 and mqtt :

OK it happens, not always at the first time. I'm working on it... Wait for the 1.6.3.b1 coming a bit later...

JF002

@daniel said in New firmware release, version 1.6.0.b1:

Just freeze my lopy. I noticed it when trying to fix another issue with ds1820 and mqtt :

OK it happens, not always at the first time. I'm working on it... Wait for the 1.6.3.b1 coming a bit later...

Yeah, thank you! You're the best!

daniel

Hello,

1.6.3.b1 is out there. In my tests the gc.collect() problem is now fixed. @livius @JF002 please test and let me know... Thanks!

Cheers,
Daniel

Neo

@daniel Still hanging on LopY. In our script this is happening after wifi STA connect ... we have a gc.mem_free call. We removed all gc collect calls.

daniel

Then it's not the gc anymore in your case. If you can share with me a piece of code that can reproduce it I'll get onto it.

Cheers,
Daniel

Neo

@daniel Unfortunatelly the code is huge. I will try to reduce the area. I chekced also one auto tests only with LoRa initiation and Lora socket reading + gc.mem_free calls ... I saw that this test is hanging. I believe the real problem ties to memory leak /bad management. Here is the memory output at each second called soket recv and dump the mem_free:
Memory 19264
Memory 18656
Memory 18080
Memory 17504
Memory 16928
Memory 16352
Memory 15776
Memory 15200
Memory 14624
Memory 14048
Memory 13472
Memory 12896
Memory 12320
Memory 11744
Memory 11168
Memory 10592
Memory 10016
Memory 9440
Memory 8864
Memory 8288
Memory 7712
Memory 7136
Memory 6560
Memory 5984
Memory 5408
Memory 4832
Memory 4256
Memory 3680

Neo

@daniel Here it is your memory leak script:

def test2():

from network import LoRa
_lora = LoRa(mode=LoRa.LORA)

import socket
_socket = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
_socket.setblocking(False)    

import gc
gc.enable()   
while True:

    try:
        bytes = self._socket.recv(512)
        if len(bytes) > 0:
            print(" %s RECV %s" %  (self, bytes))
        
    except Exception as e:   
            print("_recv", str(e))    
            
    print('Memory %d' % (gc.mem_free()))

    utime.sleep(1)

test2()

daniel

@Neo, I modified your script in order to be able to run it:

from network import LoRa
import utime

class test:
    def __init__ (self):
        self._lora = LoRa(mode=LoRa.LORA)

        import socket
        self._socket = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
        self._socket.setblocking(False)    

    def run(self):
        import gc
        gc.enable()

        while True:
            try:
                bytes = self._socket.recv(512)
                if len(bytes) > 0:
                    print("%s RECV %s" %  (self, bytes))

            except Exception as e:
                print("_recv", str(e))

            print('Memory %d' % gc.mem_free())
            utime.sleep(1)

t = test()
t.run()

I get the following output:

Memory 54880
Memory 54320
Memory 53760
Memory 53200
Memory 52640
Memory 52080
Memory 51520
Memory 50960
Memory 50400
Memory 49840
Memory 49280
Memory 48720
Memory 48160
Memory 47600
Memory 47040
Memory 46480
Memory 45920

Which is totally correct, that's the way Python works. Memory is consumed as code runs, and when there's no more memory, the garbage collector kicks in and frees memory so that code can continue running. At some point you see:

Memory 5040
Memory 4480
Memory 3920
Memory 3360
Memory 2800
Memory 2240
Memory 1680
Memory 1120
Memory 560
Memory 0
Memory 54592
Memory 54032
Memory 53472
Memory 52912

However, if I add a gc.collect() line like this:

from network import LoRa
import utime

class test:
    def __init__ (self):
        self._lora = LoRa(mode=LoRa.LORA)

        import socket
        self._socket = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
        self._socket.setblocking(False)    

    def run(self):
        import gc
        gc.enable()

        while True:
            try:
                bytes = self._socket.recv(512)
                if len(bytes) > 0:
                    print("%s RECV %s" %  (self, bytes))

            except Exception as e:
                print("_recv", str(e))

            print('Memory %d' % gc.mem_free())
            utime.sleep(1)
            gc.collect()

t = test()
t.run()

I get:

Memory 55520
Memory 55664
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648

Which at least in this test it shows that there's no memory leak...

crankshaft

@daniel - can you make a new announcement when a new beta is released, the topic of this current thread is ... version 1.6.0.b1 however I think it is now version 1.6.3.b1 ?

You have to trawl and check every post to see if a new release has been made and it's not very productive.

Or at least update the header

livius

@daniel said in New firmware release, version 1.6.0.b1:

bytes = self._socket.recv(512)

One hint if i can suggest something
what if you send something to this socket and this line recive some data?

bytes = self._socket.recv(512)

is this still the same result?
I can test it self only in the evening (and only on different socket - Wipy2) - but maybe result is different and you can find something interesting
If not then we can only wait for @Neo to provide some sample

@crankshaft good point

Colateral

@daniel It is strange that yesterday using this script the LoPy was hanging ... today I re powered and I didn't reproduce it anymore. It is true that yesterday I did a lot of operation before.

However the entire solution is still hanging and I need to understand why on 1.5.0 this is working and 1.6.x is not working.

Neo

@daniel It is still hanging. There is diff between your output and my output.... and I know why. My script is embedded in the app importing some additional modules.... that why mem is starting from 20K and not 50K as yours. Some of these modules have some static data and some of them have thread locks and Wifi data. Let me check.... the circle radius it will be reduced if I will take it one by one.

Explore Pybytes | Official Documentation | Report a Firmware Bug/Issue | GitHub

New firmware release, version 1.6.0.b1

Pycom on Twitter