New firmware release, version 1.6.0.b1

[daniel]

@Neo, I modified your script in order to be able to run it:

from network import LoRa
import utime

class test:
    def __init__ (self):
        self._lora = LoRa(mode=LoRa.LORA)

        import socket
        self._socket = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
        self._socket.setblocking(False)    

    def run(self):
        import gc
        gc.enable()

        while True:
            try:
                bytes = self._socket.recv(512)
                if len(bytes) > 0:
                    print("%s RECV %s" %  (self, bytes))

            except Exception as e:
                print("_recv", str(e))

            print('Memory %d' % gc.mem_free())
            utime.sleep(1)

t = test()
t.run()

I get the following output:

Memory 54880
Memory 54320
Memory 53760
Memory 53200
Memory 52640
Memory 52080
Memory 51520
Memory 50960
Memory 50400
Memory 49840
Memory 49280
Memory 48720
Memory 48160
Memory 47600
Memory 47040
Memory 46480
Memory 45920

Which is totally correct, that's the way Python works. Memory is consumed as code runs, and when there's no more memory, the garbage collector kicks in and frees memory so that code can continue running. At some point you see:

Memory 5040
Memory 4480
Memory 3920
Memory 3360
Memory 2800
Memory 2240
Memory 1680
Memory 1120
Memory 560
Memory 0
Memory 54592
Memory 54032
Memory 53472
Memory 52912

However, if I add a gc.collect() line like this:

from network import LoRa
import utime

class test:
    def __init__ (self):
        self._lora = LoRa(mode=LoRa.LORA)

        import socket
        self._socket = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
        self._socket.setblocking(False)    

    def run(self):
        import gc
        gc.enable()

        while True:
            try:
                bytes = self._socket.recv(512)
                if len(bytes) > 0:
                    print("%s RECV %s" %  (self, bytes))

            except Exception as e:
                print("_recv", str(e))

            print('Memory %d' % gc.mem_free())
            utime.sleep(1)
            gc.collect()

t = test()
t.run()

I get:

Memory 55520
Memory 55664
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648
Memory 55648

Which at least in this test it shows that there's no memory leak...

[Neo]

@daniel Here it is your memory leak script:

def test2():

from network import LoRa
_lora = LoRa(mode=LoRa.LORA)

import socket
_socket = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
_socket.setblocking(False)    

import gc
gc.enable()   
while True:

    try:
        bytes = self._socket.recv(512)
        if len(bytes) > 0:
            print(" %s RECV %s" %  (self, bytes))
        
    except Exception as e:   
            print("_recv", str(e))    
            
    print('Memory %d' % (gc.mem_free()))

    utime.sleep(1)

test2()

[Neo]

@daniel Unfortunatelly the code is huge. I will try to reduce the area. I chekced also one auto tests only with LoRa initiation and Lora socket reading + gc.mem_free calls ... I saw that this test is hanging. I believe the real problem ties to memory leak /bad management. Here is the memory output at each second called soket recv and dump the mem_free:
Memory 19264
Memory 18656
Memory 18080
Memory 17504
Memory 16928
Memory 16352
Memory 15776
Memory 15200
Memory 14624
Memory 14048
Memory 13472
Memory 12896
Memory 12320
Memory 11744
Memory 11168
Memory 10592
Memory 10016
Memory 9440
Memory 8864
Memory 8288
Memory 7712
Memory 7136
Memory 6560
Memory 5984
Memory 5408
Memory 4832
Memory 4256
Memory 3680

[daniel]

Then it's not the gc anymore in your case. If you can share with me a piece of code that can reproduce it I'll get onto it.

Cheers,
Daniel

[Neo]

@daniel Still hanging on LopY. In our script this is happening after wifi STA connect ... we have a gc.mem_free call. We removed all gc collect calls.

[daniel]

Hello,

1.6.3.b1 is out there. In my tests the gc.collect() problem is now fixed. @livius @JF002 please test and let me know... Thanks!

Cheers,
Daniel

[JF002]

@daniel said in New firmware release, version 1.6.0.b1:

Just freeze my lopy. I noticed it when trying to fix another issue with ds1820 and mqtt :

OK it happens, not always at the first time. I'm working on it... Wait for the 1.6.3.b1 coming a bit later...

Yeah, thank you! You're the best!

[daniel]

Just freeze my lopy. I noticed it when trying to fix another issue with ds1820 and mqtt :

OK it happens, not always at the first time. I'm working on it... Wait for the 1.6.3.b1 coming a bit later...

[JF002]

@daniel

import gc
gc.collect()

Just freeze my lopy. I noticed it when trying to fix another issue with ds1820 and mqtt : https://forum.pycom.io/topic/670/onewire-wifi-mqtt-crash/7

[daniel]

@livius can you show me an example of the gc.collect() crash?

[JF002]

@daniel With 1.6.1, I encountered some instabilities : the lopy hangs when trying to sync the project from Pymakr, and when I call gc.collect(). Are you aware of these issues? Is it possible that they are fixed in 1.6.2?

Thanks

[livius]

@daniel
3905 bytes

I will test with 1.6.2 and will see if something changed

could it be that your file system is corrupted from previous firmware versions? Could you try:

I can erase all data - but i can back tomorrow with review

[daniel]

@livius could it be that your file system is corrupted from previous firmware versions? Could you try:

import os
os.mkfs('/flash')

to format the file system and start from scratch...

Cheers,
Daniel

[daniel]

@livius what's the size of the file?

[livius]

@daniel
it is by copy file throught e.g. filezilla and without while true: loop

[daniel]

I still have problem of corruption of files stored throught ftp :(

@livius Really? We have been testing a lot during the last 2 weeks with large files and the issue seem resolved. Does it happen when you perform file operations in your script while at the same time the FTP server is busy with a transfer?

[daniel]

Hello,

We just released new firmware, version 1.6.2.b1. These are the changes:

• Fix issue when trying to send empty strings via sockets.
• Update the ESP-IDF.
• Correct RCZ3 uplink frequency and integers overflow.
• Implement ntp_sync for RTC. Add simple support for timezones. See: https://docs.pycom.io/pycom_esp32/library/machine.RTC.html#machine.rtc.ntp_sync
• Correctly set the DNS server. Also disable and enable the right TCP adapter interface.

Cheers,
Daniel

[livius]

@daniel

i still have problem of corruption of files stored throught ftp :(

[Pete]

@Colateral I ordered some of these: LinkIt Smart 7688 at the same time as my WiPys. Similar form factor, but runs OpenWrt, a Linux distro originally developed for customising Linksys routers. They arrived at the start of this week. I haven't even opened the box yet, as I'd like to get the WiPy working if I can - my use case is pretty simple and I'd rather not have to admin a whole Linux if I can just write one Python script instead. From the point of view of attack surface if nothing else. But it's a second basket for my eggs if required.

Pete

[Colateral]

@Pete said in New firmware release, version 1.6.0.b1:
Which different brand of WiFi IoT module are you using?