Free memory issues ?



  • Ok guys, my Python skills are nearly 0.5, but this is driving me nuts...
    I did a small script that scans for BLE advertisers and pushes their data thru mqtt. Every scan cycle I also change led color and test free memory, asking for gc() if mem falls below 30000.
    WDT is enabled and feeded after any advertisement received. What i cannot understand is why free memory falls often well below 30000 (seen even only around 1K free), and after a while WDT kicks in and wipy 2 restarts. Why ? What stupid thing am I doing ?

    Thanks for everyone assistance

    Claudio

    from network import Bluetooth
    from binascii import hexlify
    from network import WLAN
    import utime
    import gc
    import pycom
    import machine
    import os
    import sys
    import ujson as json
    from umqtt import MQTTClient
    from machine import WDT
    
    def settimeout(duration):
         pass
    
    # json.dumps()
    fwver  = os.uname()[2]
    ipaddr = WLAN().ifconfig()[0]
    mymac  = hexlify(machine.unique_id()).decode()
    bt = Bluetooth()
    pycom.heartbeat(False)
    pubmsg = ""
    free = 0
    wdt = WDT(timeout=20000)
    client = MQTTClient(mymac, "10.168.193.10", port=1883)
    client.settimeout = settimeout
    client.connect(clean_session=True)
    
    if machine.reset_cause() == machine.WDT_RESET:
        pubmsg = "{\"ts\":" + str(utime.time()) + ",\"fw\":\"" + str(fwver) +"\",\"ip\":\"" + str(ipaddr) + "\",\"mac\":\"" + mymac + "\",\"why\":\"WDT_RESET\"}"
        client.publish("/stat", pubmsg, retain=True, qos=1)
    
    while True:
        try:
            free = gc.mem_free()
    
            if free < 30000:
                gc.collect()
    
            pycom.rgbled(0x007f00)
            utime.sleep_ms(10)
    
            pubmsg = "{\"ts\":" + str(utime.time()) + ",\"fw\":\"" + str(fwver) +"\",\"ip\":\"" + str(ipaddr) + "\",\"mac\":\"" + mymac + "\",\"free\":" + str(free) + "}"
            client.publish("/stat", pubmsg, retain=True, qos=1)
            bt.start_scan(30)
    
            while bt.isscanning():
                pycom.rgbled(0x7f0000)
                adv = bt.get_adv()
    
                if adv:
                    mac = hexlify(adv.mac).decode()
    
                    try:
                        mfg_data = bt.resolve_adv_data(adv.data, bt.ADV_MANUFACTURER_DATA)
                        manu_data = hexlify(mfg_data).decode()
                    except:
                        manu_data = "no data"
    
                    name = bt.resolve_adv_data(adv.data, bt.ADV_NAME_CMPL)
    
                    if name:
                        pubmsg = "{\"ts\":" + str(utime.time()) + ",\"mac\":\"" + mymac + "\",\"cli\":\"" + name + "\",\"manu_data\":\"" + manu_data + "\",\"rssi\":" + str(adv.rssi) + "}"
                        client.publish("/tomqtt", pubmsg, retain=True, qos=1)
    
                pycom.rgbled(0x000000)
                wdt.feed()
    
        except KeyboardInterrupt:
            if bt.isscanning():
                bt.stop_scan()
            pycom.heartbeat(True)
            sys.exit()
    


  • @duffo64

    (Ops... must wait 10 minutes before posting again).

    I have upvoted your post to prevent this ;-)



  • @livius
    Thanks, will try, but at this point I think that WDT is not to blame. I suppose that watchdog kicks in since the environment becomes clogged due to memory shortage.
    Updates are always welcome, nonetheless :)

    (Ops... must wait 10 minutes before posting again).

    Claudio



  • @RichardTXD
    Hi,

    and... you nailed it. Deletion and recreation seems the key here.
    Even if I am not asking for a whole week of uptime, doing at least a clock round would suffice. What's great is that after following your suggestion I am at three hours on 3 different units, and not a single reboot since. Will see...

    So, big thanks !

    Claudio



  • @RichardTXD said in Free memory issues ?:

    as it says 'sigfox'

    Yes, sigfox in name in my opinion is misleading - but not my choice. And yes this repo catch all boards.

    WDT will timeout I presume?

    I guess so too...



  • @livius
    Thanks - didn't know about the issue tracking. Probably would not have paid it attention otherwise as it says 'sigfox' - maybe that means it is a catch all.

    However, if the Bluetooth crashes the WiPy then the WDT will timeout I presume? Might be relevant?
    He is using different parts of the system tho - it was just my experience with Bluetooth...



  • @RichardTXD
    Crash are different thing and should be analyzed deeply by pycom.
    If you have reproducible test case report it here:
    https://github.com/pycom/pycom-micropython-sigfox/issues

    @duffo64
    about WDT - try current firmware 1.7.5 maybe something is changed.



  • @duffo64
    Hi.
    I had similar issues just reading BLE advertising packets and printing them to the console. Nothing clever. After a few minutes or a few hours it would crash. It was somewhat random.

    I did not have a WDT implemented.

    I tried gc in various forms, manual, automatic etc here and there, also watched free memory, various combinations all failed. I think about 3 hours was the best I did with this.

    What worked for me.... I told it to scan for just 10 seconds ( I see you set 30 seconds... I used 10for no good reason... didn't try 30). When it stops scanning, you can detect that, deinitizlise the ble object, reinitialize it and start scanning again. It does this really quite quickly.

    I got the above (sadly not clever) scheme to run for 6+ days without a hiccup on a WiPy 2 and expansion board. We have a lot of beacons in the office, it was getting about 8 to 10 packets a second. That seemed to be the best it would do.

    Hope it helps. Something to try anyway. I've not tried by ble code on the last 2 firmware revisions tho.



  • Thanks for your reply, Livius.
    Unfortunately, feeding after start_scan causes the wdt to kick in even more:

    {"ts":3,"fw":"1.7.4.b1","ip":"199.0.0.111","mac":"240axxxxxxxx","why":"WDT_RESET"}
    {"ts":3,"fw":"1.7.4.b1","ip":"199.0.0.111","mac":"240axxxxxxxx","free":50960}
    

    After less than 15 seconds, another reboot:

    {"ts":3,"fw":"1.7.4.b1","ip":"199.0.0.111","mac":"240axxxxxxxx","why":"WDT_RESET"}
    {"ts":3,"fw":"1.7.4.b1","ip":"199.0.0.111","mac":"240axxxxxxxx","free":50960}
    

    ... and so on.

    At this point I cannot understand if it's me, the wdt, the garbage collector, or what.



  • @duffo64 said in Free memory issues ?:

    bt.start_scan(30)

    what if you feed it after above line?


Log in to reply
 

Looks like your connection to Pycom Forum was lost, please wait while we try to reconnect.