Free memory issues ?

[duffo64]

Ok guys, my Python skills are nearly 0.5, but this is driving me nuts...
I did a small script that scans for BLE advertisers and pushes their data thru mqtt. Every scan cycle I also change led color and test free memory, asking for gc() if mem falls below 30000.
WDT is enabled and feeded after any advertisement received. What i cannot understand is why free memory falls often well below 30000 (seen even only around 1K free), and after a while WDT kicks in and wipy 2 restarts. Why ? What stupid thing am I doing ?

Thanks for everyone assistance

Claudio

from network import Bluetooth
from binascii import hexlify
from network import WLAN
import utime
import gc
import pycom
import machine
import os
import sys
import ujson as json
from umqtt import MQTTClient
from machine import WDT

def settimeout(duration):
     pass

# json.dumps()
fwver  = os.uname()[2]
ipaddr = WLAN().ifconfig()[0]
mymac  = hexlify(machine.unique_id()).decode()
bt = Bluetooth()
pycom.heartbeat(False)
pubmsg = ""
free = 0
wdt = WDT(timeout=20000)
client = MQTTClient(mymac, "10.168.193.10", port=1883)
client.settimeout = settimeout
client.connect(clean_session=True)

if machine.reset_cause() == machine.WDT_RESET:
    pubmsg = "{\"ts\":" + str(utime.time()) + ",\"fw\":\"" + str(fwver) +"\",\"ip\":\"" + str(ipaddr) + "\",\"mac\":\"" + mymac + "\",\"why\":\"WDT_RESET\"}"
    client.publish("/stat", pubmsg, retain=True, qos=1)

while True:
    try:
        free = gc.mem_free()

        if free < 30000:
            gc.collect()

        pycom.rgbled(0x007f00)
        utime.sleep_ms(10)

        pubmsg = "{\"ts\":" + str(utime.time()) + ",\"fw\":\"" + str(fwver) +"\",\"ip\":\"" + str(ipaddr) + "\",\"mac\":\"" + mymac + "\",\"free\":" + str(free) + "}"
        client.publish("/stat", pubmsg, retain=True, qos=1)
        bt.start_scan(30)

        while bt.isscanning():
            pycom.rgbled(0x7f0000)
            adv = bt.get_adv()

            if adv:
                mac = hexlify(adv.mac).decode()

                try:
                    mfg_data = bt.resolve_adv_data(adv.data, bt.ADV_MANUFACTURER_DATA)
                    manu_data = hexlify(mfg_data).decode()
                except:
                    manu_data = "no data"

                name = bt.resolve_adv_data(adv.data, bt.ADV_NAME_CMPL)

                if name:
                    pubmsg = "{\"ts\":" + str(utime.time()) + ",\"mac\":\"" + mymac + "\",\"cli\":\"" + name + "\",\"manu_data\":\"" + manu_data + "\",\"rssi\":" + str(adv.rssi) + "}"
                    client.publish("/tomqtt", pubmsg, retain=True, qos=1)

            pycom.rgbled(0x000000)
            wdt.feed()

    except KeyboardInterrupt:
        if bt.isscanning():
            bt.stop_scan()
        pycom.heartbeat(True)
        sys.exit()

[livius]

@duffo64 said in Free memory issues ?:

bt.start_scan(30)

what if you feed it after above line?

[duffo64]

Thanks for your reply, Livius.
Unfortunately, feeding after start_scan causes the wdt to kick in even more:

{"ts":3,"fw":"1.7.4.b1","ip":"199.0.0.111","mac":"240axxxxxxxx","why":"WDT_RESET"}
{"ts":3,"fw":"1.7.4.b1","ip":"199.0.0.111","mac":"240axxxxxxxx","free":50960}

After less than 15 seconds, another reboot:

{"ts":3,"fw":"1.7.4.b1","ip":"199.0.0.111","mac":"240axxxxxxxx","why":"WDT_RESET"}
{"ts":3,"fw":"1.7.4.b1","ip":"199.0.0.111","mac":"240axxxxxxxx","free":50960}

... and so on.

At this point I cannot understand if it's me, the wdt, the garbage collector, or what.

[RichardTXD]

@duffo64
Hi.
I had similar issues just reading BLE advertising packets and printing them to the console. Nothing clever. After a few minutes or a few hours it would crash. It was somewhat random.

I did not have a WDT implemented.

I tried gc in various forms, manual, automatic etc here and there, also watched free memory, various combinations all failed. I think about 3 hours was the best I did with this.

What worked for me.... I told it to scan for just 10 seconds ( I see you set 30 seconds... I used 10for no good reason... didn't try 30). When it stops scanning, you can detect that, deinitizlise the ble object, reinitialize it and start scanning again. It does this really quite quickly.

I got the above (sadly not clever) scheme to run for 6+ days without a hiccup on a WiPy 2 and expansion board. We have a lot of beacons in the office, it was getting about 8 to 10 packets a second. That seemed to be the best it would do.

Hope it helps. Something to try anyway. I've not tried by ble code on the last 2 firmware revisions tho.

[livius]

@RichardTXD
Crash are different thing and should be analyzed deeply by pycom.
If you have reproducible test case report it here:
https://github.com/pycom/pycom-micropython-sigfox/issues

@duffo64
about WDT - try current firmware 1.7.5 maybe something is changed.

[RichardTXD]

@livius
Thanks - didn't know about the issue tracking. Probably would not have paid it attention otherwise as it says 'sigfox' - maybe that means it is a catch all.

However, if the Bluetooth crashes the WiPy then the WDT will timeout I presume? Might be relevant?
He is using different parts of the system tho - it was just my experience with Bluetooth...

[livius]

@RichardTXD said in Free memory issues ?:

as it says 'sigfox'

Yes, sigfox in name in my opinion is misleading - but not my choice. And yes this repo catch all boards.

WDT will timeout I presume?

I guess so too...

[duffo64]

@RichardTXD
Hi,

and... you nailed it. Deletion and recreation seems the key here.
Even if I am not asking for a whole week of uptime, doing at least a clock round would suffice. What's great is that after following your suggestion I am at three hours on 3 different units, and not a single reboot since. Will see...

So, big thanks !

Claudio

[duffo64]

@livius
Thanks, will try, but at this point I think that WDT is not to blame. I suppose that watchdog kicks in since the environment becomes clogged due to memory shortage.
Updates are always welcome, nonetheless :)

(Ops... must wait 10 minutes before posting again).

Claudio

[livius]

@duffo64

(Ops... must wait 10 minutes before posting again).

I have upvoted your post to prevent this ;-)