How do I diagnose MemoryError?

[BetterAuto]

Need help diagnosing this. I have nothing in boot.py or main.py.

The err.py and data_store.py modules have been tested individually and did not throw any MemoryErrors. The err.py module captures error messages and stores them via data_store.py to my server if available, or on the local filesystem if the server is not available, for sending at the next opportunity. When testing err.py I necessarily had to also test data_store.py.

What should my next steps be?

>>> mem_info()
stack: 736 out of 7168
GC: total: 67008, used: 7616, free: 59392
 No. of 1-blocks: 23, 2-blocks: 6, max blk sz: 384, max free sz: 3702
>>> import test_motor
Starting test_motor
[SUCCESS] motor.voltage
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test_motor.py", line 19, in <module>
  File "motor.py", line 51, in run
  File "err.py", line 17, in __init__
  File "data_store.py", line 23, in __init__
MemoryError:
>>> mem_info()
MemoryError
>>>

[BetterAuto]

Confirmed that the problem is in err.py and/or data_store.py. Commented it out and no more MemoryError. But how to troubleshoot those, I don't know. As mentioned, those don't error when tested directly.

[robert-hh]

@betterauto Your code probably simply runs out of heap space.
And that happens in the creation of class instances. You should look if you create these instances once or repeatedly. If once, look how much memory they allocate and try to reduce that. If repeatedly, try to insert gc.collect() before creating the class instances.
Otherwise, you can save a lot or RAM by putting known working modules into frozen bytecode, at which point their code does not use RAM any more, at the penalty of a few percent slower execution.

[BetterAuto]

How do I determine how much memory they use? What commands?

[robert-hh]

@betterauto You can use gc.mem_free() or gc.mem_alloc() to look at the change in allocation before after some activities. That's a roigh indication. It can still happen that even if mem_free() shows sufficient memory, you cannot allocate a large buffer because that free memory is fragmented. gc.collect() can be used to force garbage collection, which often helps.

[BetterAuto]

I do gc.collect() often but for some reason was not doing so inside this module. I'll try that. Good info on the other gc functions, will try those.

[BetterAuto]

Getting the same error in a different module but this one uses copious gc.collect(). I am at a loss what to do next. Frozen bytecode? Move to a different platform? Suggestions appreciated.

This module tests my update functionality. The updates module downloads either new system files or new data files. It uses the cloud module, which in turn uses the mqtt module, which in turn uses the Adafruit robust MQTT client. Too many modules?

I edited the Adafruit module to run gc.collect() very frequently, almost every other line.

You can see from running gc.collect() manually on the REPL that it doesn't free much. But why would running it there make any difference if I am also running it copiously within the code?

Frustrating.

>>> import test_updates
Starting test_updates
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test_updates.py", line 9, in <module>
  File "updates.py", line 7, in <module>
  File "cloud.py", line 4, in <module>
  File "mqtt.py", line 12, in <module>
  File "umqtt/robust.py", line 7, in <module>
MemoryError: memory allocation failed, allocating 82 bytes
>>> import gc
>>> gc.mem_free()
18688
>>> gc.mem_alloc()
48432
>>> gc.collect()
>>> gc.mem_free()
19808
>>> gc.mem_alloc()
47280
>>>

[robert-hh]

@betterauto Frozen bytecode is it. gc.collect() won't change anything, if the majority of RAm is used by bytecode. Frozen bytecode moves that to flash.

[BetterAuto]

Cool thanks for the rapid reply. Do you have a link for a handy guide for doing this? Edit: This looks good:
https://learn.adafruit.com/micropython-basics-loading-modules/frozen-modules

[robert-hh]

@betterauto It's not difficult, but uses a few steps, which may fail. But once you succeded, thigns get much more convenient.
You have to set-up your own build environment.
a) clone this repository https://github.com/pycom/pycom-micropython-sigfox
b) follow the instructions in https://github.com/pycom/pycom-micropython-sigfox/README.md, section ESP32 (not Pyboard)
c) Once you've cloned it, goto your_path_to_the_clone/pycom-micropython-sigfox/esp32 and follow the instructions in the esp32/README.md. That includes the cloning of another repository, https://github.com/pycom/pycom-esp-idf
d) once you have set-up everything, you'll end up using make for everything:

• cleaning: make clean
• build bootloader: make TARGET=boot
• build for WIPY: make BOARD=WIPY
• build for LOPY, make BOARD=LOPY LORA_BAND = USE_BAND_868
• upload firmware: make flash (needs jumper G23->GND too, unless you use a non-Pycom board)
• erase board: make erase

The best environment for that is Linux. Some people use OS X too
Scripts to go into frozen bytecode have to be placed in the esp32/frozen directory. by default, this directory only contains _boot.py. Then run make and make flash.
The binaries built during that process are in esp32/build/xxPY/release/appimg.bin. That file can be upload by OTA too.

[BetterAuto]

Excellent, thanks. Will try to grok this when I'm more awake.

In the interim I also will trim down the modules to the barest minimum to test the functionality I'm after, then will port the code fixes into the fuller modules, freeze the code, and viola. That way I'm not taking a lot of extra time to recompile the image every time just to debug some functionality.

[robert-hh]

@betterauto The adafruit link is for esp8266. For Pycom esp32, the method is the same, just another directory. And obviously you can put stable modules in frozen bytecode, and the ones in progress in RAM. The code looks the same.

[BetterAuto]

Robert rather than compile into the flash can I use .mpy files instead? I read somewhere that WiPy runs everything out of RAM, even frozen bytecode. I found your mpy compiler:
https://forum.micropython.org/viewtopic.php?t=2299

[robert-hh]

@betterauto You can do so, but that only helps if you run into memory problems during import. If you use the compilers from my repository, be awary that the versions for micropython.org and pycom.io are different. Let me update the pycom.io version.

Edit: Updated. May not have been necessary.

[BetterAuto]

@robert-hh looking at the test_updates error I pasted earlier is that an error on import do you think? Not sure how to tell the difference.

[robert-hh]

@betterauto Import error happen at the import statement. so if File "umqtt/robust.py", line 7, in <module> is an import statement, the pre-compiling can help.

[BetterAuto]

MOST helpful. How can I thank you beyond upvoting your replies?

[BetterAuto]

@robert-hh said in How do I diagnose MemoryError?:

b) follow the instructions in https://github.com/pycom/pycom-micropython-sigfox/README.md, section ESP32 (not Pyboard)

Correct URL
https://github.com/pycom/pycom-micropython-sigfox/blob/master/README.md