MemoryError: memory allocation failed, allocating 2048 bytes - LOPY - 1.2.0.b1

[mkharibalaji]

Hi Team,
I am trying to make use of SSD1306 OLED 128x64 Display with WLAN http_Client library to POST some JSON data to the cloud.I am getting Memory Error : memory allocation failed allocating 2048 bytes

Configuration :
Lopy (sysname='LoPy', nodename='LoPy', release='1.2.0.b1', version='v1.8.6-290-gdac16bd on 2017-01-05', machine='LoPy with ESP32')

In Future,I need to use some mqtt library feature as well.How can i sort out this issue ? Please advice.

Thanks & regards,
Hari

[Colateral]

@Shaun I can enumerate at least 5 active people from this forum that required the frozen functionality.

[Shaun]

I too require "frozen" code for my large application and hope the git repository will be updated and that "frozen" functionality will be available sooner rather than later.

I am ecstatic that the threading and GC issues have been resolved and that I can continue with development at this point ;-)

[robert-hh]

@Colateral: No, I cannot. And since the repository is not updated, I cannot make a suitable build. That's kind of sad. I asked Daniel, and he said (as always) that they will do so soon. But like you asked, enabling pre-compiled or frozen bytecode is just a small change in mpconfigport.h, however at the cost of additional firmware code size (~2k).
It could be an issue of testing and support.

[Colateral]

@robert-hh With the latest release 1.6.7 are you able to run on fronzen bytecode?

[robert-hh]

Hello @Shaun, I recall something like that in the past. I think I did the same. Did you update the pycom-esp-idf too? You should.
And updating th esp-idf might have required deleting it and cloning again.

[Shaun]

@robert-hh
I did have an issue compiling mpy-cross initially. (make -C mpy-cross)

In file included from ../py/parse.h:32:0,
from ../py/scope.h:29,
from ../py/emit.h:31,
from ../py/emitnative.c:50:
../py/obj.h:33:22: fatal error: esp_attr.h: No such file or directory
compilation terminated.

To fix this I ended up manually copying the
~/pycom-esp-idf/components/esp32/include/esp_attr.h
file into
~/pycom-micropython/py/esp_attr.h

Perhaps something is not correct?

[robert-hh]

Hello @Shaun, That's strange. I used that method to embed two large scripts, a text editor and a set of unix--shell-like commands, which I typically have on all boards, and these work fine.
I'm using the latest available Lopy repository form this place: https://github.com/pycom/pycom-micropython.

>>> os.uname()
 (sysname='LoPy', nodename='LoPy', release='1.5.0.b2', version='v1.8.6-437-gea056e6-dirty on 2017-02-10', machine='LoPy with ESP32')

I guess you do that too. Did you use mpy-cross from the same build? I never had experienced that it make a difference, but you never know. Maybe it's a size issue. Did you try first with smaller files?
Although, the code I have embedded totals to about 750 lines, so it is not tiny.

[Shaun]

@robert-hh
I duplicated the steps you have listed and all appeared to work correctly.
The *.py files were converted to *.mpy files and then frozen and compiled into an object file, frozen_mpy.o
That file has been linked into the final image and the module does not give an import error, so is very much part of the image.

But I get errors like;

File "main.py", line 28, in <module>
TypeError: 'int' object is not callable
MicroPython v1.8.6-412-gf55ba50-dirty on 2017-02-10; LoPy with ESP32

and if I use dir(my_module) the listed content is not correct.

It is as if the frozen module content is in some way corrupted.

[robert-hh]

@Shaun: This one worked for me:

In mpconfigport.h, goto the line

#define MICROPY_MODULE_FROZEN (0)

and change it to the three lines:

#define MICROPY_MODULE_FROZEN (1)
#define MICROPY_MODULE_FROZEN_MPY (1)
#define MICROPY_QSTR_EXTRA_POOL mp_qstr_frozen_const_pool

Then, put your python code like before into the directory modules and run:
make FROZEN_MPY_DIR=modules BOARD=LOPY

[robert-hh]

At first glance, everything should be there. Next step: I will try to add these to lines to mpconfigport.h:

#define MICROPY_MODULE_FROZEN_MPY           (1)
#define MICROPY_QSTR_EXTRA_POOL             mp_qstr_frozen_const_pool

That activates the required code in pyexec.c. Maybe then, a load error pops up, and I'll seet further.

[Shaun]

@robert-hh
No, frozen-mpy.o does not appear in the application.map file, so it does not seem to be linked in.

[robert-hh]

@Shaun: Did you verify that the module frozen-mpy.o is the loaded into the image? It should be in the loader map file. If not, itÄs not in the image. If yes, it still might not be searched for in the exec() function. I try to find that myself tonight. That feature should better be in the master branch, because it's very useful especially for large data tables like fonts. B.t.w.: Do you know the work of @pythoncoder about fonts? https://github.com/peterhinch/micropython-font-to-py

[Shaun]

@robert-hh

That seemed to work; (Compiler output)

MPY modules/keypad.py
MPY modules/crc.py
MPY modules/tw.py
MPY modules/OLED.py
Creating build/LOPY/release/frozen_mpy.c
CC build/LOPY/release/frozen_mpy.c

But...after deleting the *.mpy files that the above modules used to reside in, I get;

File "main.py", line 6, in <module>
ImportError: no module named 'keypad'
MicroPython v1.8.6-412-gf55ba50-dirty on 2017-02-10; LoPy with ESP32

And if I remove the import statement I get;

File "main.py", line 9, in <module>
NameError: name 'keypad' is not defined

So, if the modules are indeed frozen into the image, do I have to access them in a different way?

[robert-hh]

That's how it works with the micropython.org build. You simply put your files into a dicrectory (default name 'modules') and create the firmware image with make. Then these files will by compiled as part of the make process and move into an embedded file like memory area, which is scanned to by the VM when it performs an import statement. For your code, you'll see no difference. Only these frozen files are not visible e.g. in a listdir() call. It did not look yet for the hooks of that mechanism in the code. You could give it a try by putting some scripts in esp32/modules and call:
make FROZEN_MPY_DIR=modules
mpy-cross must be in PATH for that.

[Shaun]

@robert-hh
Thanks for your reply, Robert.

Yes, I understand the mechanism, I think.
I have an LCD module containing many font tables and bmp images stored as bytes() data.
This module is debugged and working, but all that const data is consuming many kB of ram when it should stay in flash.

I was thus hoping it was a simple case of setting a #define FROZEN in a config file and then placing my lcd.py script into a "/frozen_mod/" folder for example, and compiling it into the micropython image to reside and run from flash.

[robert-hh]

@shaun: That method is called frozen bytecode and exists in the micropython.org set of implementtaions. It is already impemented in their esp32 branch. And since pycom.io and micropython.org will join their development, There is a good chance that it will be provided for LoPy too. The basic mechanisms are already there.
The advantage is that the bytecode will then reside in flash, and much lager code is possible. At the downside, developemnt will required a full flash for every test iteration.

[Shaun]

@robert-hh
Thanks Robert, I have managed to duplicate this method and thus can continue development for a little longer with the ram savings achieved.

Do you have similar instructions for compiling one's own *.py files alongside micropython thus embedding them into the firmware image such that they run from flash and are not loaded to ram, thus saving even more ram?

[livius]

@leelive
I am also interested because i still play for fun with different sensors
but RAM limit cause stop in main development

UPDATE @administrators
I see now on web that info:
Contact us on sales@pycom.io for volumes above 100 units.
does it mean that it will be also available in single unit sales?
I suppose that yes - minimal 10units?

previously there was only shipping above 500units

You really should inform us somehow about such changes in shipping :)
errata
i see that you inform about this on 16 january
i have email in spam folder (unknow reason)

[leelive]

@abilio My company is hoping to use your LoPY OEM module. However, even after the firmware has matured, it looks like RAM will be a problem for at least one of our use cases. Offering an OEM module with increased RAM (like the FiPy) would be a really important feature for us. Is this a possibility? Thanks!