30 sec boot time vs deep sleep

[jcaron]

@martijnthe Are you sure the 30 seconds are really just loading the code?

If you have network init, sleeps, or large computations, those can probably be optimised. 30 seconds seems like a lot of work for an IoT device.

Also, I don't know what your actual use case is, but most IoT devices (especially battery-powered, with a long life) should really have a very basic scenario: wake up, collect data, send it, go to sleep. CPU use should be very limited. In my experience, there's more time waiting (e.g. for the RX window in the case of LoRaWAN, or for an IP address if using DHCP over Wi-Fi) than anything else.

If you have lots of processing that is done locally, you should probably think about moving that to the server side of things if at all possible, that's really where things should happen, not on the device.

[martijnthe]

@jcaron said in 30 sec boot time vs deep sleep:

@martijnthe Are you sure the 30 seconds are really just loading the code?

Pretty sure. I measured before and after the imports in boot.py and main.py and that's where the time is going. The imports themselves don't "do" anything but declare classes and functions.

There's very little to no processing going on.

[robert-hh]

@martijnthe That seems to be a pretty large code, if it is pre-compiled and still requires 30 seconds for loading. Frozen bytecode would be faster because that is not moved to RAM, but stays in flash.

[martijnthe]

@crumble said in 30 sec boot time vs deep sleep:

Check your code for all sleeps. Sounds like you wait a lot for the network inits and other stuff.

There are no sleeps. The 30 secs is just for the import statements to complete (imports of modules that only contains function and class definitions).

[reidfo]

@martijnthe said in 30 sec boot time vs deep sleep:

@crumble said in 30 sec boot time vs deep sleep:

Check your code for all sleeps. Sounds like you wait a lot for the network inits and other stuff.

There are no sleeps. The 30 secs is just for the import statements to complete (imports of modules that only contains function and class definitions).

That's really a huge amount of time and code for an embedded device, especially if you want it to sleep. As suggested earlier, can you break up your code and only load what is necessary on each wakeup? My own code is too large to deep sleep, and even with sleeps, LTE connection, and MQTT connection, plus several imports it only takes about 5-10 seconds before everything is ready.

[martijnthe]

@robert-hh said in 30 sec boot time vs deep sleep:

Frozen bytecode would be faster because that is not moved to RAM, but stays in flash.

Thanks for that tip.
That sounds much more attractive than breaking up the code to only load what's necessary.

@reidfo said:

My own code is too large to deep sleep, and even with sleeps, LTE connection, and MQTT connection, plus several imports it only takes about 5-10 seconds before everything is ready.

TBH, 5-10 seconds latency to be able to respond to an external interrupt is unacceptably long. I'm used to microcontrollers where this takes only a couple microseconds!

I'm also more used to microcontrollers where you (as developer) can decide what portions of RAM you want to keep alive during sleep so that don't loose everything when going to sleep. The ESP32 has a couple Ks of "RTC memory" which is kept alive in deep sleep, but I'm not sure if 1) it's accessible somehow from .py and 2) whether/for what it's currently used.

Thanks for all the suggestions. Ideas I may try:

• Bake the code into the firmware using "frozen bytecode" (still does not )
• Investigate keeping PSRAM alive and re-using its contents + look into RTC RAM.

[robert-hh]

@martijnthe About frozen bytecode: The 30 seconds boot time lets me gues that you have about 30.000 lines of code. Freezing that may overflow the partition size, which requires to rework the partition table.

[jcaron]

@robert-hh which brings me back to my earlier point: who needs 30K lines of code on a battery-powered IoT sensor? Does it really need to run on the device itself, rather than on a server?

Also note that the more code you have on the device, the more difficult updates are going to be, especially if you use very low bandwidth networks (especially on the downlinks) such as LoRaWAN.

@martijnthe the Pycom modules try to bring the advantages of (micro)Python to an embedded platform (which makes development usually easier and quicker), but this will obviously have drawbacks if you compare that to raw assembly running directly "on the metal". If you want to get the highest performance (in terms of speed, code size, memory use...) out of the devices, and have full control of the ESP32, you'll probably have to switch to C and/or assembler directly. And if you have something around 30K of micro python, that would be an ever larger (and more difficult to maintain) codebase in C or assembler...

But do you really need that on an IoT sensor?

[robert-hh]

@jcaron I do not know how large the code is. So 30k lines uis just an extrapolation from the times I have seen. If the code is much smaller and still takes 30 to start, it's likely not just loading, but also a lot of code execution performed.

[Paul Thornton]

Id like to take a look at this for you over the christmas holidays as 30 seconds really is abnormally long,

If your code could make its way over to paul@pycom.io with a link to this thread Ill keep it private and see what i can do :)

[andreas]

Dear @martijnthe,

we have been able to successfully get the boot time of our Terkin-Datalogger [1] from about 25 seconds down to about 3 seconds by freezing the modules like outlined by @robert-hh. The code base including all third-party modules is around 10 kLOC and we have released the result as builds called Annapurna [2].

With kind regards,
Andreas.

[1] https://github.com/hiveeyes/terkin-datalogger
[2] https://community.hiveeyes.org/t/annapurna-firmware-for-pycom-esp32/2754