New firmware release v1.17.2.b1 (Flash encryption support)

[daniel]

Hello everyone,

We have released version 1.17.2.b1 with the major highlight being encryption support. For the time being, in order to enable flash encryption and secure boot, you need to build the firmware locally (so that you can use your own key to encrypt the flash).

These are the release notes:

• esp32: Always use the DR specified during the join request (even for retries).
• esp32, lib: Fix deepsleep issues with LoRaWAN losing the channelmask and pending MAC commands.
• esp32/Makefile: If make SECURE=on then ENCRYPT=on is enabled by default.
• esp32: Added secure boot reflashable bootloader in make build-flow.
• esp32: Added Makefile flash encryption support, added comments in updater.c.
• esp32: Enabled Secure Boot, if make SECURE=on.
• esp32: Add argument to lte.attach() to specify the bands.
• esp32: Add option to setup the LTE carrier. Also allow for roaming registration.
• esp32: Move WLAN and LTE socket functions to a common module.
• esp32: Correct socket timeout implementation for WLAN.
• esp32: Change vTaskDelay for mp_hal_delay_ms where appropriate. This unlocks the GIL making possible better multi-threading.
• esp32: Use the correct IDF function for the Pin hold functionality.
• esp32: Revert to using the chip revision to determine the flash size. This works better with secure boot and flash encryption.
• esp32: Add WDT on boot API.
• esp32: Allow baud rates of up to 1000 Kbps for CAN bus.
• esp32: Make compliance script configurable to support all 4 regions and both activation methods.
• esp32: Update IDF libraries moving _gettimeofday to IRAM.
• esp32: Change LoRaWAN OTAA duty cycle from 15 to 10 seconds.
• esp32: Place hal_tick_ms function also in IRAM.
• esp32: Get latest binary.c file from MicroPython that fixes struct packing issues.
• lib/lora: Keep the data rate after a successful join accept.
• esp32: Fix the LoRa Rx messages time stamping. Thanks to @robert-hh.
• esp32: Run heartbeat on Core0, ensure pycom.heartbeat(False) is complete to avoid pycom.rgbled command to fail if used immediately after pycom.heartbeat(False)

For the instructions to enable flash encryptions and secure boot, please read here: https://github.com/pycom/pycom-micropython-sigfox#steps-for-using-secure-boot-and-flash-encryption.

Also, from the next release onwards we will no longer publish the appimg.bin as this is now identical to the regular firmware bin file. This is just as a heads-up so people can change their processes if anyone happens to use it.

Cheers,
Daniel

[livius]

@daniel
Impressive work. Thank you for flash encryption :) I have to test it on the weekend.
Thank you for fixes also.

1. Question - what do wdt on boot? I suppose it can be turned on/off?
   But how to access to the object instance then?
   We must create wdt object self and feed it or it is created on boot time with some timeout?
   Maybe i misunderstood this..
2. https://github.com/pycom/pycom-micropython-sigfox#steps-for-using-secure-boot-and-flash-encryption
   if SECURE=on by default - why?

[robert-hh]

@daniel Hi Daniel, I have a simple script for testing the µs-timer. Thios script crashed with the new firmware on my FiPy. I cloned the repository new, updated esp-idf. Here is the script:

import utime
from machine import Pin

pin = Pin("P10", Pin.OUT)
pin(0)

diff = const(2500000)
while(True):
    start = utime.ticks_us()
    pin(1)
    while utime.ticks_diff(start, utime.ticks_us()) < diff:
        pass
    start = utime.ticks_us()
    pin(0)
    while utime.ticks_diff(start, utime.ticks_us()) < diff:
        pass

It just creates an ~0.2 Hz square signal on P10.

>>> import ttest
Guru Meditation Error: Core  0 panic'ed (LoadProhibited)
. Exception was unhandled.
Register dump:
PC      : 0x4008cc27  PS      : 0x00060333  A0      : 0x8008b6ed  A1      : 0x3ffd4320  
A2      : 0x3ffd4554  A3      : 0x00060323  A4      : 0x3ffd4538  A5      : 0x00000001  
A6      : 0x000000fe  A7      : 0x00060023  A8      : 0x00000000  A9      : 0x3ffd4554  
A10     : 0x3ffd4554  A11     : 0x00000000  A12     : 0x3ffd4538  A13     : 0x00000001  
A14     : 0x000000fe  A15     : 0x00060023  SAR     : 0x0000000c  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x00000004  LBEG    : 0x400e64bc  LEND    : 0x400e64f8  LCOUNT  : 0x00000000  

Backtrace: 0x4008cc27:0x3ffd4320 0x4008b6ea:0x3ffd4340 0x4008a0a7:0x3ffd4360 0x4008375d:0x3ffd43a0 0x4008380b:0x3ffd43d0 0x400838da:0x3ffd43f0 0x400996cc:0x3ffd4410 0x40083da5:0x3ffd4430 0x400e7bc8:0x3ffd4460 0x401436d9:0x3ffd4480 0x4008b249:0x3ffd44a0

[daniel]

@robert-hh can you post the crash log?

[Xykon]

@livius said in New firmware release v1.17.2.b1 (Flash encryption support):

if SECURE=on by default - why?

It is not possible to use secure boot without flash encryption.

[robert-hh]

@daniel Done, added.

[livius]

@xykon
but if i do not need to encrypt flash?

[daniel]

@livius said in New firmware release v1.17.2.b1 (Flash encryption support):

Question - what do wdt on boot? I suppose it can be turned on/off?
But how to access to the object instance then?
We must create wdt object self and feed it or it is created on boot time with some timeout?
Maybe i misunderstood this..

https://docs.pycom.io/chapter/firmwareapi/pycom/pycom.html#pycomwdtonboottimeouttimeout

Just use the WDT as usual when your app starts.

[livius]

@daniel

Just use the WDT as usual

Now its clear :) and docs are refreshed :)

It is not possible to use secure boot without flash encryption.

but if i do not need to encrypt flash?

[Xykon]

@livius said in New firmware release v1.17.2.b1 (Flash encryption support):

@xykon
but if i do not need to encrypt flash?

Flash encryption and secure boot are optional features.

If you want to use secure boot then you also have to use flash encryption.

[livius]

@xykon
Then if i do not specify SECURE=on it is not defaulted to on? as you say it is optional.

ach!! i ommit ":" it is not about default on only what is default after turned on ;-)

if SECURE=on by default:
encryption is enabled
secure_boot_signing_key.pem is the secure boot key, located relatively to Makefile
flash_encryption_key.bin is the flash encryption key, located relatively to Makefile

i go to sleep and rest ;-)

[daniel]

@robert-hh thanks. I can reproduce it. We will analyse and resolve ASAP.

[robert-hh]

@daniel It crashes not all the time with a crash log. Similar often it simply stalls and cannot even be interrupted with Ctrl-C.

[robert-hh]

@daniel Changing the script into the below makes it stable. Just sleep_ms(1) added to the loop:

import utime
from machine import Pin

pin = Pin("P10", Pin.OUT)
pin(0)

diff = const(2500000-1000)

while(True):
    start = utime.ticks_us()
    pin(1)
    while utime.ticks_diff(start, utime.ticks_us()) < diff:
        utime.sleep_ms(1)
    start = utime.ticks_us()
    pin(0)
    while utime.ticks_diff(start, utime.ticks_us()) < diff:
        utime.sleep_ms(1)

[robert-hh]

@daniel Similar problems on a simple reboot with Ctrl-D. Just tried it a few times. Empty main.py, empty boot.py.

MicroPython 1bf377b on 2018-03-16; FiPy with ESP32
Type "help()" for more information.
>>> 
PYB: soft reboot

network config: ('10.0.0.149', '255.255.255.0', '10.0.0.240', '10.0.0.240')
Guru Meditation Error: Core  1 panic'ed (StoreProhibited)
. Exception was unhandled.
Register dump:
PC      : 0x4008c239  PS      : 0x00060133  A0      : 0x8008b9be  A1      : 0x3ffe3040  
A2      : 0x00fe4dc8  A3      : 0x00060120  A4      : 0x00000001  A5      : 0x0000abab  
A6      : 0xb33fffff  A7      : 0x3ffc2d3c  A8      : 0x0000cdcd  A9      : 0x3ffe3040  
A10     : 0x0000abab  A11     : 0x00060123  A12     : 0x00060120  A13     : 0x3ffb4fa0  
A14     : 0x00000003  A15     : 0x00060023  SAR     : 0x00000018  EXCCAUSE: 0x0000001d  
EXCVADDR: 0x00fe4dc8  LBEG    : 0x4009b2ac  LEND    : 0x4009b2b7  LCOUNT  : 0x00000000  

Backtrace: 0x4008c239:0x3ffe3040 0x4008b9bb:0x3ffe3070 0x40102cb3:0x3ffe30b0

================= CORE DUMP START =================
zDoAABEAAABsAQAA
dHf9P3Bf/T8gYf0/
cF/9P8Bg/T9XcQAAGFL8PxhS/D90d/0/EFL8PwcAAACQw/0/kMP9P3R3/T8AAAAA
EgAAACRT/T90aVQAdAClJkZyBeD8qegA////fyBh/T8AAAAAIQAGABIAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAke/0/jHv9P/R7/T8AAAAAAAAAAAEAAAAAAAAA
01lAPwAAAACouAlAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAINehw==

[daniel]

@robert-hh yeap, I'm seeing that as well....

[robert-hh]

This post is deleted!

[robert-hh]

@daniel I reverted back to 1.17.0b1. It looks like the soft reset problem already existed there.
Update: No soft reset problem in version 1.16.0.b1

[ssmith]

@Xykon You said awhile back that you were working on a release program for FW releases and possibly Beta testing before releasing FW. How close are you to implementing this? I'm having a real hard time accepting the fact that PyCom keeps releasing new FW with very little testing. I have seen issues in almost every FW release that just a minimal QA test should discover. If PyCom's target customers are hobbyists, you have a great product. If PyCom wants to sell products to companies that are trying to base a business off using these devices then something needs to change ASAP. This kind of behavior from PyCom will put many start-ups relying on PyCom out of business.

[timh]

@ssmith I would agree with your assessment.

I am endeavoring to use GPY & FiPY to monitor a number of devices in a commercial setting (between 50 and 150 high value devices) which implies that many pycom boards.

The package, functionality (LTE, sigfox, gps) etc.. all go together to make it an ideal platform.

However not being able to get LTE to work, finding a way to repeatably hang device in latest firmware doesn't instill a great deal of confidence.

I have a little bit of time up my sleeve however if I have no traction resolving LTE in the next few weeks, then I must look to another solution.