Firmware release 1.7.3.b1

daniel

Hello,

A new firmware release is out. The version is 1.7.3.b1. Here's the change log:

esp32: Update libraries from the IDF and implement TLS timeouts.

This fixes some TLS crashes and instability issues.
esp32: Add run-time support for the OEM modules based on the ESP32 revision.
esp32: Remove unedded initialization of UART 1 and 2.
esp32: Reduce SD card max frequency to 10MHz.

This fixes the crash seen when stressing the SD card drivers.
This is nedded due to the fact that with the expansion board
we only use the weak internall pull-ups of the ESP32 for
the SD card lines.
esp32: Always stop BLE scanning before connecting.
esp32: Add critical section protection to timer alarm routines.
esp32: Reduce priority of the callback thread.
esp32/frozen: De-init UART 1 and 2 at the end of _boot.py
docs: Document rtc.synced() method.
docs: Added win 7 driver info
docs: Added interrupt explainations to pysense/track
docs: Corrected OEM zip datasheets
docs: Fixed the issues with missing datasheets for pytrack/pysense

The github repo has also been updated accordingly. Make sure to clone the customised IDF from Pycom again as we have pushed new changes.

Cheers,
Daniel

robert-hh

@daniel Hello Daniel. Thank you for the update and also for updating the repository at the same time. I hope it makes many people happy.

I tested the SD card write failure, and I'm sorry to say that this still exists. The SD card access and especially writing is much slower, but it failed at about 10 MBtye of roughly 10.000 cyles of the test code.

livius

@daniel
I see in sources in mentioned above _boot.py that there is

 import os		
 from machine import UART	
 os.dupterm(UART(0, 115200))

then i see that old files like boot.py should not contain dupterm and uart init, and we can remove it from our sources without consequences.

jellium

@daniel Thanks for the continuous work and detailed changelog.

... However, where can we actually download the firmware? I cannot find anything in:

So... where is the release? Thanks.

daniel

@jellium Thanks!

Just get the updater tool from here: https://www.pycom.io/downloads/ and the latest firmware will be applied to your board.

If you want to build the firmware from sources, check out this repo: https://github.com/pycom/pycom-micropython-sigfox

jellium

@daniel Thanks for you quick reply. I got confused with the name of the tarball for the updater (pycom_update_1.1.1.b2.tar.gz) and forgot that applying a firmware update is assisted by the upgrader tool.
In the meantime I had started building it from the sources anyway :D

jellium

@daniel said in Firmware release 1.7.3.b1:

If you want to build the firmware from sources, check out this repo: https://github.com/pycom/pycom-micropython-sigfox

I have followed the instructions (install the Espressif ESP32 toolchain, add the binaries to PATH, add pycom-esp-idf to PATH as IDF_PATH, build the mpy-cross compiler) and face the following error after make BOARD=LOPY -j5 TARGET=boot:

LINK build/LOPY_868/release/bootloader/bootloader.elf
xtensa-esp32-elf-gcc -nostdlib -Wl,-Map=build/LOPY_868/release/bootloader/bootloader.map -Wl,--no-check-sections -u call_user_start_cpu0 -Wl,-static -Wl,--undefined=uxTopUsedPriority -Wl,--gc-sections -T esp32.bootloader.ld -T esp32.rom.ld -T esp32.peripherals.ld -T esp32.bootloader.rom.ld -Wl,--start-group -Lbootloader/lib -Lbootloader -Lbuild/LOPY_868/release/bootloader -L/home/jelle/Dev/Pycom/pycom-esp-idf/components/esp32/lib -Lbootloader -llog -lcore -lbootloader_support -lspi_flash -lsoc -lmicro-ecc build/LOPY_868/release/bootloader/bootloader.a -Wl,--end-group -Wl,-EL -o build/LOPY_868/release/bootloader/bootloader.elf
/opt/espressif/esp32/bin/../lib/gcc/xtensa-esp32-elf/5.2.0/../../../../xtensa-esp32-elf/bin/ld: cannot find -lcore
collect2: error: ld returned 1 exit status
application.mk:348: recipe for target 'build/LOPY_868/release/bootloader/bootloader.elf' failed
make: *** [build/LOPY_868/release/bootloader/bootloader.elf] Error 1

Any idea? Thanks for your help.

jmarcelino

@jellium
Did you do git submodule update --init after cloning the pycom-esp-idf?

jellium

@jmarcelino Good point! Thanks, it's all good now.

peekay123

Firmware 1.7.3.b2 seems to be working well now. @robert-hh, can you share the SD test code so I can run it myself?

robert-hh

@peekay123 Hello, Here's the little script, which by the way is the one @this.wiederkehr was using, with minor tweaks.

import machine
import os
import utime
import gc

sd = machine.SD()
os.mount(sd, '/sd')

f = open('/sd/test', 'a')
t1 = utime.ticks_ms()
n = 0
while True:

#    print('about to initialize buffer')
    k = bytearray(1024)
#    print('about to write to file')
    f.write(k)
#    print('about to flush to file')
    f.flush()
    t2 = utime.ticks_ms()
#    print("I'm done ", (t2-t1))
    t1 = t2
    gc.collect()
    n += 1
    print("Round", n, "\r", end="")

I did not see that there is a version 1.7.3.b2

peekay123

@robert-hh, thanks for the code. I'll get it to work with the expansion board. I only found out about the new firmware version when I ran the updater.

peekay123

@robert-hh, made minor adjustments to get it to work with latest firmware:

import machine
import os
import utime
import gc
from machine import SD

sd = SD()
os.mount(sd, '/sd')

# check the content
os.listdir('/sd')

f = open('/sd/test', 'a')
t1 = utime.ticks_ms()
ttot = 0
n = 0
while True:

#    print('about to initialize buffer')
    k = bytearray(1024)
#    print('about to write to file')
    f.write(k)
#    print('about to flush to file')
    f.flush()
    t2 = utime.ticks_ms()
#    print("I'm done ", (t2-t1))
    ttot = (ttot + (t2 - t1)) /2
    gc.collect()
    n += 1
    print("Round", n, ", ", ttot, "ms \n\r", end="")
    t1 = utime.ticks_ms()

I calculate a running average for the write time. The code runs without errors. On the first run where the file is created, after Round 1888 the average time is 37.71667 ms. I break, hard reset and run again without erasing the file and get the same results (Round 1340 , 37.70892 ms). I noticed that the average jumps up to 46ms at time but that may be due to the gccollect() time. I adjusted the code to reset the time AFTER the gccollect and print statements (see above)

At no time did I get an error. I ran the tests with an 8GB Class 10 microSD formatted with FAT32. I am letting the test run a while to see if anything changes.

... after letting it run for a while, the output was Round 143048 , 35.17431 ms and the resulting file was 146MB in size. :)

robert-hh

@peekay123 I have seen crashes after about 10000 rounds = 10 MBytes. Just let it run. The speed is not so much in my interest. Tomorrow I'll give it a try too.

peekay123

@robert-hh, I ran 300,000 iterations without errors. I'm running it again without erasing the test file, which is at 500MB+ after the first test.

robert-hh

@peekay123 That's interesting. I never got beyond 10.000 iterations. I also tried a WiPy build on my LoPy, just in case the Lora stuff being the culprit.

robert-hh

@peekay123 @daniel I simplified the test code further, since measuring the time is not in focus, and removed the flush statement. Then it ran up to 50000 iterations, before it keeled over. The sd card was empty before, and after the crash, the file showed a size of 0, even if 50 MBytes should have been written.

import os
import machine

sd = machine.SD()
os.mount(sd, '/sd')

f = open('/sd/test', 'w')
n = 0
k = bytearray(1024)

while True:
    f.write(k)
    n += 1
    print("Round", n)

peekay123

@robert-hh, the file shows as zero size since it never gets closed/committed, thus the reason for the flush(). One oddity I did not report is that on one run of 500K, TeraTerm seems to have frozen or it was the console output on the LoPy, I can't be sure which. On my latest code, I only print a "Round" statement every 1000 iterations.

this.wiederkehr

@daniel many thanks that you did look into the issues with the sd card!

@peekay123, @robert-hh I currently don't have the possibility to test this firmware myself (eagerly looking foreward to it :-) but as I have reported on the two earlier releases I had three different issues: two different errors (load prohibited with different stack trace) and the freeze/slow down of the of the interpreter which also manifested itself in the cycle time of the loop. Actually by calling flush only each five seconds I got rid of one error and the freezes/slow down.

robert-hh

@this.wiederkehr @peekay123 I have run a few test cycles with and without flush in the loop. My impression. It stays unstable. Some results:
Passes without flush()
~50000
~300
292095
Passes with flush()
~150
~1500
541827 (!)
~9000
~4500
Timing is an influencing factor, which could lead to a stable state if found.
One final test I'm going to make is the depedence on power supply. It might be that the coincidence of an SD card write and WiFi activity causes a short power drop. For that, I'll add a stable lab power supply not going through the USB connector, but soldered directly to the board.

Explore Pybytes | Official Documentation | Report a Firmware Bug/Issue | GitHub

Firmware release 1.7.3.b1

Pycom on Twitter