Firmware Release v1.20.1

[iwahdan]

Hello everyone, we have release a new Firmware release v1.20.1

sources can be found on Github Here

Release Notes:

​

• SmartConfig (Smart provisioning of Wifi Credentials)
  • To disable Smart Provisioning use pybytes.smart_config(False)
• IDF V3.2
• Micropython V1.11
• Secure BLE (Server side )
• Pybytes v1.1.2
• LittleFS v2.0
• Light sleep support
• RTC memory support
• Wifi promiscuous mode
• CoAp module
• Esp32 touchpad functionality exposed to micropy
• uasyncio module
• Uzlib module
• Add support for LoRa regions CN470 and IN865
• Added support for LoRa Class C multicast
• Increase App partition Size For FIPy, GPy and LoPy4 to 1980K
• Updated Build ENV for detecting Flash Size
• Update Make Environment for IDF version check and checkout
• Support OTA from older v1.18 Firmwares through intermediate firmware v1.18.3
• Updates / Bug fixes to Pycom module
• Updates / bug fixes to UART module
• Updates to machrmt module
• Updates / bug fixes to machrtc module
• Updates / Bug fixes to modlora
• Updates / Bug Fixes to Bluetooth module
• Updates / Bug fixes to WLAN module
• Updates / Bug fixes to LTE module
• Updates / Bug fixes to machine module
• Updates / bug fixing for OS module
• Updates / Bug fixing for modusocket module
• General fixes and improvements to the LoRaWAN stack
• General Makefile Updates
• General Bug Fixing
  ​

Here is the summary of changes which can cause compatibility issues updating from 1.18 FW along with new APIs added

​

machrmt:

• "pulses_send()" API can be configured as non-blocking
  "wait_tx_done" parameter is added
  ​

machrtc:

• ntp_sync() API has new parameter: "backup_server" -> not mandatory
  memory() API is added
  ​

machuart:

• RX Buffer length can be maximum of 512 byte on all boards
  • Previously it was 4096 bytes on FIPY and GPY
• New parameter of UART constructor:
  • rx_buffer_size
  • New Exception is dropped if "rx_buffer_size" is bigger than 512 bytes
    ​

modlora:

• New API: LoRa.reset() to reset sx1272 chip on LoPy1 Boards
  ​

modbt:

• Bluetooth's constructor has new parameter: "modem_sleep”, “mtu”
  • Exception is dropped if sleep failed
• New API of characteristic class: read_descriptor()
• Bluetooth.deinit() closes all connections
• Bluetooth.connect() from now drops TimeoutError and not OSError when timeout happens
• New parameter of Bluetooth.connect()
  • Has a "timeout" parameter defining timeout for establishing connection
  • Previously it was set to 5ms
• New API: Bluetooth.set_advertisement_params()
• New API: Bluetooth.set_advertisement_raw()
• New API: Bluetooth.modem_sleep()
• New API: Bluetooth.gatts_mtu() to get mtu of the client connection to Pycom device
• New API: Bluetooth.tx_power()
  ​

modcoap:

New module
​

modled:

• set_color() has new parameter: "wait_tx"
  • Can be used to make underlying RMT operation block
    ​

modlte:

• lte_iccid() API drops new OSError exception if SIM is not detected
• send_at_cmd() API's "delay" parameter is removed
• send_at_cmd() raises new TypeError exception if "cmd" is not passed
• New Callback registering API for registering a callback function to be called when LTE coverage is Lost (Note: Depending on the Sequans modem FW it might be supported or not , if not the callback will not be triggered - an update on Sequans FW supporting this Feature will be announced later )
  ​

modmachine:

• deepsleep_wakeup() API is renamed to sleep_wakeup
• sleep() API is implemented for light sleep (it was an empty function)
  ​
  ​

modpycom:

• New API: bootmgr()
• New API: get_free_heap()
• New APIs: related to Pybytes
• New APIs for setting/getting stored Wifi Credentials for STA and AP mode
• heartbeat() API drops new OSError excpetion if RGB_LED is disabled (can be disabled during build)
• rgb_led() API drops new OSError excpetion if RGB_LED is disabled (can be disabled during build)
• nvs_get() API returns user defined "NoExistValue" if the object does not exist in NVS. Otherwise it raises a ValueError!.
  * To maintain previous behaviour, use pycom.nvs_get('<item>', None)
  ​

moduos:

• New file system type is added: Little FS
• Virtual File System concept is used
  • The /flash can be LittleFS or FatFS, the SD Card can be FatFS
  • Files System related functions are replaced, they are used via VFS
• mkfs() API is removed and is replaced by fsformat()
  ​

moduqueue:

• queue_put() releases the GIL during operation -> REPL will work
• queue_get() releases the GIL during operation -> REPL will work
  ​

modusocket:

• readall() API is removed
  • read() without parameters does the same
• connect() API can be set as Non-BLocking
  • Only if timeout is defined: new OSError exception is dropped if settimeout operation of lwip fails
  • Only if timeout is defined: new TimeoutError exception is dropped if connection times out
  • Only if timeout is defined: new OsError exception may dropped if SSL is configured and underlying ssl operation fails
• recv() API drops TimeoutError exception if underlying lwip returns with SSL_TIMEOUT
  • Before it only dropped this exception if MP_EAGAIN is returned
• recvfrom() API drops TimeoutError exception if underlying lwip returns with SSL_TIMEOUT
  • Before it only dropped this exception if MP_EAGAIN is returned
• New API: sendto()
  ​

modutime:

• Parameters of ticks_diff() API are swapped
  • Was: start_in, end_in
  • New: end_in, start_in
    ​

modwlan:

• New API: bandwidth()
• New API: hostname()
• New API: ap_sta_list()
• New API: max_tx_power()
• New API: country()
• New API: joined_ap_info()
• New API: wifi_protocol()
• New API: send_raw()
• New API: promiscuous()
• New API: callback()
• New API: events()
• New API: smartConfig()
• New API: Connected_ap_pwd
• New API: wifi_packet()
• New API: ctrl_pkt_filter()
• WLAN's contructor/init():
  • New optional parameter: bandwidth
  • New optional parameter: max_tx_pwr
  • New optional parameter: country -> if not given as tuple TypeError is dropped
• API scan() has new optional parameters:
  • ssid, bssid, channel, show_hidden, type, scantime -> drops exception if given incorrectly
• API mac() may drop new exceptions depends on number of arguments and it returns Mac of both AP and STA
• New Triggers for Wlan Callback

MQTT aws lib

• Removed from frozen code but can be found on Github Here

Please Note Documentation for v1.20 FW is still being updated and can be found Here

Issue submission and Collaboration:

For better experience and support for Pycom community, we have restructured our GitHub public Repo to have a separate branches for 1.18 and 1.20 FWs (under Release) - Daily development will be done on a new default branch (Dev) (previously we used to do the main Dev privately and do weekly/monthly releases to public repo).
This will make it easier for people to Contribute to Our Firmware plus better traceability of opened Issues.
we recommend you open any pull request on that Branch (Dev).
​

OTA to 1.20 FW from 1.18

Between version 1.18.2 and 1.20 the partition table is changed, the OTA_0 partition, among several others, is relocated because the size of the firmware has been increased due to the extended functionality of the 1.20 version. Due to this reason when updating from version 1.18.2 to 1.20 via OTA, an update to an intermediate version, numbered as 1.18.3, must be performed.
The intermediate version makes sure that when OTA update to 1.20 is requested, the partition table contains correct entries and the new image will be flashed to the proper location.

The intermediate version, 1.18.3, contains the following EXTRA steps when OTA update is requested:

• Updates bootloader
• Moves the content of the "ota data" partition to its new location
• Updates the partition table

Note: Please note that if bootloader or partition table update fails (e.g.: due to a power-loss), the device will not boot and recover and a new image must be flashed via a cable.

When the device is being updated to the 1.18.3 intermediate version it may be flashed onto "OTA_0" partition depending on the where current active image is loaded from. In this case on next boot it automatically copies itself from "OTA_0" to "Factory" partition as a preparation for being able to receive the 1.20 image in the future. Without this step when the 1.20 image is being flashed on the "Factory" partition due to its increased size it would overwrite the first segments of the curently running firmware (which is the 1.18.3 and located on "OTA_0" partition).

Therefore the correct update chain from 1.18 to 1.20 via OTA looks like: 1.18.2 -> 1.18.3 -> 1.20

Downgarding from 1.20 back to 1.18 is possible, however the target version must be 1.18.3. The 1.18.3 image able to detect whether a downgrade happened and in this case it will not perform the steps explained above, it behaves as the 1.18.2r7 image just with updated partition table and bootloader. Updating back again to 1.20 is possible and allowed.

Note: Updating devices with encrypted flash is not supported.

Note: Downgrading via OTA from 1.18.3 to 1.18.2.xx is not allowed as this might casuse further OTA operations to fail.

Note: Before doing OTA to 1.20 firmware make sure that the Scripts on device are updated (if necessary) to be compatible with 1.20 FW to avoid exceptions once device is updated to 1.20 which might lead to loosing connection to the remote device.

You can find firmware images for 1.18.3 here:

Wipy
FiPy
GPy
LoPy
LoPy4
SiPy

Note: This new Release has support for LittleFS filesystem if you updated from 1.18 FW your Filesystem is set to FatFS and if you choose to switch to LittleFS this will cause the filesystem to be formatted .

Note: previous stable and development firmware can now be found as “legacy” in the firmware updater, so the new firmware updater has the 3 firmware types pybytes, legacy and pybytes-legacy

Note: Upgrading from legacy (1.18) FW also downgrading from 1.20 to 1.18 will cause Nvs data to be erased as formatting under new FW is updated

Note: There is an issue regarding BLE connection from Pycom Device (client) to a mobile phone (tested on Iphone) acting as Server, where connection always times out - connection from Pycom Device to another does not have that issue - will keep you updated with the fix for that problem.

[robert-hh]

@iwahdan Hellol Islam. Thank you for the update. I have a few question and comments:

1. Why was the update pushed to the branch called Release/v1.20.1 and not master. These many new branches confuse.
2. The bug in the LTE module I reported in Issue #338 still persists, but may not always pop up:
   In handling of the attach command inside modlte.c, the length of the command AT+SMDD (and others) is not set! Lines affected: 310, 654, 670. Compare to these place the use of the function lteppp_send_at_command in line 179, where the data length IS set. A test confirmed the bug: if I specify band=8, I get an error: band 8 not supported. If I omit band=8, attach works.
3. There is still the glitch in handling the bits=xx argument in adc.init. Before, the only accepted argument & value for adc.init() was bits=12, and even that was discarded. With that change, the function meets the documentation and works as expected. See PR #327

Edit: Setting dataLen in the calls to lteppp_send_at_command() makes attach wotk again with specifying the band.
Edit2: Thank you for keeping the espressif reset mode option

[iwahdan]

@robert-hh said in Firmware Release v1.20.1:

@iwahdan Hellol Islam. Thank you for the update. I have a few question and comments:

1. Why was the update pushed to the branch called Release/v1.20.1 and not master. These many new branches confuse.

we have restructured our GitHub public Repo to have a separate branches for 1.18 and 1.20 FWs (under Release) - Daily development will be done on a new default branch (Dev) (previously we used to do the main Dev privately and do weekly/monthly releases to public repo).
This will make it easier for people to Contribute to Our Firmware plus better traceability of opened Issues.
we recommend you open any pull request on that Branch (Dev).

2. The bug in the LTE module I reported in Issue #338 still persists, but may not always pop up:
   In handling of the attach command inside modlte.c, the length of the command AT+SMDD (and others) is not set! Lines affected: 310, 654, 670. Compare to these place the use of the function lteppp_send_at_command in line 179, where the data length IS set. A test confirmed the bug: if I specify band=8, I get an error: band 8 not supported. If I omit band=8, attach works.

Can you open a PR on Dev branch?

3. There is still the glitch in handling the bits=xx argument in adc.init. Before, the only accepted argument & value for adc.init() was bits=12, and even that was discarded. With that change, the function meets the documentation and works as expected. See PR #327

Just changed Base of your PR to Dev, can you please update?

Edit: Setting dataLen in the calls to lteppp_send_at_command() makes attach wotk again with specifying the band.

[rskoniec]

@iwahdan I'm not able to change pycom.wifi_on_boot

>>> os.uname()
(sysname='FiPy', nodename='FiPy', release='1.20.1', version='v1.11-12f4ce0 on 2019-10-06', machine='FiPy with ESP32', lorawan='1.0.2', sigfox='1.0.1', pybytes='1.1.2')
>>> import pycom
>>> pycom.wifi_on_boot()
False
>>> pycom.wifi_on_boot(True)
>>> pycom.wifi_on_boot()
False

[robert-hh]

@rskoniec said in Firmware Release v1.20.1:

pycom.wifi_on_boot(True)

Does it change if you use:

pycom.wifi_on_boot(True, True)

The code allows a second parameter, which is not documented. But as I understand it, it forces the immediate startup of WiFi. It should not affect the stored state.

[robert-hh]

@rskoniec Do you use the PyBytes variant? According to the code this variant always returns False (and it does not set this flag). See the code snippet from pycom_config.c:

bool config_set_wifi_on_boot (uint8_t wifi_on_boot) {
    if (pycom_config_block.wifi_config.wifi_on_boot != wifi_on_boot) {
        pycom_config_block.wifi_config.wifi_on_boot = wifi_on_boot;
#if (VARIANT != PYBYTES)
        return config_write();
#endif
    }
    return true;
}

bool config_get_wifi_on_boot (void) {
#if (VARIANT == PYBYTES)
    return false;
#endif
    return pycom_config_block.wifi_config.wifi_on_boot;
}

[River]

Hello @iwahdan, promissing update! Thanks to all people involved.

I will test new firmware flashing a module later today. I have two questions regarding this update:

1. It's now ULP co-processor supported? We have a product with custom hardware + lopy4 and we want to add capability to count pulses on two digital inputs while sleeping (measuring water consumption, etc..)
2. It's possible to change wifi mode to LR (Espressif's Long Range protocol) by code?

If still not supported (1 or 2) I don't know if may open an issue on github in order to pycom keep track of improvements, or maybe is better to leave written here.

Regards

[cmisztur]

Thank you.

Regarding 'legacy' vs 'pybytes' variants. Does the PyBytes variant auto connect to PyBytes platform and maintain connection? What is the difference between 'legacy' and 'pybytes'?

[robert-hh]

@cmisztur Sorry, I never looked at pybytes.

[cmisztur]

Breaking change for me from 1.20.0.rc13 to 1.20.1 is usage of machine.idle().
Below, line 7 is never reached.

import utime
import _thread

def t():
    print('thread start')
    utime.sleep_ms(4000)
    print('thread run')

_thread.stack_size(8192)
_thread.start_new_thread(t, ())

utime.sleep_ms(1000)

while 1:
    machine.idle()
    #utime.sleep_ms(50)

[cmisztur]

@robert-hh said in Firmware Release v1.20.1:

@cmisztur Sorry, I never looked at pybytes.

How are you flashing 1.20.1?

[robert-hh]

@cmisztur I build the image myself, and as part of that, i can call:

make BOARD=xxPy flash

There are also the pyupgrade tools, you can download from the pycom web site.

[robert-hh]

@cmisztur That finding with idle is strange. Comparing the Pycom variant with the Micropython.org shows no difference in the machine.idle() implementation. But the Micropython.org version works. That's how the function looks in both variants:

STATIC mp_obj_t machine_idle(void) {
    taskYIELD();
    return mp_const_none;
}

[robert-hh]

@cmisztur I compared the related functions in machine.idle() and utime.sleep_ms() for both v1.20.0.rc13 and v1.20.1. They are identical, and just call taskYIELD() (idle) resp. vTaskDelay() (sleep_ms). So the change must be cause by moving from esp_idf v3.1 to esp_if v3.2.

[Clemens]

I have some devices with firmware 1.20.0.rc13 before and now 1.20.1. Unfortunately the old software is not running with the new firmware and I got e.g.

File "main.py", line 11, in <module>
File "webserver.py", line 124, in <module>
File "/flash/lib/microDNSSrv.py", line 19, in Create
File "/flash/lib/microDNSSrv.py", line 161, in Start
OSError: Network card not available

or

File "/flash/lib/terkin/network/core.py", line 110, in start_services
File "/flash/lib/terkin/network/core.py", line 131, in start_httpserver
File "/flash/lib/terkin/api/http.py", line 70, in start
File "/flash/dist-packages/microWebSrv.py", line 219, in Start
OSError: Network card not available

and my former working OTA-Code is stopping also

  File "main.py", line 44, in <module>
  File "/flash/lib/OTA.py", line 176, in connect
NameError: name 'SSID' isn't defined

[robert-hh]

@Clemens I had a similar problem too on one Lopy4 device - Wifi was not available. Erasing the flash and reloading the firmware solved it.

[Clemens]

@robert-hh said in Firmware Release v1.20.1:

pycom.wifi_on_boot(True, True)

>>> pycom.wifi_on_boot()
False
>>> pycom.wifi_on_boot(True, True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: the requested operation failed

[robert-hh]

@Clemens That wifi_on_boot() issue is a different topic and has been fixed with today's build for the Dev branch. AFIK it is not yet in the 'official' version.
Edit: It looks like there is a lag in the Pycom update. The updater does not show V1.20.1, and the download page for firmware packages is also not updated.

[rcolistete]

Please update the links for all Pycom devices in :
https://docs.pycom.io/advance/downgrade/
to include the version 1.20.1 for manual firmware download.

[Clemens]

One software is after the repeated firmware update with the checked option erase filesystem now running. The second one outputs still the same error.