New Beta Release v1.20.3.b3


  • Global Moderator

    Dear Pycom community!

    We have a new Beta release available: v1.20.3.b3.

    This firmware contains major upgrades of the IDF and Micropython and a range of changes to Pycom specific features. The IDF and Micropython are foundational pieces of our firmware. The IDF is provided by Espressif, the makers of the ESP32. Compared to the latest v1.20.2 releases this new beta firmware brings the IDF from v3.3.1 to v4.1. The Micropython version is upgraded to version 1.13.

    This is a Beta release which means, we do not advise you install this on production devices. Some reports we have received describe stability improvements, while others point to some problems which still need investigation. Some known issues are listed at the bottom.

    We would love to hear your feedback! What works better for you? What has regressed compared to the regular v1.20.2 firmware?

    Changes:

    • IDF v4.1
    • Micropython v1.13
    • LoRaWAN 1.0.3 (Class A& C)
    • Support for new regions (CN779, RU864 and KR920)
    • BLE Mesh
    • Reworked Coap with DTLS support
    • ESP-NOW
    • WiPy2, SiPy, and LoPy1 and are not supported anymore since they only have 4MB flash.
    • WiPy3, LoPy4, FiPy, GPy, W01, L01, L04 and G01 are all supported
    • improved wear levelling on LittleFS (thank you @robert-hh!)
    • bugfixes

    This firmware also contains improvements to the OTA (over the air) update functionality:

    • the main firmware partition is larger than before
    • there is a small 'factory' firmware partition, just enough for OTA
    • support for compression and
    • support for differential updates

    The documentation and infrastructure (software server, installer, etc) for these OTA features will be released separately.

    Known issues:

    • Soft reset (Ctrl-D) causes core dumps
    • No OTA support from Pybytes yet
    • No ethernet support on Pygate yet
    • Pygate on Gpy and Lopy4 seems to run out of memory. On Wipy there is just enough memory - For testing with wifi one could flash the Wipy firmware.
    • Secure boot is not supported

    You can



  • I've tested new Beta Release (again thanks for the update)

    Regarding ULP question, sadly is not included here but check out my post ULP Testing (pycom firmware with ULP support) as I've managed to "copy" micropython ULP part and integrated more or less on pycom firmware.

    Regarding LoRa DeviceTimeReq, DeviceTimeAns not tested, but checking the code I think is not supported yet.

    Side note, the Beta release requires to flash entire memory: differential update doesn't work and it complains with wrong size.

    Regards



  • @peterp said in New Beta Release v1.20.3.b3:

    Maybe some git submodule stuff?

    There are sometimes warnings when changing the branch, that directories cannot be removed because they are empty. In combination with the unneeded lib includes for -L$(ESP_IDF_COMP_PATH)/esp32/lib and -L$(ESP_IDF_COMP_PATH)/bt/libin Makefile line 182ff that could have cause the trouble for my old repository. But I do not understand why that happened too with the new cloned versions of esp-idf. Maybe some caching of paths.


  • Global Moderator

    @robert-hh said in New Beta Release v1.20.3.b3:

    @peterp said in New Beta Release v1.20.3.b3:

    Soft reset (Ctrl-D) causes core dumps

    Commenting out 'uart_deinit_all();' on line 381 in mptask.c lets the code survive soft reset.
    So maybe the delay is to short for the 'MPY:soft reboot' message to get sent.
    Yup: Increasing the delay at line 379 to 50000 does the trick.

    Awesome @robert-hh I'll check that today!


  • Global Moderator

    @robert-hh said in New Beta Release v1.20.3.b3:

    two additional lib directories in components/bt and components/esp32. But HOW did they get into the fresh cloned esp-idf? They are in the linker search directories, hmmm.

    Maybe some git submodule stuff? I recall some trouble when doing a git checkout that is "too far". E.g., git clone idf3.3.1 and then git checkout idf4.1 would lead to some messed up git submodules and it was just easier to make a fresh git clone from a branch that is just a few commits away from the IDF_HASH without any major changes to submodules and then you can git checkout IDF_HASH and have all the submodules correct.

    Did those two directories not show up with git status? I'm sure there is a " correct " way with git to do this checkout in any existing clone, but I couldn't figure it out and just settled on a new clone from a "close" branch as shown above.



  • @peterp said in New Beta Release v1.20.3.b3:

    Soft reset (Ctrl-D) causes core dumps

    Commenting out 'uart_deinit_all();' on line 381 in mptask.c lets the code survive soft reset.
    So maybe the delay is to short for the 'MPY:soft reboot' message to get sent.
    Yup: Increasing the delay at line 379 to 50000 does the trick.



  • @peterp Building in a up-to-now rarely used mint linux virtual machine worked. So if have to tell now what is interfering with the other build process.
    Edit: copying back the pycom-esp-idf directory from the virtual machine to my primary machine worked too. So: Why did the esp-idf not install properly?
    Edit2: Found the culprit: two additional lib directories in components/bt and components/esp32. But HOW did they get into the fresh cloned esp-idf? They are in the linker search directories, hmmm.



  • @DualMatic said in New Beta Release v1.20.3.b3:

    PS. cannot reply on time because not sufficient reputation

    I added a few.



  • @robert-hh said in New Beta Release v1.20.3.b3:

    P.S.: I had quite a few build systems creations in the past

    Yeah I know ;) (I'm novice user RG build enviroments) just I wanted to help you a little bit and return your past support my old username

    @robert-hh said in New Beta Release v1.20.3.b3:

    @DualMatic No Sir. No joy. The SSH key was installed. Would you mind to compare the list of submodule versions to your's?

    /esp/pycom-micropython-sigfox# git submodule status
     d270f79aa16dd8fd4ae3b6c14544283dcb992e9c lib/asf4 (heads/master)
     43a6e6bd3bbc03dc501e16b89fba0ef042ed3ea0 lib/axtls (heads/master-38-g43a6e6b)
     35aaec4418ad78628a3b935885dd189d41ce779b lib/berkeley-db-1.xx (heads/master-7-g35aaec4)
     c8b9823f68c6af0fa52e2c4e009aba4dbf257232 lib/btstack (v1.2~13^2~33)
     e9de7e35f2339598b16cbb375f9992643ed81209 lib/libffi (v3.2.1-146-ge9de7e3)
     159e31b689577dbf69cf0683bbaffbd71fa5ee10 lib/lwip (STABLE-2_1_2_RELEASE)
     3f8d78411a26e833db18d9fbde0e2f0baeda87f0 lib/mbedtls (mbedtls-2.17.0)
     97ce3eacaaa79e8ed6cf71717149ced4f5328ee7 lib/mynewt-nimble (nimble_1_3_0_tag)
     7a4c9d946cf1801771fc180acdbf7b878f270093 lib/nrfx (v2.0.0)
     b618cb1d521cc9e133bdcd0fca154dee2d925dfe lib/nxp_driver (b618cb1)
     58fee7c92bd576814d3f2afd92fbc62990270ecc lib/stm32lib (F0-1.9.0+F4-1.16.0+F7-1.7.0+H7-1.6.0+L0-1.11.2+L4-1.8.1+WB-1.6.0-19-g58fee7c)
     a6b916ba85bef6aad50f1652532b02984dfe2484 lib/tinyusb (0.5.0-680-ga6b916ba)
    
    /esp/pycom-esp-idf# git submodule status
     3b66e5b051381fb70de9c2791df70a06181c64e3 components/asio/asio (asio-1-12-0-200-g3b66e5b0)
     d037ec89546fad14b5c4d5456c2e23a71e554966 components/bootloader/subproject/components/micro-ecc/micro-ecc (v1.0)
     238fc047f2f63d99482aa51996f14ca134096310 components/bt/controller/lib (238fc04)
     fead24e5d5c0f4bd9b8c3d71cf72a87d75631399 components/bt/host/nimble/nimble (remotes/origin/master-254-gfead24e5)
     d2dd95cb8841d88d5a801e3ef9c328fd6200e7bd components/cbor/tinycbor (v0.5.2-17-gd2dd95c)
     98954eb30a2e728e172a6cd29430ae5bc999b585 components/coap/libcoap (v4.2.0-rc4-114-g98954eb)
     a17182e903a9e161a98930648eb6ff78019708ab components/esp_wifi/lib (a17182e)
     a66ee2da5ad059bb4d90e8a7317f907add1d8419 components/esptool_py/esptool (v2.8-23-ga66ee2d)
     968b8cc46dbee47b83318d5f31a8e7907199614b components/expat/expat (R_2_2_5)
     3c8935676a97c7c97bf006db8312875b4f292f6c components/json/cJSON (v1.7.12)
     70170c28c844a4786e75efc626e1aeebc93caebc components/libsodium/libsodium (1.0.12)
     50e4e0d1821a6e5deab0ead9f35ffb4e60625ea6 components/lwip/lwip (remotes/origin/master-194-g50e4e0d1)
     9ef92c551eb8d92677034c3ec8078a8076febf41 components/mbedtls/mbedtls (mbedtls-2.13.1-1358-g9ef92c551)
     6bc94add892437d0fd50f26bfabe78c646648c13 components/mqtt/esp-mqtt (ESP-MQTT_FOR_IDF_3.1-170-g6bc94ad)
     3bcc416e13cc790e2fb45fcfe9111d38609c5032 components/nghttp/nghttp2 (v1.24.0)
     dac1a65feac4ad72f612aab99f487056fbcf5c1a components/protobuf-c/protobuf-c (v1.3.0)
     f5e26c4e933189593a71c6b82cda381a7b21e41c components/spiffs/spiffs (0.2-221-gf5e26c4)
     7d2bf62b7e6afaf38153041a9d53c21aeeca9a25 components/unity/unity (v2.4.3-51-g7d2bf62)
     7e8e249990ec491ec15990cf95b6d871a66cf64a examples/build_system/cmake/import_lib/main/lib/tinyxml2 (2.0.2-659-g7e8e249)
    

    PS. cannot reply on time because not sufficient reputation



  • @DualMatic No Sir. No joy. The SSH key was installed. Would you mind to compare the list of submodule versions to your's?



  • @robert-hh you must generate an SSH key in your machine:
    Generating SSH key

    Add the key to your SSH Agent
    Add to SSH Agent

    Once generated copy and paste it on your github settings:
    Add SSH key to Github Account

    Clone again the repository with --recursive option

    Best regards, hope this is useful to you!



  • @peterp Did the same steps with the same subdir: Same output besides the final linking step.
    P.S.: I had quite a few build systems creations in the past



  • @DualMatic said in New Beta Release v1.20.3.b3:

    I had to add github SSH key from my machine to github to avoid submodules access error

    That could be the problem since all the missing links are related to wifi. How did you add the WIFi key?

    But this lib exists. Here is the list of submodules.

     3b66e5b051381fb70de9c2791df70a06181c64e3 components/asio/asio (asio-1-12-0-200-g3b66e5b0)
     d037ec89546fad14b5c4d5456c2e23a71e554966 components/bootloader/subproject/components/micro-ecc/micro-ecc (v1.0)
     238fc047f2f63d99482aa51996f14ca134096310 components/bt/controller/lib (238fc04)
     fead24e5d5c0f4bd9b8c3d71cf72a87d75631399 components/bt/host/nimble/nimble (remotes/origin/master-254-gfead24e5)
     d2dd95cb8841d88d5a801e3ef9c328fd6200e7bd components/cbor/tinycbor (v0.5.2-17-gd2dd95c)
     98954eb30a2e728e172a6cd29430ae5bc999b585 components/coap/libcoap (v4.2.0-rc4-114-g98954eb)
     a17182e903a9e161a98930648eb6ff78019708ab components/esp_wifi/lib (a17182e)
     a66ee2da5ad059bb4d90e8a7317f907add1d8419 components/esptool_py/esptool (v2.8-23-ga66ee2d)
     968b8cc46dbee47b83318d5f31a8e7907199614b components/expat/expat (R_2_2_5)
     3c8935676a97c7c97bf006db8312875b4f292f6c components/json/cJSON (v1.7.12)
     70170c28c844a4786e75efc626e1aeebc93caebc components/libsodium/libsodium (1.0.12)
     50e4e0d1821a6e5deab0ead9f35ffb4e60625ea6 components/lwip/lwip (remotes/origin/master-194-g50e4e0d1)
     9ef92c551eb8d92677034c3ec8078a8076febf41 components/mbedtls/mbedtls (mbedtls-2.13.1-1358-g9ef92c551)
     6bc94add892437d0fd50f26bfabe78c646648c13 components/mqtt/esp-mqtt (ESP-MQTT_FOR_IDF_3.1-170-g6bc94ad)
     3bcc416e13cc790e2fb45fcfe9111d38609c5032 components/nghttp/nghttp2 (v1.24.0)
     dac1a65feac4ad72f612aab99f487056fbcf5c1a components/protobuf-c/protobuf-c (v1.3.0)
     f5e26c4e933189593a71c6b82cda381a7b21e41c components/spiffs/spiffs (0.2-221-gf5e26c4)
     7d2bf62b7e6afaf38153041a9d53c21aeeca9a25 components/unity/unity (v2.4.3-51-g7d2bf62)
     7e8e249990ec491ec15990cf95b6d871a66cf64a examples/build_system/cmake/import_lib/main/lib/tinyxml2 (2.0.2-659-g7e8e249)
    


  • @robert-hh For me is compiling okay. I started under a fresh debian virtual machine BTW.

    Just two warnings:

    In file included from main.c:54:
    /home/esp/pycom-esp-idf/components/esp_event/include/esp_event_loop.h:2:2: warning: #warning "esp_event_loop.h is deprecated, please include esp_event.h instead" [-Wcpp]
     #warning "esp_event_loop.h is deprecated, please include esp_event.h instead"
      ^~~~~~~
    main.c: In function 'app_main':
    main.c:140:5: warning: 'esp_event_loop_init' is deprecated [-Wdeprecated-declarations]
         ESP_ERROR_CHECK(esp_event_loop_init(app_sys_event_handler, NULL));
         ^~~~~~~~~~~~~~~
    In file included from /home/esp/pycom-esp-idf/components/esp_event/include/esp_event.h:26,
                     from /home/esp/pycom-esp-idf/components/esp_event/include/esp_event_loop.h:3,
                     from main.c:54:
    /home/esp/pycom-esp-idf/components/esp_event/include/esp_event_legacy.h:227:11: note: declared here
     esp_err_t esp_event_loop_init(system_event_cb_t cb, void *ctx) __attribute__ ((deprecated));
    
    
    mods/modnetwork.c: In function 'network_select_nic':
    mods/modnetwork.c:217:18: warning: assignment discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers]
             nic_type = mp_obj_get_type(nic_chosen);
    

    I had to add github SSH key from my machine to github to avoid submodules access error, maybe this is your problem?

    Please make sure you have the correct access rights
    and the repository exists.
    fatal: clone of 'git@github.com:espressif/esp32-wifi-lib.git' into submodule path '/home/esp/pycom-esp-idf/components/esp_wifi/lib' failed
    Failed to clone 'components/esp_wifi/lib'. Retry scheduled
    

    BR



  • @peterp I did the git submodule updates & everything. Made a new clone of the micropython and esp-idf directories. No luck.



  • Hello @peterp

    Regarding:

    • Micropython v1.13

      • It's ULP implemented from original micropython? I want to do some tests or figure out ULP processor, since I'm waiting Pycom to support in the firmware.
        .
    • LoRaWAN 1.0.3 (Class A& C)

      • What are the improvements from version 1.0.2. I'm looking if DeviceTimeReq, DeviceTimeAns is implemented on Pycom.

    Thanks in advance and good looking release!! ;)


  • Global Moderator

    it works for me. please do check the git submodule update --init --recursive

    my test:

    mkdir ~/build-check
    cd ~/build-check
    
    git clone --recursive -b idf_v4.1 https://github.com/pycom/pycom-esp-idf.git
    cd pycom-esp-idf
    git checkout d647204 && git submodule sync && git submodule update --init --recursive
    cd ..
    
    git clone --recursive -b master_idf_v4.1 https://github.com/pycom/pycom-micropython-sigfox.git
    export IDF_PATH=~/build-check/pycom-esp-idf/
    cd pycom-micropython-sigfox/
    cd mpy-cross
    make
    cd ..
    
    cd esp32
    xtensa-esp32-elf-gcc --version
    xtensa-esp32-elf-gcc (crosstool-NG esp-2020r3) 8.4.0
    make BOARD=LOPY4
    
    ...
    CC build/LOPY4/release/pins.c
    AR build/LOPY4/release/application.a
    CPP build/LOPY4/release/esp32_out.ld
    LINK build/LOPY4/release/application.elf
    xtensa-esp32-elf-size build/LOPY4/release/application.elf
       text	   data	    bss	    dec	    hex	filename
    1727558	 283960	  87726	2099244	 20082c	build/LOPY4/release/application.elf
    Building partitions from lib/partitions_4MB.csv...
    Building partitions from lib/partitions_8MB_small_factory_fw.csv...
    IMAGE build/LOPY4/release/lopy4.bin
    esptool.py v2.9-dev
    
    


  • @peterp Git status in the source directory:

    Auf Branch master_idf_v4.1
    Ihr Branch ist auf demselben Stand wie 'upstream/master_idf_v4.1'.
    

    git status in pycom-esp-idf:

    HEAD losgelöst bei d647204d1
    

    I did not make a new close, but checked out in the existing directories. I can clone new copies.
    I just call make, which calls a GNUmakefile first, which sets LoPy4 as default device and sets PATH and IDF_PATH.


  • Global Moderator

    @robert-hh

    git status ; git submodule update --init --recursive
    in both folders please - that is, assuming you did get the code via git clone, yes?!

    what make command you use to build? I can try a new clone ... but that's what Jenkins did anyway, well, I'm trying ...



  • @peterp There is a toolchain and idf version check in Makefile. They pass. And if you look at the short excerpt, you see xtensa-esp32-elf-gcc8_4_0-esp-2020r3-linux-amd64.


Log in to reply
 

Pycom on Twitter