Some questions on Pybytes release deployment

[Oliver Mayer]

Re: Deploy release via pybytes

Hi,
I am testing the deployment of releases via Pybytes to some FiPys on firmware 1.20.2.R4, but cannot get it to work. After the 'your release has been deployed' confirmation message nothing happens, just like in the referenced topic. I even tried the my-release.zip from the examples with no success.

import pycom; pycom.nvs_set('pybytes_debug', 99)

does not show any information about release deployment.

A partner of ours is interested in the release deployment feature for an production environment. Thus I'd be very grateful if anyone from Pycom ( @Gijs ?) could clarify the following questions:

• Do you plan on releasing a stable version of the release deployment feature in Pybytes? If so, when?
• Could you provide us the constraints on the release zip file for successful deployment?
• Are there ways to debug the problems with release deployment? Do you plan on providing error messages for debugging at some point? If so, when?

[ahmadelmasri]

Hi @Oliver-Mayer,

Thank you for your feedback.

Pybytes team will take a look, and see why the release feature is not working for you.

I will get back to answer your questions.

Best Regards,
Ahmad EL Masri

[Alexander Pilhar]

I have the same problem (GPy 1.0, Expansionboard 3.1, FW 1.20.2r4, PyBytes 1.6.1).
How long does it usually take until the device knows something about the new deployment?

[peterp]

@Alexander-Pilhar over wifi it should be more or less immediately. Please turn on debug like this:

import pycom; pycom.nvs_set('pybytes_debug', 99)

then reset and try again and paste the output. Please check and remove passwords before posting

[Alexander Pilhar]

@peterp here you go:

>>> import pycom
>
>>>> pycom.nvs_set('pybytes_debug',99)
>
>>>> ets Jun  8 2016 00:22:57
>
>
>
>rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
>
>configsip: 0, SPIWP:0xee
>
>clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
>
>mode:DIO, clock div:1
>
>load:0x3fff8020,len:8
>
>load:0x3fff8028,len:2128
>
>load:0x4009fa00,len:19760
>
>entry 0x400a05bc
>
>{'wifi': {'ssid': 'REAP01_B827EB8C1B3B', 'password': '##############'}, 'pybytes_autostart': True, 'username': 'alexander.pilhar@#######.##', 'cfg_msg': 'Pybytes configuration read from /flash/pybytes_config.json', 'server': 'mqtt.pybytes.pycom.io', 'lte': {'apn': None, 'cid': None, 'reset': False, 'carrier': None, 'band': None, 'type': None}, 'ssl': False, 'device_id': '########-####-####-####-############', 'network_preferences': ['wifi'], 'dump_ca': False, 'wlan_antenna': 0, 'ota_server': {'port': 443, 'domain': 'software.pycom.io'}}
>
> ssl_params={'ca_certs': '/flash/cert/pycom-ca.pem'}
>
>LIS2HH12 not imported
>
> ssl_params={'ca_certs': '/flash/cert/pycom-ca.pem'}
>
>WMAC: 30AEA42CD8FC
>
>Firmware: 1.20.2.r4
>
>Pybytes: 1.6.1
>
>{'wifi': {'ssid': 'REAP01_B827EB8C1B3B', 'password': '##############'}, 'pybytes_autostart': True, 'username': 'alexander.pilhar@#######.##', 'cfg_msg': 'Pybytes configuration read from /flash/pybytes_config.json', 'server': 'mqtt.pybytes.pycom.io', 'lte': {'apn': None, 'cid': None, 'reset': False, 'carrier': None, 'band': None, 'type': None}, 'ssl': False, 'device_id': '########-####-####-####-############', 'network_preferences': ['wifi'], 'dump_ca': False, 'wlan_antenna': 0, 'ota_server': {'port': 443, 'domain': 'software.pycom.io'}}
>
>Attempting to connect with network wifi
>
>Initialized watchdog for WiFi and LTE connection with timeout 1260000 ms
>
>WLAN connected? False
>
>Wifi connection attempt: 1
>
>WLAN connected? False
>
>(ssid='REAP01_B827EB8C1B3B', bssid=b"\xba'\xeb\x8c\x13;", sec=3, channel=11, rssi=-52)
>
>(ssid='#######-WLAN', bssid=b'H\x8fZ\xc8k\xbf', sec=5, channel=8, rssi=-59)
>
>(ssid='MikroTik-57007C', bssid=b',\xc8\x1bW\x00|', sec=4, channel=1, rssi=-82)
>
>(ssid='#######-WLAN', bssid=b'H\x8fZ\xc8Y]', sec=5, channel=9, rssi=-84)
>
>Connecting with REAP01_B827EB8C1B3B and ##############
>
>WiFi connection established
>
>MQTT Protocol
>
>Packet sent. (Length: 117)
>
>This is PybytesProtocol.start_MQTT
>
>Packet sent. (Length: 44)
>
>Connected to MQTT mqtt.pybytes.pycom.io
>
>Pybytes connected successfully (using the built-in pybytes library)
>
>This is pack_info_message()
>
>__pack_message: b'010504a00584000000'
>
>MQTT Protocol
>
>Packet sent. (Length: 50)
>
>Pybytes configuration read from /flash/pybytes_config.json
>
>Pycom MicroPython 1.20.2.r4 [v1.11-ffb0e1c] on 2021-01-12; GPy with ESP32
>
>Pybytes Version: 1.6.1
>
>Type "help()" for more information.
>
>>>> This is PybytesProtocol.__keep_connection(alarm=<Alarm>)

[peterp]

@Alexander-Pilhar mhm. that's not much. this is when you try to deploy? does the device show as online on pybytes? can you send_signal from the device and do they reach the website? does online pymakr work?

[Alexander Pilhar]

@peterp sorry, didn't start the deployment on first one. Here is the new output:

AutoConnect enabled, ignoring 'address' setting (see Global Settings)
Searching for PyCom boards on serial...
Connecting to COM11...

>>> import pycom
>>> pycom.nvs_set('pybytes_debug',99)
>>> ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff8020,len:8
load:0x3fff8028,len:2128
load:0x4009fa00,len:19760
entry 0x400a05bc
{'wifi': {'ssid': 'REAP01_B827EB8C1B3B', 'password': '##############'}, 'pybytes_autostart': True, 'username': 'alexander.pilhar@#######.##', 'cfg_msg': 'Pybytes configuration read from /flash/pybytes_config.json', 'server': 'mqtt.pybytes.pycom.io', 'lte': {'apn': None, 'cid': None, 'reset': False, 'carrier': None, 'band': None, 'type': None}, 'ssl': False, 'device_id': '########-####-####-####-############', 'network_preferences': ['wifi'], 'dump_ca': False, 'wlan_antenna': 0, 'ota_server': {'port': 443, 'domain': 'software.pycom.io'}}
 ssl_params={'ca_certs': '/flash/cert/pycom-ca.pem'}
LIS2HH12 not imported
 ssl_params={'ca_certs': '/flash/cert/pycom-ca.pem'}
WMAC: 30AEA42CD8FC
Firmware: 1.20.2.r4
Pybytes: 1.6.1
{'wifi': {'ssid': 'REAP01_B827EB8C1B3B', 'password': '##############'}, 'pybytes_autostart': True, 'username': 'alexander.pilhar@#######.##', 'cfg_msg': 'Pybytes configuration read from /flash/pybytes_config.json', 'server': 'mqtt.pybytes.pycom.io', 'lte': {'apn': None, 'cid': None, 'reset': False, 'carrier': None, 'band': None, 'type': None}, 'ssl': False, 'device_id': '########-####-####-####-############', 'network_preferences': ['wifi'], 'dump_ca': False, 'wlan_antenna': 0, 'ota_server': {'port': 443, 'domain': 'software.pycom.io'}}
Attempting to connect with network wifi
Initialized watchdog for WiFi and LTE connection with timeout 1260000 ms
WLAN connected? False     
Wifi connection attempt: 1
WLAN connected? False     
(ssid='REAP01_B827EB8C1B3B', bssid=b"\xba'\xeb\x8c\x13;", sec=3, channel=11, rssi=-59)
(ssid='#######-WLAN', bssid=b'H\x8fZ\xc8k\xbf', sec=5, channel=8, rssi=-63)   
(ssid='MikroTik-57007C', bssid=b',\xc8\x1bW\x00|', sec=4, channel=1, rssi=-83)
(ssid='#######-WLAN', bssid=b'H\x8fZ\xc8Y]', sec=5, channel=9, rssi=-87)      
(ssid='UPCA61872A', bssid=b'\xac"\x05\xe9<^', sec=4, channel=11, rssi=-90)    
(ssid='UPC Wi-Free', bssid=b'6,\x94:\xb7N', sec=5, channel=6, rssi=-91)       
(ssid='UPC Wi-Free', bssid=b'\xae"\x15\xe9<^', sec=5, channel=11, rssi=-91)   
Connecting with REAP01_B827EB8C1B3B and ##############
WiFi connection established
MQTT Protocol
Packet sent. (Length: 117)
This is PybytesProtocol.start_MQTT
Packet sent. (Length: 44)
Connected to MQTT mqtt.pybytes.pycom.io
Pybytes connected successfully (using the built-in pybytes library)
This is pack_info_message()
__pack_message: b'010504a00584000000'
MQTT Protocol
Packet sent. (Length: 50)
Pybytes configuration read from /flash/pybytes_config.json
Pycom MicroPython 1.20.2.r4 [v1.11-ffb0e1c] on 2021-01-12; GPy with ESP32
Pybytes Version: 1.6.1
Type "help()" for more information.
>>> This is PybytesProtocol.__process_recv_message()
This is PybytesLibrary.unpack_message(message=b'\n{"releaseId":"2b9b8bfd-879b-495a-9abe-dcc5b2adaaf4","applicationId":"615d87aa5efa203f90a7db0d"}')
header: 10 body: b'{"releaseId":"2b9b8bfd-879b-495a-9abe-dcc5b2adaaf4","applicationId":"615d87aa5efa203f90a7db0d"}'
network_type: 0
message_type: 10
body {'applicationId': '615d87aa5efa203f90a7db0d', 'releaseId': '2b9b8bfd-879b-495a-9abe-dcc5b2adaaf4'}
manifest URL: https://pyconfig.eu-central-1.elasticbeanstalk.com/manifest.json?app_id=615d87aa5efa203f90a7db0d&target_ver=2b9b8bfd-879b-495a-9abe-dcc5b2adaaf4&current_ver=None&device_id=########-####-####-####-############
letResp: {'networkConfig': {'wifi': {'ssid': '#######-Airport', 'password': '##############', 'secureType': 'PASSWORD'}, 'networkPreferences': ['wifi']}, 'target_version': {'id': '615d8ddd712ec5523be5496f', 'codeFileName': '2b9b8bfd-879b-495a-9abe-dcc5b2adaaf4', 'version': 3}, 'newFiles': [{'fileName': '/pycom-pybytes/boot.py', 'checksums': 'd25486dc397cd4831a05df8438a78385f03afc1bea8263b283a2accde6e12b59'}, {'fileName': '/pycom-pybytes/main.py', 'checksums': '9b90635754cfc8a3366a7aa5beecd9bc87f4b9b988291b723a0631038c8c8640'}, {'fileName': '/pycom-pybytes/pybytes_config.json', 'checksums': 'e1e0552ed695b7a15a32fa9f4435855fb0b76f8af04d067f851a01d487154ec5'}, {'fileName': '/pycom-pybytes/lib/rgbled_client.py', 'checksums': 'aa606bda613daa7063c515f911253414f70d5ca0cfa258bbdef2ab7a7340c3f3'}], 'updatedFiles': [], 'firmware': {'version': '1.20.2.r4'}, 'deletedFiles': []}
Updating file [/pycom-pybytes/boot.py]
Unhandled exception in thread started by <bound_method>
Traceback (most recent call last):
  File "_msg_handl.py", line 259, in _io_thread_func
  File "_msg_handl.py", line 197, in _receive_packet
  File "_mqtt_core.py", line 398, in _recv_callback
  File "_mqtt_core.py", line 368, in _parse_publish
  File "_mqtt_core.py", line 341, in _notify_message
  File "_mqtt.py", line 91, in set_callback
  File "_pybytes_protocol.py", line 184, in __process_recv_message
  File "_pybytes_protocol.py", line 833, in deploy_new_release
  File "_pybytes_protocol.py", line 718, in update_files
  File "_flash_control_OTA.py", line 39, in update_file_content
OSError: [Errno 19] ENODEV```

[peterp]

Sorry for the long wait @Alexander-Pilhar . I've now tried to upload using both of the zip files you shared and it fails, but with a different error:

network_type: 0
message_type: 6
receiving FCOTA request
Unhandled exception in thread started by <bound_method>
Traceback (most recent call last):
  File "_msg_handl.py", line 259, in _io_thread_func
  File "_msg_handl.py", line 197, in _receive_packet
  File "_mqtt_core.py", line 398, in _recv_callback
  File "_mqtt_core.py", line 368, in _parse_publish
  File "_mqtt_core.py", line 341, in _notify_message
  File "_mqtt.py", line 91, in set_callback
  File "_pybytes_protocol.py", line 286, in __process_recv_message
UnicodeError: 

and indeed when I open the files inside the zip I find some unreadable data. I think the files are somehow corrupted

[Alexander Pilhar]

@peterp thank you for your response and for finding this weird problem with this file included in the .zip!
I will resolve this and try again soon!

[peterp]

Now I've also managed to reproduce the

  File "_flash_control_OTA.py", line 39, in update_file_content
OSError: [Errno 19] ENODEV

error. The project release feature expects the zip file to have a single top level folder named "flash" and it doesn't handle it gracefully when that is not the case.

[Alexander Pilhar]

@peterp thank you very much for your help!
Simply renamed the root folder to "flash" and now it is working as expected!

[khtmhai khtmhai]

I have the same problem (GPy 1.0, Expansionboard 3.1, FW 1.20.2r4, PyBytes 1.6.1).