GPy rebooting because of Pybytes watchdog?
-
I have a GPy that is currently connected via wifi to my home network. The wifi network is stable as far as I can tell, but it seems like the GPy is failing regularly to connect to the Pybytes MQTT broker and reboots the device.
Is there a way to disable this (rebooting when Pybytes fails to connect via MQTT)?Below is the log output with
pycom.nvs_set('pybytes_debug', 99)
. It shows that first theProtocol.__keep_connection(alarm=<Alarm>)
calls succeed, but then fail withSocket send error [Errno 104] ECONNRESET
. Once this happens the GPy reboots, and it happens regularly (sometimes every 10 minutes, sometimes every 20 or 30 minutes, but usually multiple times per hour.Checking the content of
machine.reset_cause()
returns2
, which according to the documentation meansmachine.WDT_RESET: Reset by watchdog timer
. I'm not using a custom watchdogmachine.WDT
, so I assume it is an internal Pybytes one.All lines starting with
[INFO]
are my own log output, and I set a simpleTimer.Alarm
to output the current time once per minute to keep track of when this issue happens.AutoConnect enabled, ignoring 'address' setting (see Global Settings) Searching for PyCom boards on serial... Connecting to /dev/tty.usbmodemPy85be0a1... Wifi connection attempt: 2 WLAN connected? False (...list of wifi networks...) Connecting with <SSID> and <PASSWORD> WiFi connection established MQTT Protocol Packet sent. (Length: 114) This is PybytesProtocol.start_MQTT Packet sent. (Length: 44) Connected to MQTT mqtt.pybytes.pycom.io Pybytes connected successfully (using the built-in pybytes library) This is pack_info_message() __pack_message: b'010504a00584000000' MQTT Protocol Packet sent. (Length: 50) Pybytes configuration read from /flash/pybytes_config.json [INFO] Starting firmware [INFO] Network connection: WLAN [INFO] Waiting for time syncronization... [INFO] 2021-05-21T11:27:00Z Time synchronized Pycom MicroPython 1.20.2.r4 [v1.11-ffb0e1c] on 2021-01-12; GPy with ESP32 Pybytes Version: 1.6.1 Type "help()" for more information. >>> [INFO] 2021-05-21T11:27:00Z [INFO] 2021-05-21T11:28:00Z [INFO] 2021-05-21T11:29:00Z [INFO] 2021-05-21T11:30:00Z [INFO] 2021-05-21T11:31:00Z [INFO] 2021-05-21T11:32:00Z [INFO] 2021-05-21T11:33:00Z [INFO] 2021-05-21T11:34:00Z [INFO] 2021-05-21T11:35:00Z [INFO] 2021-05-21T11:36:00Z This is PybytesProtocol.__keep_connection(alarm=<Alarm>) MQTT Protocol Packet sent. (Length: 42) This is PybytesProtocol.__process_recv_message() This is PybytesLibrary.unpack_message(message=b'\x07') header: 7 body: b'' network_type: 0 message_type: 7 [INFO] 2021-05-21T11:37:00Z [INFO] 2021-05-21T11:38:00Z [INFO] 2021-05-21T11:39:00Z [INFO] 2021-05-21T11:40:00Z [INFO] 2021-05-21T11:41:00Z [INFO] 2021-05-21T11:42:00Z [INFO] 2021-05-21T11:43:00Z [INFO] 2021-05-21T11:44:00Z [INFO] 2021-05-21T11:45:00Z [INFO] 2021-05-21T11:46:00Z This is PybytesProtocol.__keep_connection(alarm=<Alarm>) MQTT Protocol Packet sent. (Length: 42) [INFO] 2021-05-21T11:47:00Z [INFO] 2021-05-21T11:48:00Z [INFO] 2021-05-21T11:49:00Z [INFO] 2021-05-21T11:50:00Z [INFO] 2021-05-21T11:51:00Z [INFO] 2021-05-21T11:52:00Z [INFO] 2021-05-21T11:53:00Z [INFO] 2021-05-21T11:54:00Z [INFO] 2021-05-21T11:55:00Z [INFO] 2021-05-21T11:56:00Z This is PybytesProtocol.__keep_connection(alarm=<Alarm>) MQTT Protocol Socket send error [Errno 104] ECONNRESET [INFO] 2021-05-21T11:57:00Z ets Jun 8 2016 00:22:57 rst:0x7 (TG0WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff8020,len:8 load:0x3fff8028,len:2128 load:0x4009fa00,len:19760 entry 0x400a05bc {'wifi': {'ssid': 'SSID', 'password': 'PASSWORD'}, 'cfg_msg': 'Pybytes configuration read from /flash/pybytes_config.json', 'dump_ca': False, 'username': 'mail@example.org', 'server': 'mqtt.pybytes.pycom.io', 'ssl': False, 'lte': {'apn': 'em', 'cid': 1, 'reset': False, 'carrier': 'standard', 'band': 20, 'type': 'IP'}, 'device_id': '00000000-0000-0000-0000-000000000000', 'network_preferences': ['wifi', 'lte'], 'ota_server': {'port': 443, 'domain': 'software.pycom.io'}, 'wlan_antenna': 0, 'pybytes_autostart': True} ssl_params={'ca_certs': '/flash/cert/pycom-ca.pem'} LIS2HH12 not imported ssl_params={'ca_certs': '/flash/cert/pycom-ca.pem'} WMAC: 840D8E122018 Firmware: 1.20.2.r4 Pybytes: 1.6.1 {'wifi': {'ssid': 'SSID', 'password': 'PASSWORD'}, 'cfg_msg': 'Pybytes configuration read from /flash/pybytes_config.json', 'dump_ca': False, 'username': 'mail@example.org', 'server': 'mqtt.pybytes.pycom.io', 'ssl': False, 'lte': {'apn': 'em', 'cid': 1, 'reset': False, 'carrier': 'standard', 'band': 20, 'type': 'IP'}, 'device_id': '00000000-0000-0000-0000-000000000000', 'network_preferences': ['wifi', 'lte'], 'ota_server': {'port': 443, 'domain': 'software.pycom.io'}, 'wlan_antenna': 0, 'pybytes_autostart': True} Attempting to connect with network wifi Initialized watchdog for WiFi and LTE connection with timeout 1260000 ms WLAN connected? False Wifi connection attempt: 1 WLAN connected? False (...list of wifi networks...) Connecting with <SSID> and <PASSWORD> Wifi connection attempt: 2 WLAN connected? False (...list of wifi networks...) Connecting with <SSID> and <PASSWORD> WiFi connection established MQTT Protocol Packet sent. (Length: 114) This is PybytesProtocol.start_MQTT Packet sent. (Length: 44) Connected to MQTT mqtt.pybytes.pycom.io Pybytes connected successfully (using the built-in pybytes library) This is pack_info_message() __pack_message: b'010504a00584000000' MQTT Protocol Packet sent. (Length: 50) Pybytes configuration read from /flash/pybytes_config.json [INFO] Starting firmware [INFO] Network connection: WLAN [INFO] Waiting for time syncronization... [INFO] 2021-05-21T11:58:07Z Time synchronized Pycom MicroPython 1.20.2.r4 [v1.11-ffb0e1c] on 2021-01-12; GPy with ESP32 Pybytes Version: 1.6.1 Type "help()" for more information. >>> [INFO] 2021-05-21T11:59:07Z [INFO] 2021-05-21T12:00:07Z [INFO] 2021-05-21T12:01:07Z >>> machine.reset_cause() 2 >>> [INFO] 2021-05-21T12:02:07Z [INFO] 2021-05-21T12:03:07Z [INFO] 2021-05-21T12:04:07Z [INFO] 2021-05-21T12:05:07Z [INFO] 2021-05-21T12:06:07Z [INFO] 2021-05-21T12:07:07Z [INFO] 2021-05-21T12:08:07Z This is PybytesProtocol.__keep_connection(alarm=<Alarm>) MQTT Protocol Packet sent. (Length: 42) [INFO] 2021-05-21T12:09:07Z [INFO] 2021-05-21T12:10:07Z [INFO] 2021-05-21T12:11:07Z [INFO] 2021-05-21T12:12:07Z [INFO] 2021-05-21T12:13:07Z [INFO] 2021-05-21T12:14:07Z [INFO] 2021-05-21T12:15:07Z [INFO] 2021-05-21T12:16:07Z [INFO] 2021-05-21T12:17:07Z [INFO] 2021-05-21T12:18:07Z This is PybytesProtocol.__keep_connection(alarm=<Alarm>) MQTT Protocol Socket send error [Errno 104] ECONNRESET ets Jun 8 2016 00:22:57 rst:0x7 (TG0WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:1 load:0x3fff8020,len:8 load:0x3fff8028,len:2128 load:0x4009fa00,len:19760 entry 0x400a05bc {'wifi': {'ssid': 'SSID', 'password': 'PASSWORD'}, 'cfg_msg': 'Pybytes configuration read from /flash/pybytes_config.json', 'dump_ca': False, 'username': 'mail@example.org', 'server': 'mqtt.pybytes.pycom.io', 'ssl': False, 'lte': {'apn': 'em', 'cid': 1, 'reset': False, 'carrier': 'standard', 'band': 20, 'type': 'IP'}, 'device_id': '00000000-0000-0000-0000-000000000000', 'network_preferences': ['wifi', 'lte'], 'ota_server': {'port': 443, 'domain': 'software.pycom.io'}, 'wlan_antenna': 0, 'pybytes_autostart': True} ssl_params={'ca_certs': '/flash/cert/pycom-ca.pem'} LIS2HH12 not imported ssl_params={'ca_certs': '/flash/cert/pycom-ca.pem'} WMAC: 840D8E122018 Firmware: 1.20.2.r4 Pybytes: 1.6.1 {'wifi': {'ssid': 'SSID', 'password': 'PASSWORD'}, 'cfg_msg': 'Pybytes configuration read from /flash/pybytes_config.json', 'dump_ca': False, 'username': 'mail@example.org', 'server': 'mqtt.pybytes.pycom.io', 'ssl': False, 'lte': {'apn': 'em', 'cid': 1, 'reset': False, 'carrier': 'standard', 'band': 20, 'type': 'IP'}, 'device_id': '00000000-0000-0000-0000-000000000000', 'network_preferences': ['wifi', 'lte'], 'ota_server': {'port': 443, 'domain': 'software.pycom.io'}, 'wlan_antenna': 0, 'pybytes_autostart': True} Attempting to connect with network wifi Initialized watchdog for WiFi and LTE connection with timeout 1260000 ms WLAN connected? False Wifi connection attempt: 1 WLAN connected? False (...list of wifi networks...) Connecting with <SSID> and <PASSWORD> Wifi connection attempt: 2 WLAN connected? False (...list of wifi networks...) Connecting with <SSID> and <PASSWORD> WiFi connection established MQTT Protocol Packet sent. (Length: 114) This is PybytesProtocol.start_MQTT Packet sent. (Length: 44) Connected to MQTT mqtt.pybytes.pycom.io Pybytes connected successfully (using the built-in pybytes library) This is pack_info_message() __pack_message: b'010504a00584000000' MQTT Protocol Packet sent. (Length: 50) Pybytes configuration read from /flash/pybytes_config.json [INFO] Starting firmware [INFO] Network connection: WLAN [INFO] Waiting for time syncronization... [INFO] 2021-05-21T12:19:34Z Time synchronized Pycom MicroPython 1.20.2.r4 [v1.11-ffb0e1c] on 2021-01-12; GPy with ESP32 Pybytes Version: 1.6.1 Type "help()" for more information. >>> [INFO] 2021-05-21T12:20:34Z [INFO] 2021-05-21T12:21:34Z [INFO] 2021-05-21T12:22:34Z [INFO] 2021-05-21T12:23:34Z [INFO] 2021-05-21T12:24:34Z
-
Ah sorry, my mistake! Good to hear it works :)
-
Thanks for the quick reply! This sounds like what I am looking for, but when I try to set the configuration key I get this error:
>>> pybytes.update_config("connection_watchdog", value=False) Error updating configuration! Traceback (most recent call last): File "_pybytes.py", line 339, in update_config KeyError: connection_watchdog
Edit: This seems to fail because the
pybytes_config.json
does not have this key present. It works when usingpybytes.set_config(...)
though :)
-
I've suspected this issue for my devices for the past few days as well (but they did not have the pybytes debugging turned on, so Im not 100% sure). We did have some issues with the Pybytes service, but all should be operational now.
Disabling the watchdog can be done in the pybytes_config.json file by adding the key 'connection_watchdog' with value 'False'. You can use
pybytes.update_config(...)
as described here: https://docs.pycom.io/pybytes/api/#pybytesupdate_configkey-valuenone-permanenttrue-silentfalse-reconnectfalse