LTE M1 cellular firmware updating to Gpy stucks, says, Waiting for updater to load...

[reidfo]

Here is the output I get when running the posted upgrade script that bypasses modem init.

Pycom MicroPython 1.20.0.rc8 [7b83c6d] on 2019-03-12; GPy with ESP32
Type "help()" for more information.
>>>
>>> run('/sd/CATM1-39529.dup', 921600)
<<< Welcome to the SQN3330 firmware updater >>>
Starting STP (DO NOT DISCONNECT POWER!!!)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 108, in run
OSError: Invalid answer 'b''' from the device
>>>

[fidaforever]

@reidfo
Quite hopeless!
Today I have used a Fipy (instead of Gpy) and Expansion Board 3.0 (instead of 3.1 that I have used previously) and ended with same result.
Experiencing exactly the same scenario with brand new Fipy module with Expansion board 3.
This time all I have noticed as:
for the very first attempt of Cellular Firmware up-gradation outcome,

>>> import sqnsupgrade
>>> sqnsupgrade.run('/sd/CATM1-39529.dup', '/sd/updater.elf')
<<< Welcome to the SQN3330 firmware updater [1.2.3] >>>
>>> FiPy with firmware version 1.18.2.r2
Attempting AT wakeup...
Session opened: version 1, max transfer 8192 bytes
Sending 55570 bytes: [########################################] 100%
Attempting AT auto-negotiation...
ERROR sending AT command... no response? b''
AT auto-negotiation failed! Exiting.
Unable to load updater from /sd/updater.elf

It dealt with, transfer 8192 bytes and sends 55570 bytes before crashing.
I can ensure that the updater.elf was in the SD card,

>>> from machine import SD
>>> sd = SD()
>>> os.mount(sd,'/sd')
>>> os.listdir('/sd')
['System Volume Information', 'CATM1-39529.dup', 'updater.elf', 'upgdiff_33080-to-39529.dup', 'upgdiff_38638-to-39529.dup', 'test.txt']

Following on, every other attempts gives the result as,

>>> import sqnsupgrade
>>> sqnsupgrade.run('/sd/CATM1-39529.dup', '/sd/updater.elf')
<<< Welcome to the SQN3330 firmware updater [1.2.3] >>>
>>> FiPy with firmware version 1.18.2.r2
Attempting AT auto-negotiation...
Session opened: version 1, max transfer 2048 bytes
Sending 429340 bytes: [########################################] 100%
Waiting for updater to load...

Waiting for updater to load... that I am dealing with!
Important here that, the transferring byte size is small, 2048 bytes always and sends 429340 bytes. with respect to the very first attempt of transfer 8192 bytes and sends 55570 bytes !

So, what's going on?
What I am missing? Anything, new with the updater.elf that I am downloading is creating such a problem?
How can I erase and clean the Firmware for redoing again?

[panda]

Exact same issues here as well. Just started to deploy a number of boards and so far we are 5 boards and 5 failures with the message "Waiting for updater to load...".

sqnsupgrade.run('/sd/deploy/lte_firmware/upgdiff_38638-to-39529.dup', debug=True)

How do we recover these modules now?!!! I have 75 more boards to do. What are we doing wrong????

[reidfo]

I was finally able to upgrade a couple of GPys based on a tip I read in another post. Downgrading the Pycom firmware to 1.18.1.r1 allowed the Sequans updater to successfully complete. I then flashed the GPy back to my current 1.20.0.rc8 firmware.

[panda]

Yea, I see that works as well. Thanks. Appears the latest 1.18.2 broke modem updates. Awesome...as if this deployment wasn't slow enough already :(

[panda]

Update:

While the update executes successfully with the older fw, the modems are actually not functioning. They hang periodically and refuse to attach.

I used the following according to the docs on a new device (never updated before):
sqnsupgrade.run('/sd/deploy/lte_firmware/upgdiff_38638-to-39529.dup', debug=True)

I just read here: https://github.com/pycom/pycom-libraries/tree/master/lib/sqnsupgrade

"When using differential upgrade packages (ex: upgdiff_XXXX-to-XXXX.dup) you CANNOT use updater.elf file."

So which is it? How to I fix this?

Here is the info() for the modem. I tried 2 others with the same result. They will not attach and they are hanging (blocking code) for seconds at a time.

Here is the info() after update:

<<< Welcome to the SQN3330 firmware updater [1.2.3] >>>
>>> GPy with firmware version 1.18.2.r3
The current delay is 1000
Response (+++ #1): None
Response (AT #1) None
Response (AT #3) None
Response (AT #4) b'\r\n+SYSSTART\r\n'
Response (AT #1 @ 115200) None
Response (AT #2 @ 115200) None
The current delay is 2000
Response (+++ #1): None
Response (AT #1) b'\r\nERROR\r\n'
Response (AT #2) b'\r\nOK\r\n'
Response (AT+SMOD?) b'\r\n2\r\nOK\r\n'
Converting response: 2 to int...
Modem state: 2
Your modem is in application mode. Here is the current version:
mirror? False  recover? False  resume? True  direct? True  atneg_only? False bootrom? False load_fff? False
SYSTEM VERSION
==============
  FIRMWARE VERSION
    Bootloader0  : 5.1.1.0 [41065]
    Bootloader1  : 5.1.1.0 [39529]
    Bootloader2* : 5.1.1.0 [39529]
    NV Info      : 1.1,0,0
    Software     : 5.1.1.0 [39529] by robot-soft at 2018-09-28 16:07:40
    UE           : 5.0.0.0d
  COMPONENTS
    ZSP0         : 1.0.99-13616
    ZSP1         : 1.0.99-12355

IMEI: 354347099454842
>>>

Here is the same info() for a working modem I have (updated a long time ago when 39529 came out. No idea what Pycom firmware I used)

<<< Welcome to the SQN3330 firmware updater [1.2.3] >>>
>>> GPy with firmware version 1.18.2.r3
The current delay is 1000
Response (+++ #1): None
Response (AT #1) b'\r\nERROR\r\n'
Response (AT #2) b'\r\nOK\r\n'
Response (AT+SMOD?) b'\r\n2\r\nOK\r\n'
Converting response: 2 to int...
Modem state: 2
Your modem is in application mode. Here is the current version:
mirror? False  recover? False  resume? True  direct? True  atneg_only? False bootrom? False load_fff? False
SYSTEM VERSION
==============
  FIRMWARE VERSION
    Bootloader0  : 5.1.1.0 [39529]
    Bootloader1* : 5.1.1.0 [39529]
    Bootloader2  : 5.1.1.0 [39529]
    NV Info      : 1.1,0,0
    Software     : 5.1.1.0 [39529] by robot-soft at 2018-09-28 16:07:40
    UE           : 5.0.0.0d
  COMPONENTS
    ZSP0         : 1.0.99-13616
    ZSP1         : 1.0.99-12355

IMEI: 354347092629911

[panda]

So it appears that after the upgrade these modems are constantly restarting. If I run the following:

import time
from network import LTE
lte = LTE()
time.sleep_ms(1000)
lte.attach(apn='hologram')

while lte.isattached() == False:
    r = lte.send_at_cmd("AT+CSQ").split('\r\n')
    print(r)
    time.sleep_ms(1000)

print("Attached!")

The output is as follows:

['', '+CSQ: 99,99', '', 'OK', '']
['']
['', '+SYSSTART', '']
['', '+CSQ: 99,99', '', 'OK', '']
['']
['', '+SYSSTART', '']
['', '+CSQ: 99,99', '', 'OK', '']
['']
['', '+SYSSTART', '']
['', '+CSQ: 99,99', '', 'OK', '']
['']
['', '+SYSSTART', '']

Note the SYSSTART every few seconds. Does this give anyone a clue as to what the issue is? I have tried downgrading the modem firmware. No luck. Same behavior. Then tried upgrading using the both the full .dup files and diff files. Nothing works.

[BorbonJuggler]

@reidfo Hello I am having the same problem with my boards, where did you find the FW ? Have you tested the modem capabilities ? I meant have you manage to connect to the network of your cellular carrier. It's quite frustrating all this process.

[BorbonJuggler]

@panda In the AT Commands Manual they said that is :

+SYSSTART URC indicates that the ME has started (or restarted after a
^RESET) and is ready to operate again.
+SHUTDOWN URC indicates that the ME has completed the reset procedure
and will restart.
The unsolicited command +SYSSTART is issued at each start or reboot. It
signals that the device is ready for operation.

[panda]

I just had it from before the latest was released. I have found some GPys work fine and others don't. Same sim, same firmware, same code. But different results. I have a ticket with Pycom but still no word on this. I have 75 units sitting here waiting to be updated.

[panda]

@borbonjuggler I saw that too but on modules that work I dont see this. Only on the modules that will not attach. They are also unable to see towers. I question if it is actually scanning.

[reidfo]

Most of my modules (GPy) will not attach. Some will only on AT&T, some only on Verizon, some not at all. I have tried multiple Cat-M carriers but have not had any reproducible success. Support ticket and questions here remain unanswered.

[fidaforever]

@reidfo You are right. I solved that issue a couple of days ago by downgrading the Pycom firmware to 1.18.2 (Fipy). Cause, I have had similar problem with Fipy as well. So, the latest firmware is broken!
Attention to @panda @BorbonJuggler

For Gpy: Firmware Gpy
For Fipy: Firmware Fipy

[panda]

I still believe my issues are related to the modem and not the Pycom firmware. I have already tried downgrading with no success. On the modules I have that work they connect reliably.

[panda]

Pycom was very helpful today and we determined that the modem is crashing when attaching. We determined this by looking at the debug output directly from the modem (soldering required).

At this point the only criteria to render a modem as non-functional is to perform a firmware update from the default 33080. Still not clear what is specific to this batch of GPys I have that this is an issue. Hardware related? Sequeans firmware issue? Still not clear. I have GPy modules purchased last year and they are fine.

I would be happy to use the default 33080 if I could, but I have no connection stability and excess module heat running the older firmware.

[reidfo]

@panda Have you had any further progress on this? I haven't seen anything from Pycom on this issue. Your post finally seems to confirm the behavior I am seeing. This and the enormous power consumption by GPy/Expansion board has me investigating modules from another manufacturer.

[panda]

@reidfo The issue is that an unsupported band is being used by the modem and causes it to crash and reboot. It's a bit more involved but that is the gist. I got a revised firmware that appears to have resolved it. I suppose I could PM the fw. Not sure if pycom would frown upon sharing. I would expect this is resolved in the next release after 1.18.2.r3.