FiPy LTE bricked by firmware update (Sequans Monarch SQN3330)



  • Hi Guys,

    Have I bricked the LTE on my FiPy? If not, what is the recovery process please?

    In its current state the following lines cause a hang. REPL in Atom is unresponsive.

    >>> from network import LTE
    >>> lte = LTE()
    

    I had been developing a process for firmware updates on the Sequans Monarch SQN3330 and had successfully updated a GPy about 10 times while developing documentation. On my first attempt at a FiPy the device didn't come back after Resetting. I didn't record the starting firmware version. The output from the upgrade attempt was:

    >>> import sqnsupgrade
    >>> sqnsupgrade.run('/sd/FIPY_GPY_CATM1_33988.dup', 921600)
    <<< Welcome to the SQN3330 firmware updater >>>
    Entering recovery mode
    Response: b'\r\n+CPSMS: 1,,,"10000110","00011110"\r\n\r\nOK\r\n\r\nOK\r\n\r\nERROR\r\n'
    Resetting.
    ..
    Starting STP (DO NOT DISCONNECT POWER!!!)
    STP started
    Session opened: version 1, max transfer 8192 bytes
    Sending 5665583 bytes: [########################################] 100%
    Code download done, returning to user mode
    Resetting (DO NOT DISCONNECT POWER!!!).
    .................................................................................................................................................................................................................................................................................................................................................................................................Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/flash/lib/sqnsupgrade.py", line 130, in run
      File "/flash/lib/sqnsupgrade.py", line 59, in wait_for_modem
    KeyboardInterrupt:
    >>>
    >>>
    

    I waited at the Resetting prompt for about 5 minutes before pressing Ctrl-C. Subsequent attempts ended with OSError: AT+SMSWBOOT=3,0 failed!:

    >>> import sqnsupgrade
    >>> sqnsupgrade.run('/sd/FIPY_GPY_CATM1_33988.dup', 921600)
    <<< Welcome to the SQN3330 firmware updater >>>
    Entering recovery mode
    Response: b''
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/flash/lib/sqnsupgrade.py", line 100, in run
    OSError: AT+SMSWBOOT=3,0 failed!
    >>>
    

    While back-checking I realised I wasn't on the latest FiPy firmware

    >>> import os
    >>> os.uname()
    (sysname='FiPy', nodename='FiPy', release='1.17.3.b1', version='v1.8.6-849-83e2f7f on 2018-03-19', machine='FiPy with ESP32', lorawan='1.0.2', sigfox='1.0.1')
    

    ... so updated:

    >>> import os
    >>> os.uname()
    (sysname='FiPy', nodename='FiPy', release='1.17.5.b6', version='v1.8.6-849-56d00234 on 2018-05-18', machine='FiPy with ESP32', lorawan='1.0.2', sigfox='1.0.1')
    

    Attempts to upgrade LTE firmware failed. From sqnsupgrade.py I created the following script to use serial to send a reset to the LTE modem. wait_for_modem() never returns.

    # https://github.com/pycom/pycom-libraries/blob/master/lib/sqnsupgrade/sqnsupgrade.py
    # from run()
    
    # import struct
    # import crc
    # import stp
    import time
    import os
    from machine import UART
    from machine import SD
    
    def read_rsp(s, size=None, timeout=-1):
        if timeout < 0:
            timeout = 20000
        elif timeout is None:
            timeout = 0
        if 'FiPy' in sysname or 'GPy' in sysname:
            while not s.any() and timeout > 0:
                time.sleep_ms(1)
                timeout -= 1
        else:
            while s.in_waiting <= 0 and timeout > 0:
                time.sleep(0.001)
                timeout -= 1
    
        if size is not None:
            rsp = s.read(size)
        else:
            rsp = s.read()
        if rsp is not None:
            return rsp
        else:
            return b''
            
    def wait_for_modem(s, send=True, expected=b'OK'):
        rsp = b''
        while True:
            if send:
                s.write(b"AT\r\n")
            r = read_rsp(s, size=(len(expected) + 4), timeout=50)
            if r:
                rsp += r
            if expected in rsp:
                print()
                break
            else:
                print('.', end='', flush=True)
                time.sleep(0.5)
    
    baud = 921600
    
    try:
        sysname = os.uname().sysname
    except:
        sysname = 'Windows'
    
    if 'GPy' in sysname:
        pins = ('P5', 'P98', 'P7', 'P99')
    else:
        pins = ('P20', 'P18', 'P19', 'P17')
    
    s = UART(1, baudrate=baud, pins=pins, timeout_chars=100)
    
    print("Read: {}".format(s.read()))
    
    time.sleep(1)
    print("Send: AT^RESET")
    s.write(b'AT^RESET\r\n')
    wait_for_modem(s, send=False, expected=b'+SHUTDOWN')
    time.sleep(1)
    print("Read: {}".format(s.read()))
    

    Output gives:

    >>>
    >>>
    Read: None
    Send: AT^RESET
    .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Traceback (most recent call last):
      File "<stdin>", line 70, in <module>
      File "<stdin>", line 49, in wait_for_modem
    KeyboardInterrupt:
    >
    

    Thanks for any help. Chris.



  • Just closing the loop. I haven't found a way to recover the FiPy I had with a failed firmware update. However, on a new FiPy the firmware update worked perfectly. As per documentation, I did, in order:

    • Firmware on expansion board
    • Firmware update on FiPy's ESP32 (1.18.0)
    • Firmware update to LTE device (Sequans Monarch SQN3330)

    LTE device reported:

    Pycom MicroPython 1.18.0 [v1.8.6-849-046b350] on 2018-06-01; FiPy with ESP32
    Type "help()" for more information.
    >>> import sqnsupgrade
    >>> sqnsupgrade.run('/sd/FIPY_GPY_CATM1_33988.dup', 921600)
    <<< Welcome to the SQN3330 firmware updater >>>
    Entering recovery mode
    Resetting..
    
    Starting STP (DO NOT DISCONNECT POWER!!!)
    STP started
    Session opened: version 1, max transfer 8192 bytes
    Sending 5665583 bytes: [########################################] 100%
    Code download done, returning to user mode
    Resetting (DO NOT DISCONNECT POWER!!!)..
    ........
    Deploying the upgrade (DO NOT DISCONNECT POWER!!!)...
    Resetting (DO NOT DISCONNECT POWER!!!)..
    ...
    Upgrade completed!
    Here's the current firmware version:
    UE5.0.0.0c
    LR5.1.1.0-33988
    OK
    >>>
    


  • @flo242 Thanks for your input. I might try another FiPy.



  • @jmarcelino

    Hi,

    I tried to flash the NB-Iot Firmware to the SQN 3330 by the manual provided in your docs on a FiPy running the stable release 1.18.0.
    It worked perfectly fine for the CAT-M1 Firmware 33988 (that is supposed to be flashed before the NB-IoT Firmware according to your docs), but afterwards it got stuck for the NB-IoT Firmware 35351 at the very same point that @Prawnhead described (although I waited much longer after the "Resetting ......." part.

    Instantiating the LTE object results in a freeze and direct communication with UART shows no response whatseover.
    So obviously the modem is dead now.

    There is this little hint in your docs

    • If your module is running the factory LTE chip firmware, you MUST first perform an update to the latest CAT-M1 firmware before trying to upgrade to the NB-IoT firmware. Skipping this step will cause your radio to become unresponsive and it will require access to the test points in order to re-flash the firmware.*

    Is there any documentation how the modem can be re-flashed manually by using certain test points?
    I'm reluctant to brick another device with this rather unstable process and if I had a way to re-flash the modem by brute force I would be more inclined to give it another try.

    Thanks for any help in advance.



  • Hello Prawnhead,

    I ran into the exact same problem and noone could help - see https://forum.pycom.io/post/18583. The modem upgrade seems to be far from working reliable. Writing an eMail to the sales support two months ago with very specific questions didn't get any answer at all, not even an automated reply.
    It just so happens that I tried to upgrade the modem of another FiPy today, this time using the newer "streaming" method (instead of SD-card). After a few tries I gave up, it always stopped at uploading (at different percentages) and sat there forever. Luckily it didn't brick the modem so I had unlimited tries. I finally used the SD-card method and this time it worked.



  • Hey J Marcelino, thanks for the help. Unfortunately, I get the same result every time. It seems there's no response from the modem on the serial port. Also tried:

    • reformatting SD card
    • downloading new copies of the firmware files
    • running with NB1 firmware instead
    • a second SD card
    • a different expansion board
      I think the poor little fella is roasted. :-)
    >>>
    Uploading project (main folder)...
    Not safe booting, disabled in settings
    
    Reading file status
    Failed to read project status, uploading all files
    Creating dir lib
    [1/9] Writing file boot.py
    [2/9] Writing file lib/codec.py
    [3/9] Writing file lib/crc.py
    [4/9] Writing file lib/sqnsrecovery.py
    [5/9] Writing file lib/sqnsupgrade.py
    [6/9] Writing file lib/stp.py
    [7/9] Writing file list-sd.py
    [8/9] Writing file main.py
    [9/9] Writing file recovery.py
    Upload done, resetting board...
    OKets Jun  8 2016 00:22:57
    
    rst:0x7 (TG0WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
    configsip: 0, SPIWP:0xee
    clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
    mode:DIO, clock div:1
    load:0x3fff8028,len:8
    load:0x3fff8030,len:1728
    load:0x4009fa00,len:0
    load:0x4009fa00,len:14584
    entry 0x400a059c
    MicroPython v1.8.6-849-56d00234 on 2018-05-18; FiPy with ESP32
    Type "help()" for more information.
    >>> Disconnected. Click here to reconnect.
    Connecting on COM4...
    MicroPython v1.8.6-849-56d00234 on 2018-05-18; FiPy with ESP32
    Type "help()" for more information.
    >>> import sqnsrecovery
    >>> sqnsrecovery.run('/sd/FIPY_GPY_CATM1_33988.dup', 921600)
    <<< Welcome to the SQN3330 firmware updater >>>
    Starting STP (DO NOT DISCONNECT POWER!!!)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/flash/lib/sqnsrecovery.py", line 110, in run
    OSError: Invalid answer 'b''' from the device
    >>>
    


  • This post is deleted!


  • @prawnhead
    If the modem is stuck in recovery mode you can try this version of sqnupgrade script:

    #!/usr/bin/env python
    import struct
    import crc
    import stp
    import time
    import os
    from machine import UART
    from machine import SD
    
    
    FFF_FMT = "<4sIIIIIIIHHIHHIHHHH"
    FFF_SLIM_FMT = "<4sIIIIIIIHHHH"
    FFF_FEATURES_SLIM = 1 << 0
    FFF_MAGIC = "FFF!"
    
    SFFF_MAGIC = "SFFF"             # Firmware
    SUFF_MAGIC = "SUFF"             # Updater
    TEST_MAGIC = "TEST"             # Test
    DIFF_MAGIC = "DIFF"             # Diff Upgrade
    UPGR_MAGIC = "UPGR"             # Generic raw upgrade
    RASR_MAGIC = "RASR"             # Generic raw rasterize
    
    
    def read_rsp(s, size=None, timeout=-1):
        if timeout < 0:
            timeout = 20000
        elif timeout is None:
            timeout = 0
        while not s.any() and timeout > 0:
            time.sleep_ms(1)
            timeout -= 1
    
        if size is not None:
            rsp = s.read(size)
        else:
            rsp = s.read()
        if rsp is not None:
            return rsp
        else:
            return b''
    
    def print_pretty_response(rsp):
        lines = rsp.decode('ascii').split('\r\n')
        for line in lines:
            if line:
                print(line)
    
    
    def wait_for_modem(s, send=True, expected=b'OK'):
        while True:
            if send:
                s.write("AT\r\n")
            rsp = read_rsp(s, timeout=50)
            if expected in rsp:
                print()
                break
            else:
                print('.', end='', flush=True)
                time.sleep(0.5)
    
    def run(file_path, baudrate):
        abort = False
    
        print('<<< Welcome to the SQN3330 firmware updater >>>')
    
        if '/sd' in file_path:
            sd = SD()
            time.sleep(0.5)
            os.mount(sd, '/sd')
            time.sleep(0.5)
    
        blobsize = os.stat(file_path)[6]
        blob = open(file_path, "rb")
    
        if 'FiPy' in os.uname().sysname:
            pins = ('P20', 'P18', 'P19', 'P17')
        else:
            pins = ('P5', 'P98', 'P7', 'P99')
    
        s = UART(1, baudrate=baudrate, pins=pins, timeout_chars=100)
        s.read()
    
        # disable echo
        s.write("ATE0\r\n")
        response = read_rsp(s)
    
    #     print('Entering recovery mode')
    #     s.write("AT+SMSWBOOT=3,0\r\n")
    #     response = read_rsp(s)
    #     if b'OK' in response:
    #         print('Resetting.', end='', flush=True)
    #         s.write('AT^RESET\r\n')
    #         wait_for_modem(s, send=False, expected=b'+SHUTDOWN')
    #         time.sleep(2)
    #         wait_for_modem(s)
    #         s.write("AT\r\n")
    #         s.write("AT\r\n")
    #     else:
    #         raise OSError('AT+SMSWBOOT=3,0 failed!')
    # 
    #     time.sleep(1)
    #     s.read()
    
        print('Starting STP (DO NOT DISCONNECT POWER!!!)')
        s.write('AT+SMSTPU=\"ON_THE_FLY\"\r\n')
        response = read_rsp(s, size=4)
        if response != b'OK\r\n' and response != b'\r\nOK' and response != b'\nOK':
            raise OSError("Invalid answer '%s' from the device" % response)
            blob.close()
        s.read()
        try:
            stp.start(blob, blobsize, s, baudrate, AT=False)
            print('Code download done, returning to user mode')
        except:
            blob.close()
            print('Code download failed, aborting!')
            abort = True
    
        time.sleep(1.5)
        s.read()
        s.write("AT+SMSWBOOT=1,0\r\n")
        response = read_rsp(s)
    
        print('Resetting (DO NOT DISCONNECT POWER!!!).', end='', flush=True)
        time.sleep(1.5)
        s.write("AT^RESET\r\n")
        wait_for_modem(s, send=False, expected=b'+SHUTDOWN')
        time.sleep(2)
        wait_for_modem(s, send=False, expected=b'+SYSSTART')
    
        if not abort:
            time.sleep(0.5)
            print('Deploying the upgrade (DO NOT DISCONNECT POWER!!!)...')
            s.write("AT+SMUPGRADE\r\n")
            response = read_rsp(s, timeout=120000)
    
            print('Resetting (DO NOT DISCONNECT POWER!!!).', end='', flush=True)
            time.sleep(1.5)
            s.write("AT^RESET\r\n")
            wait_for_modem(s, send=False, expected=b'+SHUTDOWN')
            time.sleep(2)
            wait_for_modem(s, send=False, expected=b'+SYSSTART')
            s.write("AT\r\n")
            s.write("AT\r\n")
            time.sleep(0.5)
            s.read()
            print('Upgrade completed!')
            print("Here's the current firmware version:")
            time.sleep(0.5)
            s.read()
            s.write("ATI1\r\n")
            response = read_rsp(s)
            print_pretty_response(response)
    

    The difference from the existing script is this removes the section where the code tries to initialise (and fail) the modem since when it's in recovery mode it doesn't answer back correctly.

    Also make sure you have a SIM card inserted, that helps.


 

Pycom on Twitter