New LTE Firmware releases v1.18.1.r4 (stable) & 1.19.0.b5 (development) & NB-IoT firmware 40343

[danielm]

@dan
Is there any time estimation for fix of socket issue?

[tlanier]

@dan
If it helps, note that the socket error occurs even if I'm using WIFI and connecting to a static IP. I don't think the problem has anything to do with the LTE module.

[danielm]

@tlanier
Is the behaviour identical on 1.19.0.b5 and 1.18.1.r4?

[serafimsaudade]

@tlanier
I'm using mqtt that use sockets over wifi connection with no problem.

[tlanier]

@tlanier

The problem with the previous code was my own fault. The "ssl.wrap_socket(s)" line should not be used when connecting to a non-secure site.

I now believe that the problem only occurs when connecting to a secure site. I can successfully connect using WIFI to a non-secure site. If I try to connect to a secure site (www.google.com), I get a stack overflow error.

[tlanier]

@tlanier
I'm now successfully connecting and downloading from a non-secure site using either LTE or WIFI. The errors occur if I try to connect to a secure site (www.google.com) using either LTE or WIFI.

# TEST - uses cell modem
def get1():
    try:
        log('creating LTE object - carrier verizon')
        # lte = LTE()
        lte = LTE(carrier="verizon")
        # lte = LTE(carrier="at&t")         # instantiate the LTE object
        log('resetting modem, may take up to 5 secs...')
        lte.reset()
        log('attaching...')
        lte.attach(apn='apn.hologram.io')        # attach the cellular modem to a base station
        # lte.attach(band=13)        # attach the cellular modem to a base station
        
        i = 0
        while not lte.isattached():
            log('waiting for isattached...')
            utime.sleep_ms(1000)

        log('connecting...')
        # lte.connect()       # start a data session and obtain an IP address
        lte.connect(cid=3)    # cid=3 for verizon - start a data session and obtain an IP address
        while not lte.isconnected():
            log('waiting for isconnected...')
            utime.sleep_ms(500)

        log('create socket')
        s = socket.socket()

        log("Connect to: www.ces-web.com")
        sockaddr = socket.getaddrinfo('www.ces-web.com', 80)[0][-1]

        #log("Connect to: www.google.com")
        #s = ssl.wrap_socket(s)
        #sockaddr = socket.getaddrinfo('www.google.com', 443)[0][-1]

        log("socket address: {}".format(sockaddr))

        s.connect(sockaddr)
        log("send GET request")
        s.send(b"GET / HTTP/1.0\r\n\r\n")
        log("receive response")
        log(s.recv(16))
        s.close()
        log("socket closed")
        lte.disconnect()
        log("disconnected")
        lte.dettach()
        log("dettached")
        lte.deinit()
        log("deinit")
    except Exception as ex:
        sys.print_exception(ex)
        sys.exit()

[panda]

With this latest version on a GPy and latest CAT M1 modem firmware I can no longer connect with hologram SIM.

lte.connect() will dettach the modem and the REPL input hangs.

Previously I could successfully do:

lte = LTE('verizon')
lte.attach(apn='apn.hologram.io')
...
lte.connect(cid=3)

Going back to 1.18.1.r1 and 38638 modem fw works fine. Why would this change? How do I connect to Hologram with the latest updates?

The creg info is:

['+CEREG: 2,5,"3D03","00F0F703",7', 'OK']

[Xykon]

Thanks everyone for the reports.

@iwahdan is looking into the SSL issue as this doesn't seem to be directly related to the LTE changes I made.

@panda Unfortunately we don't have hologram.io in The Netherlands so all I can do is emulate Verizon on our Amarisoft eNB.

Could you please try the following 3 commands between attach and connect?

print(lte.send_at_cmd('AT+CGATT?'))
print(lte.send_at_cmd('AT+CFUN?'))
print(lte.send_at_cmd('AT!="showphy"'))

You mention that the REPL input hangs after connect()... if you do a hard reset does it keep hanging?

If you are able to reset the module and enter code, can you please run the 3 commands again without calling attach first?

[panda]

@xykon said in New LTE Firmware releases v1.18.1.r4 (stable) & 1.19.0.b5 (development) & NB-IoT firmware 40343:

Could you please try the following 3 commands between attach and connect?

text
print(lte.send_at_cmd('AT+CGATT?'))
print(lte.send_at_cmd('AT+CFUN?'))
print(lte.send_at_cmd('AT!="showphy"'))

['+CGATT: 1', 'OK']

['+CFUN: 1', 'OK']

['DL SYNCHRO STATISTICS', '=====================', ' Synchro state : DRX_SLEEP', ' PPU SIB1 ACQ watchdog : 0', ' Frequency Hypothesis RF (Hz) : 0', ' RSRP (dBm) : -94.06', ' RSRQ (dB) : -10.79', ' Channel estimation state (Cell-spec.) : HIGH CINR', ' Channel estimation state (UE-spec.) : LOW CINR', ' Channel estimation state (MBSFN) : LOW CINR', ' Channel estimation CINR : 10.10', ' Channel length : SHORT', ' AGC', ' AGC RX gain (dB) : 53.71', ' RX PSD BO (dBFs) : -20.63', ' RX PSD (dBm) : -99.33', ' Noise level RS (dBm) : -104.16', ' Digital gain (dB) : 9.03', ' CINR RS (dB) : 10.10', ' NARROWBANDS', ' Last DL NB : Central', ' Last UL NB : N/A', ' AFC', ' Frequency offset RF (Hz) : -316', ' Frequency offset BB (Hz) : 0', ' PBCH', ' MIB received quantity : 1239', ' MIB timeout quantity : 0', 'OK']

You mention that the REPL input hangs after connect()... if you do a hard reset does it keep hanging?

No, reset solves it.

If you are able to reset the module and enter code, can you please run the 3 commands again without calling attach first?

['+CGATT: 0', 'OK']

['+CFUN: 4', 'OK']

['DL SYNCHRO STATISTICS', '=====================', ' Synchro state : OFF', ' PPU SIB1 ACQwatchdog : 0', ' Frequency Hypothesis RF (Hz) : 0', ' RSRP (dBm) : 0.00', ' RSRQ (dB) : 0.00', ' Channel estimation state (Cell-spec.) : LOW CINR',' Channel estimation state (UE-spec.) : LOW CINR', ' Channel estimation state (MBSFN) : LOW CINR', ' Channel estimation CINR : 0.00', ' Channel length : SHORT', ' AGC', ' AGC RX gain (dB) : 0.00', ' RX PSD BO (dBFs) : 0.00', ' RX PSD (dBm) : 0.00', ' Noise level RS (dBm) : 0.00', ' Digital gain (dB) : 0.00', ' CINR RS (dB) : 0.00', ' NARROWBANDS', ' Last DL NB :0', ' Last UL NB : 0', ' AFC', ' Frequency offset RF (Hz) : 0', ' Frequency offset BB (Hz) : 0', ' PBCH', ' MIB received quantity : 0', ' MIB timeout quantity : 0', 'OK']

[iwahdan]

Hi @tlanier ,I am not able to reproduce the stack overflow issue you got yet, however I've made some improvements to LTE and sockets in a pre-release v1.18.1.r5 :
https://github.com/pycom/pycom-micropython-sigfox/releases/tag/v1.18.1.r5
maybe you can see if that solves the problem.

[danielm]

@iwahdan
I am testing 1.18.1.r5 on FiPy with NB1-40343 running following code in main.py:

import network
import time
import urequests

class httpPostTest():
	def __init__(self):	
		self.apn = None
		self.band = None
		self.LTEisAttached = False
		self.LTEisAttachedLast = False
		self.LTEisConnected = False
		self.LTEisConnectedLast = False

		self.start = True

	def initLTE(self, band, apn):
		self.apn = apn
		self.band = band
		print("INFO", "Starting initialization of LTE")
		
		try:
			self.lte = network.LTE()
			print("INFO", "LTE initialized")
		except:
			print("WARNING", "LTE init command failed")	

		print("LTE is attached after init:", self.lte.isattached())
		print("LTE is connected after init:", self.lte.isconnected())

	def run(self, url):	
		n = 0
		while True:

			self.LTEisAttached = self.lte.isattached()

			if not self.LTEisAttached and (self.LTEisAttachedLast or self.start):
				if not self.start:
					print("WARNING", "Dettached from LTE network")
				try:
					self.lte.attach(band = self.band, apn = self.apn)
					print("INFO", "LTE attach command sent")
					self.start = False
				except:
					print("WARNING", "LTE attach command failed")

			if self.LTEisAttached and not self.LTEisAttachedLast:
				print("INFO", "Attached to LTE network")
				try:
					self.lte.connect()
					print("INFO", "LTE connect command sent")
				except:
					print("WARNING", "LTE connect command failed")

			self.LTEisAttachedLast = self.LTEisAttached

			self.LTEisConnected = self.lte.isconnected()

			if self.LTEisConnected and not self.LTEisConnectedLast:
				print("INFO", "Connected to LTE network")

			elif not self.LTEisConnected and self.LTEisConnectedLast:
				print("WARNING", "Disconnected from LTE network")

			self.LTEisConnectedLast = self.LTEisConnected

			if self.LTEisConnected:
				n+=1
				try:
					response = urequests.post(url, headers={"Content-Type": "application/json", "Accept": "application/json"}, data='{"time": "%s"}' % (time.time()))
					print("INFO", "%s. HTTP POST successful: %s" % (n, response.text))
				except Exception as e:
					print("WARNING", "%s. HTTP POST failed: %s" % (n, e))

			time.sleep(0.25)

test = httpPostTest()
test.initLTE(20, "nbiot.telekom.sk")
test.run(url = "https://54.167.116.227/post")

It is also available here for download including slightly modified urequests library:
https://drive.google.com/open?id=19hA2KGq4iwkkv36d2cuW1M4j0Tab9_YP

Issues:

1. The code often hangs during LTE initialization (self.lte = network.LTE())
2. If it makes it through init only first request is sent successfully and then the modem seems to dettach from the network:

INFO Starting initialization of LTE
INFO LTE initialized
LTE is attached after init: False
LTE is connected after init: False
INFO LTE attached command sent
INFO Attached to LTE network
INFO LTE connect command sent
INFO Connected to LTE network
INFO 1. HTTP POST successful: {"args":{},"data":{"time":"13"},"files":{},"form":{},"headers":{"x-forwarded-proto":"https","host":"54.167.116.227","content-length":"14","accept":"application/json","content-type":"application/json","x-forwarded-port":"443"},"json":{"time":"13"},"url":"https://54.167.116.227/post"}
WARNING 2. HTTP POST failed: Avialable Interfaces are down
WARNING Dettached from LTE network

Application code written for our product behaves in a similar way since 1.18.1.r3.

[danielm]

In the meantime I left the test code running on 1.18.1.r2 with NB1-40343 modem firmware.
It was necessary to replace line

self.lte.attach(band = self.band, apn = self.apn)

with:

self.lte.send_at_cmd('AT+CGDCONT=1,"IP","%s"' % (self.apn))
self.lte.attach(band = self.band)

because lte.attach() does not support "apn" parameter in 1.18.1.r2.

After almost 5000 requests the test failed with error codes as follows:

INFO 4866. HTTP POST successful: {"args":{},"data":{"time":"41384"},"files":{},"form":{},"headers":{"x-forwarded-proto":"https","host":"54.167.116.227","content-length":"17","accept":"application/json","content-type":"application/json","x-forwarded-port":"443"},"json":{"time":"41384"},"url":"https://54.167.116.227/post"}
INFO 4867. HTTP POST successful: {"args":{},"data":{"time":"41392"},"files":{},"form":{},"headers":{"x-forwarded-proto":"https","host":"54.167.116.227","content-length":"17","accept":"application/json","content-type":"application/json","x-forwarded-port":"443"},"json":{"time":"41392"},"url":"https://54.167.116.227/post"}
INFO 4868. HTTP POST successful: {"args":{},"data":{"time":"41399"},"files":{},"form":{},"headers":{"x-forwarded-proto":"https","host":"54.167.116.227","content-length":"17","accept":"application/json","content-type":"application/json","x-forwarded-port":"443"},"json":{"time":"41399"},"url":"https://54.167.116.227/post"}
WARNING 4869. HTTP POST failed: -76
WARNING 4870. HTTP POST failed: -1
WARNING 4871. HTTP POST failed: -1
WARNING 4872. HTTP POST failed: -1
WARNING 4873. HTTP POST failed: -1
WARNING 4874. HTTP POST failed: -1
WARNING 4875. HTTP POST failed: -1
WARNING 4876. HTTP POST failed: -1
WARNING 4877. HTTP POST failed: -1
WARNING 4878. HTTP POST failed: -1
WARNING 4879. HTTP POST failed: 23
WARNING 4880. HTTP POST failed: 23
WARNING 4881. HTTP POST failed: 23
WARNING 4882. HTTP POST failed: 23
WARNING 4883. HTTP POST failed: 23
WARNING 4884. HTTP POST failed: 23

WARNING 10843. HTTP POST failed: 23
WARNING 10844. HTTP POST failed: 23
WARNING 10845. HTTP POST failed: 23
WARNING 10846. HTTP POST failed: [Errno 105] ENOBUFS
WARNING 10847. HTTP POST failed: [Errno 105] ENOBUFS
WARNING 10848. HTTP POST failed: [Errno 105] ENOBUFS

Methods lte.isattached() and lte.isconnected() returned True during entire test period.

After Ctrl-F reset (without power supply disconnection) it was not possible to init LTE (from REPL). I upgraded Pycom FW to 1.18.1.r5 via FTP OTA an LTE init failed as well.

[danielm]

@Xykon @iwahdan
I would appreciate feedback on my previous two posts.
LTE implementation in 1.18.1.r3 to 1.18.1.r5 works worse than in 1.18.1.r2. Testing code is provided, you may check yourself.

[Paul Thornton]

Hi All,

Due to moving office's and limited internet access the development team are unable to investigate the LTE issues at the moment. Thanks for being patient whilst we work on this.

@danielm Ill make sure they see your response and take a look just as soon as they can.

Paul T,

[robert-hh]

@paul-thornton Hello Paul. Welcome to the community.

[Paul Thornton]

@robert-hh

Thanks :D Ill make a proper introduction post soon. I just snuck on to post this and get you all updated. Look forward to getting stuck in the forums some cool projects being worked on in here. :)

[danielm]

@paul-thornton
Hi Paul, thanks for the information regarding moving of the office.

[combaindeft]

@Xykon ... the new firmware doesn't work with Telia NB-IOT in sweden :-( ... cannot attach anymore ...

I tried to just use the "update" dup ... but I'm trying now to use the total reset method and see if that helps ... elsewise I'll be downgrading back to the previous version that worked ...

[wghirakawa]

I was having major problems, with trying to do the update using v1.18.1.r4 and using the FW package 39529. After following multiple recommendations from other FiPy Users, as well as trying to do the method as described in the Docs, it was to no avail. Then I viewed @rskoniec post at https://forum.pycom.io/topic/3995/fipy-firmware-upgrade-failed-cat-m1/26 showed me a potential way of getting my FW at least to 38638. But after attempting downgrading my FiPy Firmware to v1.18.1.r1 and trying to upgrade to 38638 it just was not happening. It kept getting stuck on Attempting AT Auto-Negotiation, then it would eventually fail.

ERROR sending AT command... no response? b''
AT auto-negotiation failed! Exiting.
Unable to load updater from /sd/updater.elf

@rskoniec was kind enough to remotely connect to my computer and we spent roughly 3 hours trying to do the firmware upgrade to the SQNS Modem. @rskoniec knows exactly what he did, and will be replying to this post with how he was able to get my firmware on my modem upgraded. As it stands I have a FiPY + Pytrack and my sqnsupgrade.version() is as bellow.

SYSTEM VERSION
==============
  FIRMWARE VERSION
    Bootloader0  : 5.1.1.0 [33080]
    Bootloader1  : 5.1.1.0 [38638]
    Bootloader2* : 5.1.1.0 [38638]
    NV Info      : 1.1,0,0
    Software     : 5.1.1.0 [38638] by robot-soft at 2018-08-20 09:51:46
    UE           : 5.0.0.0d
  COMPONENTS
    ZSP0         : 1.0.99-13604
    ZSP1         : 1.0.99-12341

I will be attempting to upgrade to the newest SQNS Firmware after updating the FiPy to r4. I will attempt upgrading to 39529 using the method as described in the forum, and will post my results.

I cannot thank @rskoniec enough for the time he dedicated to help another developer! I hope that my post will save others valuable time, especially during this holiday season.

[iwahdan]

@danielm said in New LTE Firmware releases v1.18.1.r4 (stable) & 1.19.0.b5 (development) & NB-IoT firmware 40343:

https://54.167.116.227/post

Hello @danielm , I could reproduce the issue you had with getting disconnected after the first HTTP post request I submitted a fix now for it as pre-release v1.18.1.r6.

Thanks for your feedback