New firmware release 1.7.6.b1

[daniel]

Hello,

A new firmware release is out. The version is 1.7.6.b1. Here's the change log:

• esp32: Store the LoRaWAN state in NVS, to avoid having to re-join after waking up from deep sleep (or a power cycle). Calling lora.has_joined() after deep sleep returns True if the network was previously joined.

• esp32: Update to the latest IDF to fix issues with RTC clock accuracy during deep sleep.

• esp32: Fix bug related to setting the UART parity in the constructor.

• esp32: Fix bug related to incorrect I2C HW objects initialisation.

In order to get the new firmware it, please user the updater tool that can be downloaded from here: https://www.pycom.io/downloads/

Cheers,
Daniel

[soren]

@jmarcelino yes, of course. It's the part where I'm using the socket that is commented out right now:

import machine
from machine import UART
from network import WLAN
#from network import LoRa
#lora = LoRa(mode=LoRa.LORAWAN)
#from machine import SD
import socket
#import ssl
#import time
import os
import pycom
print('starting up..')
#sd = SD()
#os.mount(sd, '/sd')
#time.sleep(5)
#os.listdir('/sd')
#time.sleep(5)

uart = UART(0, 115200)
os.dupterm(uart)
pycom.heartbeat(False)
pycom.rgbled(0x7f0000) # red

wlan = WLAN()
#print('SD mounted!')
if machine.reset_cause() != machine.SOFT_RESET:
    print('setting wlan config...')
    wlan.init(mode=WLAN.STA)
    wlan.ifconfig(config=('10.0.0.88', '255.255.255.0', '10.0.0.1', '8.8.8.8'))

if not wlan.isconnected():
    print('looking for network...')
    wlan.connect('NETGEAR64', auth=(WLAN.WPA2, 'oddchair195'), timeout=5000)
    while not wlan.isconnected():
        machine.idle()
print('connected to wifi!')

#s = socket.socket()
#ss = ssl.wrap_socket(s)
#ss.connect(socket.getaddrinfo('www.google.com', 443)[0][-1])
#print('connected to internet!')


space = os.getfree('/flash')
print('free flash mem: ', space)
pycom.rgbled(0x007f00) # green

[jmarcelino]

@soren said in New firmware release 1.7.6.b1:

network card not available

Can you post the code you’re trying to use, or at least the part where you setup the network / socket?

[soren]

@jcaron hi, has this been fixed i wonder? i get "network card not available" right now, and i'm not using lora. don't need to.

[jcaron]

@jellium @daniel OK, so I dug a bit in the code and added a few traces, and the conclusion is that:

• there are actually two instances of the counters being reset: right from the init of the board (in TASK_LoRa, which calls LoRaMacStoreUpAndDownLinkCountersInNvs), and a second one in the LoRa constructor, which calls LoRaMacInitialization which in turn calls ResetMacParameters. Both will save the counters with their default values before they have been read from NVS.

• the TASK_LoRa is actually an infinite loop running all the time, whether LoRa is actually running or configured or not!

I'll try to find how this could be resolved, but I won't mind if someone who is more familiar with the code does it first or has any pointers!

BTW, the code on GitHub has the wrong version number, is there anything else that would have missed the commit?

[jellium]

@jcaron Hello, I cannot provide an answer to the problem you are reporting but in case you haven't read it, I draw your attention on the similar issue that I am reporting in this present thread on this and that posts.
Edit: my bad, I see that you have read the post indeed :)

[jcaron]

The LoRaWAN frame counters storage in NVS doesn't seem to work as intended when combined with deep sleep (tried with both 1.7.6.b1 and the new 1.7.7.b1): the uplink frame counter restarts at 1 after each sleep... Not sure if that's because the counter is not properly stored, not properly read, or reset in the process?

The only thing I do after wake up is:

lora = LoRa(mode=LoRa.LORAWAN)
s = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
s.setsockopt(socket.SOL_LORA, socket.SO_DR, 0)
s.send(bytes([1]));

Even though lora isn't used, if I don't include the Lora() initialiser, socket throws an error (OSError: Network card not available), not sure if that call resets the counter (I would expect only the join to do so)?

I even added a 2-second sleep after sending just to make sure the packet is actually sent, just in case the NVS storage is done only once the packet has finished sending, no change.

Is there a way to check what is stored in NVRAM, using pycom.nvs_get() for instance?

Thanks,

Jacques.

[jmarcelino]

@jellium
The source on github hasn't been updated to 1.7.6 yet

[jellium]

@jellium Has anyone found a solution for this issue? I cannot find in the sources where the keys are handled and written in the flash during the first join() method call. I believe it's somewhere in the LoRa constructor but the Python language merged in C is not easily readable for me.

Thanks!
It's the only "bug" I see left on the optimization of LoRaWAN and deepsleep together.

[robert-hh]

@peekay123 I run a similar test with every build since @this-wiederkehr raised that issue with version 1.7.3.b1. No luck. It may run very long, up to 600000 cycles on one run, a few thousand on the next. But it's still not stable.

[johncaipa]

Hi, Now I have the following messages:
NVS error=4354

Does anyone know what it means?

[this.wiederkehr]

@peekay123: Just raised a hand for help in the espressif forums about the bug with the sd-card.

See here: https://www.esp32.com/viewtopic.php?f=2&t=2421

Looks like a memory corruption bug, where some pointer gets overwritten due to an out-of-bounds access.

Unfortunatly I'm working with 1.7.5b2 sources as 1.7.6.b1 sources are still not available...

[peekay123]

Running an SD test on LoPy / Expansion board with firmware 1.7.6.b1 causes the LoPy to hang after an inconsistent number of cycles. This test ran flawlessly on 1.7.5.b2.

import machine
import os
import utime
import gc
from machine import SD

sd = SD()
os.mount(sd, '/sd')

# check the content
os.listdir('/sd')

f = open('/sd/test', 'a')
t1 = utime.ticks_ms()
ttot = 0
n = 0
print("Starting")
while n < 300000:
#   initializing buffer
    k = bytearray(1024)
#   write buffer and flush to file
    f.write(k)
    f.flush()
    t2 = utime.ticks_ms()
#	calculate running average
    ttot = (ttot + (t2 - t1)) /2
    gc.collect()
    n += 1
#    print("Round", n, ", ", ttot, "ms \n\r", end="")
    t1 = utime.ticks_ms()
    if n%1000 == 0:
        print("Rounds ", n, "\r", nend="")

print("\n\rRounds", n, ", avg ", ttot, "ms \n\r", end="")

[jellium]

@jmarcelino Thanks for the pieces of information.

Note that a call of LoRa(mode=LoRa.LORAWAN, adr=True, device_class=LoRa.CLASS_A) for instance reinitializes the frame counter. So, besides the encryption keys to communicate, part of the stored LoRaWAN context is lost or overwritten. So care must be taken not to redefine the LoRa object after a deepsleep or power cycle, in order not to reset some of the LoRaWAN context. Am I right?

Edit: it seems inconsistent to me not to redefine the LoRa object used for the "first" join, after a deepsleep or a power cycle. If one does not redefine this object, one cannot check whether the LoPy "has joined" the network. However, as I said above, reinitializing this object, at least, resets the uplink frame counter. Furthermore, if I sequentially send a frame, recall the LoRa constructor, send a frame, and so on, I only see downlinks in (Objenious) network backend, with increasing counter, but no uplink.

[jellium]

@Eric24 Hi, good point I guess. I ended up doing so yesterday, using machine.reset_cause() which returns 0 or 1 after sync or reset button, or 3 after a deep sleep. This function is quite handy to recover REPL when something goes wrong in the main program (for instance use the reset button to toggle a "debug" boolean which is set to inhibit the main program if set to true).

[iotmaker]

@iotmaker ohh i saw that a change and now you need to provide pin 9 and pin 10 on the initialization

[iotmaker]

How come when i upgrade the Sipy It says upgrading to 1.7.6.b1 and when ever i do the

os.uname().release it gives me 1.7.5.b2?

Also on 1.7.5.b2 I2C does not work...... I put a sipy with a 1.6.13.b1 on the same expansion board where the i2c sensor is and IT WORKS..

so how come upgrading the firmware broke my application?

[Eric24]

@jellium I think all you'd need to do is provide a mechanism (maybe a button press or a jumper during power-up, etc.) that the end-user could trigger a new LoRaWAN join. The new join will overwrite whatever has been stored in NVS (i.e. there's no need to specifically delete that information).

[jmarcelino]

@jellium
In LoRaWAN the concept of having "joined" a network only means you now hold the encryption keys to communicate with it. So if you've done OTAA and the keys are stored it's "joined" forever.

There is no way in LoRaWAN for a node to know if it's actually communicating with anything on the other side other than re-joining or somehow requesting a downlink. If the downlink fails to appear then your node will know it's no longer within range of the network (or maybe it's time to call lora.join() again)

[jellium]

@daniel said in New firmware release 1.7.6.b1:

• esp32: Store the LoRaWAN state in NVS, to avoid having to re-join after waking up from deep sleep (or a power cycle). Calling lora.has_joined() after deep sleep returns True if the network was previously joined.

Super good!

However, what do you mean by "if the network was previously joined"? Because now I can't manage to have lora.has_joined() returning False after a deep sleep (or a power cycle)... Suppose I have joined a LoRaWAN network. Then I switch off the board, unplug it, press its buttons to wipe any remaining power, unplug the antenna and plug the board back. From here, lora.has_joined() will return True, even if no network is actually joined. For instance, if from here I manually try to rejoin, it will fail (as expected since no antenna is plugged), and lora.has_joined() will return False, as it should (since no network is actually joined). In other words, it seems that after a deep sleep or power cycle, any call of LoRa(mode=LoRa.LORAWAN) will set has_joined() state to True even if no network is joined.
How can I manually "erase" the has_joined() state in from NVS?

Thanks!