LoPy Nano-Gateway Extended (Timeout and Retry)

Roberto

The following example is based on the one previously posted here:
LoPy Nano-Gateway

The original code shared messages over LoRa between several LoPys using one of them as a Nano-Gateway and the remaining ones as nodes.

Keep in mind that for this example the communication works as follows:

LoPy 1 -------> Nano-Gateway LoPy <------- LoPy 2

The nodes send messages to the Nano-Gateway for further processing, this might include sending the messages over WiFi to a server for example.

The following code its an extension of the original one. It includes a couple of new features:

Maximum waiting for acknowledge time:
After sending a message, each node waits for a response from the server with an acknowledge package.
In the original code this process took an infinite amount of time, meaning that if the package or the acknowledge where not delivered the node would stop sending further messages.
In this new version, a maximum waiting time is introduced and set to a default of 5 secs. This allows the node to stop listening for an acknowledge and continue with other tasks. Change the default time depending on your application, especially if you are using different data rates or your processing times of the package in the Nano-Gateway are too long

Message retry:
If the acknowledge of the message never arrives, or another message arrives for a different device or message_id, the device will re-send the current package up to a maximum of 3 times (default) before setting the package as not send. This will allow for a more robust communication and will fix issues when two devices send messages at the same time causing a collision. Keep in mid that without a proper LoRa Gateway there is no collision management and all devices can send at the same time on the same band.
To further avoid collisions, a random delay is added before trying to resend a message. If a collision occurred the involved devices will send their messages at different time intervals.

Note: If you are in an area where other LoRa devices are being used, check if the band you are using is free, is not try a different band.

Nano Gateway Files:

adrian12310407

hallo and good day
i'm beginner in micrpython
and i have this error when i want use lopy as gateway

Running gatewaypy.py
/home/danicampora/Code/pycom/esp-idf-new/components/freertos/./queue.c:1326 (xQueueGenericReceive)- assert failed!
Guru Meditation Error: Core 0 panic'ed.
Register dump:
PC : Guru Meditation Error of type LoadProhibited occurred on core 0. Exception was unhandled.
Register dump:

I do not understand why that mistake
I hope you can help me
thank you!!

livius

@adrian12310407
what version of firmware? You can heck by:
os.uname()

adrian12310407

this is the version
(sysname='ESP32', nodename='ESP32', release='0.9.0b', version='8785822 on 2016-10-06', machine='LoPy with ESP32')

livius

@adrian12310407
you must update firmware
current version is 1.4.0.b1
https://www.pycom.io/support/supportdownloads/

adrian12310407

Thanks, I'll prove it

rvandam

Thank you for these examples!

Could you add a print(msg) in the gateway code?

It is more consistent with the previous example, and you know quicker that it works (should have started after the coffee this morning :-) )

gouao

This post is deleted!

Butch

I've tried the examples, but I am not having any success. The gateway is not receiving any messages with length > 0. To the best of my knowledge, there are no other LoRa devices in use, and I am using only one gateway and one node.

Any suggestions on how to debug this?

Thanks!

gerkez

Hello Roberto,

Where do I change the band?
I don't see it in your scripts

Butch

@Butch said in LoPy Nano-Gateway Extended (Timeout and Retry):

I've tried the examples, but I am not having any success. The gateway is not receiving any messages with length > 0. To the best of my knowledge, there are no other LoRa devices in use, and I am using only one gateway and one node.

Any suggestions on how to debug this?

Thanks!

Following up on my own post (poor form, I know, but what to do?), I thought I'd provide more information:

I am trying to use one LoPy on an expansion board with antenna as a gateway, and one with the same setup as a node.
I am running this code on the gateway (http://pastebin.com/PDCnq2gF) and this on the node (http://pastebin.com/Ztf5QMWY).
I am seeing constant messages of zero length being received on the gateway:

Starting LoRaNanoGateway
Received message, length = 0
Received message, length = 0
Received message, length = 0
Received message, length = 0
Received message, length = 0
Received message, length = 0

I am seeing no acknowledgements on the node:

Starting LoRaNanoNode on device 1
Sending a message from device 1
MESSAGE FAILED
Sending a message from device 1
MESSAGE FAILED
Sending a message from device 1
MESSAGE FAILED
Sending a message from device 1
MESSAGE FAILED

I have tried four different LoPy's in two different physical locations, so I don't think I'm having any RF pollution issues.

I could really use some help with this.

Thanks!

bucknall

Hi @Butch

There's previously was a small bug in the gateway and node code where you need to specify the spread factor (e.g. sf=7).

This has since been patched in the latest firmware release, please try to update your device and your code should now work!

Cheers!

Butch

@bucknall said in LoPy Nano-Gateway Extended (Timeout and Retry):

Hi @Butch

There's previously was a small bug in the gateway and node code where you need to specify the spread factor (e.g. sf=7).

This has since been patched in the latest firmware release, please try to update your device and your code should now work!

Cheers!

It works! Thank you so much!

Butch

Colateral

@Roberto Doesn't work anymore with. Simply GW not getting the message.

(sysname='LoPy', nodename='LoPy', release='1.6.10.b1', version='v1.8.6-556-g989d5ac9 on 2017-03-30', machine='LoPy with ESP32', lorawan='1.0.0')

Any ideas why?

Butch

@bucknall Alex, I'll share my current issue here so that hopefully others can benefit from my stupidity. :-)

I have the sample code above working. The node and gateway properly exchange message/ACK. The message being sent from the node is

DEVICE 1 HERE

To make this more useful, I want to change the message to some JSON reporting a temperature:

{"id": 1, "temperature": -14.3,"unit": "C","time":"14-Apr-2017@21:48:32"}

However, this fails. But it fails in a completely unexpected way. On the gateway:

recv_pkg = lora_sock.recv(512)

constantly receives zero-length packets. All I did was change the message being sent. I did not change the header, nor the struct. If I had a malformed header or payload, I would expect the recv to at least get something with a length greater than zero bytes.

Complete sample code for gateway:

#
# LoRaNanoGateway.py
#

import socket
import struct
from network import LoRa
import time

# A basic package header
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# %ds: Formated string for string
_LORA_PKG_FORMAT = "!BBB%ds"

# A basic ack package
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# B: 1 byte for the Ok (200) or error messages
_LORA_PKG_ACK_FORMAT = "BBBB"

# Let the world know we're starting up.

print("Starting LoRaNanoGateway")

# Open a Lora Socket, use rx_iq to avoid listening to our own messages
lora = LoRa(mode=LoRa.LORA, frequency=915000000, rx_iq=True)
lora_sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
lora_sock.setblocking(False)

while (True):

    # Since the maximum body size in the protocol is 255 the request is limited to 512 bytes
    recv_pkg = lora_sock.recv(512)
    print("Received message, length = %d" % len(recv_pkg))

    # If at least a message with the header is received process it
    if (len(recv_pkg) > 3):
        print("<<<<<<<<<<<<< Received VALID message, length = %d" % len(recv_pkg))

        recv_pkg_len = recv_pkg[1]

        # If message is corrupted should not continue processing
        if (not len(recv_pkg) == recv_pkg_len + 3):
            continue

        # Unpack the message based on the protocol definition
        device_id, pkg_len, msg_id, msg = struct.unpack(_LORA_PKG_FORMAT % recv_pkg_len, recv_pkg)

        # Respond to the device with an acknowledge package
        # time.sleep(0.15)
        print("+++ Received a message: %s" % msg)
        ack_pkg = struct.pack(_LORA_PKG_ACK_FORMAT, device_id, 1, msg_id, 200)
        print("------ Sending an ACK")
        lora_sock.send(ack_pkg)

        # Do any extra processing required for the package. Keep in mind it should be as fast as posible
        # to make sure that the other clients are not waiting too long for their messages to be acknoleged

Complete sample code for node:

#
# LoRaNanoNode.py
#

import os
import socket
import time
import struct
from network import LoRa
from uos import urandom

# A basic package header
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# %ds: Formated string for string
_LORA_PKG_FORMAT = "!BBB%ds"

# A basic ack package
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# B: 1 byte for the Ok (200) or error messages
_LORA_PKG_ACK_FORMAT = "BBBB"

# This device ID, use different device id for each device
_DEVICE_ID = 0x01
_MAX_ACK_TIME = 5000
_RETRY_COUNT = 3

# Let the world know we're starting up.

print("Starting LoRaNanoNode on device %d" % _DEVICE_ID)

# Open a Lora Socket, use tx_iq to avoid listening to our own messages
lora = LoRa(mode=LoRa.LORA, frequency=915000000, tx_iq=True)
lora_sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
lora_sock.setblocking(False)

# Method to increase message id and keep in between 1 and 255
msg_id = 0
def increase_msg_id():
    global msg_id
    msg_id = (msg_id + 1) & 0xFF

# Method for acknoledge waiting time keep
def check_ack_time(from_time):
    current_time = time.ticks_ms()
    return (current_time - from_time > _MAX_ACK_TIME)

# Method to send messages
def send_msg(msg):
    global msg_id
    retry = _RETRY_COUNT
    while (retry > 0 and not retry == -1):
        retry -= 1
        pkg = struct.pack(_LORA_PKG_FORMAT % len(msg), _DEVICE_ID, len(msg), msg_id, msg)
        bytes_sent = lora_sock.send(pkg)
        print(">>>>>>>>>>>>>> Sending package %s, length = %d, bytes sent = %d" % (pkg, len(pkg), bytes_sent))

        # Wait for the response from the server.
        start_time = time.ticks_ms()

        while(not check_ack_time(start_time)):
            recv_ack = lora_sock.recv(256)
            # If a message of the size of the acknoledge message is received
            if (len(recv_ack) == 4):
                device_id, pkg_len, recv_msg_id, status = struct.unpack(_LORA_PKG_ACK_FORMAT, recv_ack)
                if (device_id == _DEVICE_ID and recv_msg_id == msg_id):
                    if (status == 200):
                        # Do some code if your message arrived at the central
                        return True
                    else:
                        return False
        time.sleep_ms(urandom(1)[0] << 2)
    return False

# Main Loop
while(True):

    msg = '{"id": %s, "temperature": -14.3,"unit": "C","time":"14-Apr-2017@21:48:32"}' % _DEVICE_ID
    # msg = "DEVICE %d HERE" % _DEVICE_ID

    print("Sending a message: %s, length = %d" % (msg, len(msg)))

    # success = send_msg("DEVICE %d HERE" % _DEVICE_ID)
    success = send_msg(msg)
    if (success):
        print("ACK RECEIVED: %d" % msg_id)
        increase_msg_id()
    else:
        print("MESSAGE FAILED")
        # Manage the error message
#
# >>>>>>>>>>>>>> Sending package b'\x01I\x00{"id": 1, "temperature": -14.3,"unit": "C","time":"14-Apr-2017@21:48:32"}', length = 76
# >>>>>>>>>>>>>> Sending package b'\x01\r\x10DEVICE 1 HERE', length = 16
#

You can see that I have dumped the raw packed structure, and that it sure looks like it's properly formed. ASCII '\r' corresponds to 13, which is the length of the payload in the original sample message, and ASCII 'I' corresponds to 73, which is the length of the payload in my new message. The device ID in the header is the same, and the message ID shouldn't matter, right?

My understanding is that the payload size is limited to 255 bytes, so I don't think I'm running into a space issue.

I should also add that I'm running 1.6.8b2 on both boards:

>>> import os
>>> os.uname()
(sysname='LoPy', nodename='LoPy', release='1.6.8.b2', version='v1.8.6-530-gca8bdcbb on 2017-03-21', machine='LoPy with ESP32', lorawan='1.0.0')

Any help anyone can provide would be greatly appreciated.

Thanks!

gouao

Can a seeeduino lorawan communicate with the lopys ?
Can it act as a node, wich is recognized by the lopy nano-gateway ?

Butch

Following up on my own post (which seems to be a theme for me), I discovered that it is, indeed, the length of the payload that's causing the problem. If I limit the payload to 61 bytes (plus a three-byte header), it works:

Sending a message: 1234567890123456789012345678901234567890123456789012345678901, length = 61
>>>>>>>>>>>>>> Sending package b'\x01=\x001234567890123456789012345678901234567890123456789012345678901', length = 64, bytes sent = 64
ACK RECEIVED: 0

If, however, I increase the length of the payload to 62 bytes (plus a three-byte header), it fails:

Sending a message: 12345678901234567890123456789012345678901234567890123456789012, length = 62
>>>>>>>>>>>>>> Sending package b'\x01>\x0012345678901234567890123456789012345678901234567890123456789012', length = 65, bytes sent = 65
>>>>>>>>>>>>>> Sending package b'\x01>\x0012345678901234567890123456789012345678901234567890123456789012', length = 65, bytes sent = 65
>>>>>>>>>>>>>> Sending package b'\x01>\x0012345678901234567890123456789012345678901234567890123456789012', length = 65, bytes sent = 65
MESSAGE FAILED

Any ideas?

Thanks!

Butch

And, once again replying to my own post, I understand why:

LoRa spec:

7.2.6 US902-928 Maximum payload size

2 The maximum MACPayload size length (M) is given by the following table. It is derived from
3 the maximum allowed transmission time at the PHY layer taking into account a possible
4 repeater encapsulation. The maximum application payload length in the absence of the
5 optional FOpt MAC control field (N) is also given for information only. The value of N might
6 be smaller if the FOpt field is not empty:
7

DataRate       M         N
0              19        11
1              61        53
2              137       129
3              250       242
4              250       242
5:7             Not defined
8              41        33
9              117       109
10             230       222
11             230       222
12             230       222
13             230       222
14:15           Not defined

So, as you can see, at data rate 1, the maximum payload size is 61 bytes. If I want to send more, I have to increase the data rate.

My question now is how do I increase the data rate? Ideally, I'd like to use data rate 3.

It looked like I could just do this:

lora_sock.setsockopt(socket.SOL_LORA, socket.SO_DR, 3)

That code completes with no complaint, but when I make the change on both the node and gateway sides, it does not change the maximum payload size.

After playing around with it some more, 4 seems to be the maximum supported, in spite of what the chart above says.

Any ideas? Please?

Thanks!

Colateral

@Butch Are you using 1.6.11b1? How exactly instantiated lora stack for the GW and the Node? I mean all can yu enumerate all params that you configured for the lora object?

Butch

@Colateral Thanks for your reply. My post below provides all the answers to the questions you asked. Not only does it have the version number of the firmware that I'm using, it also has all the code for both the node and the gateway.

Explore Pybytes | Official Documentation | Report a Firmware Bug/Issue | GitHub

LoPy Nano-Gateway Extended (Timeout and Retry)

Pycom on Twitter