LoPy Nano-Gateway Extended (Timeout and Retry)

Butch

@bucknall Alex, I'll share my current issue here so that hopefully others can benefit from my stupidity. :-)

I have the sample code above working. The node and gateway properly exchange message/ACK. The message being sent from the node is

DEVICE 1 HERE

To make this more useful, I want to change the message to some JSON reporting a temperature:

{"id": 1, "temperature": -14.3,"unit": "C","time":"14-Apr-2017@21:48:32"}

However, this fails. But it fails in a completely unexpected way. On the gateway:

recv_pkg = lora_sock.recv(512)

constantly receives zero-length packets. All I did was change the message being sent. I did not change the header, nor the struct. If I had a malformed header or payload, I would expect the recv to at least get something with a length greater than zero bytes.

Complete sample code for gateway:

#
# LoRaNanoGateway.py
#

import socket
import struct
from network import LoRa
import time

# A basic package header
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# %ds: Formated string for string
_LORA_PKG_FORMAT = "!BBB%ds"

# A basic ack package
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# B: 1 byte for the Ok (200) or error messages
_LORA_PKG_ACK_FORMAT = "BBBB"

# Let the world know we're starting up.

print("Starting LoRaNanoGateway")

# Open a Lora Socket, use rx_iq to avoid listening to our own messages
lora = LoRa(mode=LoRa.LORA, frequency=915000000, rx_iq=True)
lora_sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
lora_sock.setblocking(False)

while (True):

    # Since the maximum body size in the protocol is 255 the request is limited to 512 bytes
    recv_pkg = lora_sock.recv(512)
    print("Received message, length = %d" % len(recv_pkg))

    # If at least a message with the header is received process it
    if (len(recv_pkg) > 3):
        print("<<<<<<<<<<<<< Received VALID message, length = %d" % len(recv_pkg))

        recv_pkg_len = recv_pkg[1]

        # If message is corrupted should not continue processing
        if (not len(recv_pkg) == recv_pkg_len + 3):
            continue

        # Unpack the message based on the protocol definition
        device_id, pkg_len, msg_id, msg = struct.unpack(_LORA_PKG_FORMAT % recv_pkg_len, recv_pkg)

        # Respond to the device with an acknowledge package
        # time.sleep(0.15)
        print("+++ Received a message: %s" % msg)
        ack_pkg = struct.pack(_LORA_PKG_ACK_FORMAT, device_id, 1, msg_id, 200)
        print("------ Sending an ACK")
        lora_sock.send(ack_pkg)

        # Do any extra processing required for the package. Keep in mind it should be as fast as posible
        # to make sure that the other clients are not waiting too long for their messages to be acknoleged

Complete sample code for node:

#
# LoRaNanoNode.py
#

import os
import socket
import time
import struct
from network import LoRa
from uos import urandom

# A basic package header
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# %ds: Formated string for string
_LORA_PKG_FORMAT = "!BBB%ds"

# A basic ack package
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# B: 1 byte for the Ok (200) or error messages
_LORA_PKG_ACK_FORMAT = "BBBB"

# This device ID, use different device id for each device
_DEVICE_ID = 0x01
_MAX_ACK_TIME = 5000
_RETRY_COUNT = 3

# Let the world know we're starting up.

print("Starting LoRaNanoNode on device %d" % _DEVICE_ID)

# Open a Lora Socket, use tx_iq to avoid listening to our own messages
lora = LoRa(mode=LoRa.LORA, frequency=915000000, tx_iq=True)
lora_sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
lora_sock.setblocking(False)

# Method to increase message id and keep in between 1 and 255
msg_id = 0
def increase_msg_id():
    global msg_id
    msg_id = (msg_id + 1) & 0xFF

# Method for acknoledge waiting time keep
def check_ack_time(from_time):
    current_time = time.ticks_ms()
    return (current_time - from_time > _MAX_ACK_TIME)

# Method to send messages
def send_msg(msg):
    global msg_id
    retry = _RETRY_COUNT
    while (retry > 0 and not retry == -1):
        retry -= 1
        pkg = struct.pack(_LORA_PKG_FORMAT % len(msg), _DEVICE_ID, len(msg), msg_id, msg)
        bytes_sent = lora_sock.send(pkg)
        print(">>>>>>>>>>>>>> Sending package %s, length = %d, bytes sent = %d" % (pkg, len(pkg), bytes_sent))

        # Wait for the response from the server.
        start_time = time.ticks_ms()

        while(not check_ack_time(start_time)):
            recv_ack = lora_sock.recv(256)
            # If a message of the size of the acknoledge message is received
            if (len(recv_ack) == 4):
                device_id, pkg_len, recv_msg_id, status = struct.unpack(_LORA_PKG_ACK_FORMAT, recv_ack)
                if (device_id == _DEVICE_ID and recv_msg_id == msg_id):
                    if (status == 200):
                        # Do some code if your message arrived at the central
                        return True
                    else:
                        return False
        time.sleep_ms(urandom(1)[0] << 2)
    return False

# Main Loop
while(True):

    msg = '{"id": %s, "temperature": -14.3,"unit": "C","time":"14-Apr-2017@21:48:32"}' % _DEVICE_ID
    # msg = "DEVICE %d HERE" % _DEVICE_ID

    print("Sending a message: %s, length = %d" % (msg, len(msg)))

    # success = send_msg("DEVICE %d HERE" % _DEVICE_ID)
    success = send_msg(msg)
    if (success):
        print("ACK RECEIVED: %d" % msg_id)
        increase_msg_id()
    else:
        print("MESSAGE FAILED")
        # Manage the error message
#
# >>>>>>>>>>>>>> Sending package b'\x01I\x00{"id": 1, "temperature": -14.3,"unit": "C","time":"14-Apr-2017@21:48:32"}', length = 76
# >>>>>>>>>>>>>> Sending package b'\x01\r\x10DEVICE 1 HERE', length = 16
#

You can see that I have dumped the raw packed structure, and that it sure looks like it's properly formed. ASCII '\r' corresponds to 13, which is the length of the payload in the original sample message, and ASCII 'I' corresponds to 73, which is the length of the payload in my new message. The device ID in the header is the same, and the message ID shouldn't matter, right?

My understanding is that the payload size is limited to 255 bytes, so I don't think I'm running into a space issue.

I should also add that I'm running 1.6.8b2 on both boards:

>>> import os
>>> os.uname()
(sysname='LoPy', nodename='LoPy', release='1.6.8.b2', version='v1.8.6-530-gca8bdcbb on 2017-03-21', machine='LoPy with ESP32', lorawan='1.0.0')

Any help anyone can provide would be greatly appreciated.

Thanks!

gouao

Can a seeeduino lorawan communicate with the lopys ?
Can it act as a node, wich is recognized by the lopy nano-gateway ?

Butch

Following up on my own post (which seems to be a theme for me), I discovered that it is, indeed, the length of the payload that's causing the problem. If I limit the payload to 61 bytes (plus a three-byte header), it works:

Sending a message: 1234567890123456789012345678901234567890123456789012345678901, length = 61
>>>>>>>>>>>>>> Sending package b'\x01=\x001234567890123456789012345678901234567890123456789012345678901', length = 64, bytes sent = 64
ACK RECEIVED: 0

If, however, I increase the length of the payload to 62 bytes (plus a three-byte header), it fails:

Sending a message: 12345678901234567890123456789012345678901234567890123456789012, length = 62
>>>>>>>>>>>>>> Sending package b'\x01>\x0012345678901234567890123456789012345678901234567890123456789012', length = 65, bytes sent = 65
>>>>>>>>>>>>>> Sending package b'\x01>\x0012345678901234567890123456789012345678901234567890123456789012', length = 65, bytes sent = 65
>>>>>>>>>>>>>> Sending package b'\x01>\x0012345678901234567890123456789012345678901234567890123456789012', length = 65, bytes sent = 65
MESSAGE FAILED

Any ideas?

Thanks!

Butch

And, once again replying to my own post, I understand why:

LoRa spec:

7.2.6 US902-928 Maximum payload size

2 The maximum MACPayload size length (M) is given by the following table. It is derived from
3 the maximum allowed transmission time at the PHY layer taking into account a possible
4 repeater encapsulation. The maximum application payload length in the absence of the
5 optional FOpt MAC control field (N) is also given for information only. The value of N might
6 be smaller if the FOpt field is not empty:
7

DataRate       M         N
0              19        11
1              61        53
2              137       129
3              250       242
4              250       242
5:7             Not defined
8              41        33
9              117       109
10             230       222
11             230       222
12             230       222
13             230       222
14:15           Not defined

So, as you can see, at data rate 1, the maximum payload size is 61 bytes. If I want to send more, I have to increase the data rate.

My question now is how do I increase the data rate? Ideally, I'd like to use data rate 3.

It looked like I could just do this:

lora_sock.setsockopt(socket.SOL_LORA, socket.SO_DR, 3)

That code completes with no complaint, but when I make the change on both the node and gateway sides, it does not change the maximum payload size.

After playing around with it some more, 4 seems to be the maximum supported, in spite of what the chart above says.

Any ideas? Please?

Thanks!

Colateral

@Butch Are you using 1.6.11b1? How exactly instantiated lora stack for the GW and the Node? I mean all can yu enumerate all params that you configured for the lora object?

Butch

@Colateral Thanks for your reply. My post below provides all the answers to the questions you asked. Not only does it have the version number of the firmware that I'm using, it also has all the code for both the node and the gateway.

fbt

Hi @Roberto , in case i wanted to support more than 2 nodes, i.e 8 nodes, which parameters should i modify on the gateway code?. thanks

fbt

Hi @gouao, did you get any answer about the compatibility of seeeduino lorawan and lopy? thanks

gouao

@fbt hello mate, unfortunately i could't get any informations concerning this..
If you find out something please let me know !

All the best!

fbt

Thanks @gouao ... any reply about this issue @jmarcelino ? thanks

jmarcelino

@fbt
For a LoRaWAN device you'd usually want to be using https://github.com/pycom/pycom-libraries/tree/master/examples/lorawan-nano-gateway and not the code on the post. This code is for raw LoRa only.

8 devices on the LoRaWAN gateway shouldn't be a problem as long as they don't transmit at the same time.

heut

Hi @Roberto and @Butch

Thank you very much for the interesting discussion. I'm currently trying to get the plain code (node_1.py and nano_gateway.py) working on my LoPy(s), without success so far. I have both updated to the latest Firmware for Switzerland and sitting in an extension board with antennas attached. When running the code respective code on the LoPys, not a single message is delivered and acknowledged. All messages that are received on the gateway node have the size 0.
From time to time the node_1.py LoPy shows a longer LED blink in blue instead of the heartbeat.

Any idea what I might be doing wrong?

Best
heut

seb

@heut try adding a delay in your loop, e.g. 50ms, this seemed to vastly improve the situation for me

heut

@seb Thank you for the hint. However, that didn't make a difference in my situation at all. There is still not a single message transmitted. Any other suggestions what it might be?

Just to be sure, here the sources I'm using.

Gateway:

import socket
import struct
from network import LoRa
import time

# A basic package header
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# %ds: Formated string for string
_LORA_PKG_FORMAT = "!BBB%ds"

# A basic ack package
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# B: 1 byte for the Ok (200) or error messages
_LORA_PKG_ACK_FORMAT = "BBBB"

# Open a Lora Socket, use rx_iq to avoid listening to our own messages
#lora = LoRa(mode=LoRa.LORA, tx_iq=True, frequency=870000000, sf=7)
lora = LoRa(mode=LoRa.LORA, tx_iq=True)
lora_sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
lora_sock.setblocking(False)

while (True):

    # Since the maximum body size in the protocol is 255 the request is limited to 512 bytes
    recv_pkg = lora_sock.recv(512)


    # If at least a message with the header is received process it
    if (len(recv_pkg) > 3):
        print("Package received")
        recv_pkg_len = recv_pkg[1]

        # If message is corrupted should not continue processing
        #if (not len(recv_pkg) == recv_pkg_len + 3):
            #continue

        # Unpack the message based on the protocol definition
        device_id, pkg_len, msg_id, msg = struct.unpack(_LORA_PKG_FORMAT % recv_pkg_len, recv_pkg)

        # Respond to the device with an acknoledge package
        # time.sleep(0.15)
        ack_pkg = struct.pack(_LORA_PKG_ACK_FORMAT, device_id, 1, msg_id, 200)
        lora_sock.send(ack_pkg)

        print("sent ack")

    time.sleep(0.2)

        # Do any extra processing required for the package. Keep in mind it should be as fast as posible
        # to make sure that the other clients are not waiting too long for their messages to be acknoleged

Node:

import os
import socket
import time
import struct
from network import LoRa
from uos import urandom

# A basic package header
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# %ds: Formated string for string
_LORA_PKG_FORMAT = "!BBB%ds"

# A basic ack package
# B: 1 byte for the deviceId
# B: 1 byte for the pkg size
# B: 1 byte for the messageId
# B: 1 byte for the Ok (200) or error messages
_LORA_PKG_ACK_FORMAT = "BBBB"

# This device ID, use different device id for each device
_DEVICE_ID = 0x01
_MAX_ACK_TIME = 5000
_RETRY_COUNT = 3


# Open a Lora Socket, use tx_iq to avoid listening to our own messages
#lora = LoRa(mode=LoRa.LORA, tx_iq=True, frequency=870000000, sf=7)
lora = LoRa(mode=LoRa.LORA, tx_iq=True)
lora_sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
lora_sock.setblocking(False)

# Method to increase message id and keep in between 1 and 255
msg_id = 0
def increase_msg_id():
    global msg_id
    msg_id = (msg_id + 1) & 0xFF

# Method for acknoledge waiting time keep
def check_ack_time(from_time):
    current_time = time.ticks_ms()
    return (current_time - from_time > _MAX_ACK_TIME)

# Method to send messages
def send_msg(msg):
    print("SEND MSG")
    print(msg)
    global msg_id
    retry = _RETRY_COUNT
    while (retry > 0 and not retry == -1):
        retry -= 1
        pkg = struct.pack(_LORA_PKG_FORMAT % len(msg), _DEVICE_ID, len(msg), msg_id, msg)
        lora_sock.send(pkg)

        # Wait for the response from the server.
        start_time = time.ticks_ms()

        while(not check_ack_time(start_time)):
            #print("RECV_ACK")
            recv_ack = lora_sock.recv(256)
            # If a message of the size of the acknoledge message is received
            if (len(recv_ack) == 4):
                print("SIZE CORRECT")
                device_id, pkg_len, recv_msg_id, status = struct.unpack(_LORA_PKG_ACK_FORMAT, recv_ack)
                if (device_id == _DEVICE_ID and recv_msg_id == msg_id):
                    if (status == 200):
                        # Do some code if your message arrived at the central
                        return True
                    else:
                        return False
        time.sleep_ms(urandom(1)[0] << 2)
    return False

# Main Loop
while(True):

    success = send_msg("DEVICE %d HERE" % _DEVICE_ID)
    if (success):
        print("ACK RECEIVED: %d" % msg_id)
        increase_msg_id()
    else:
        print("MESSAGE FAILED")
        # Manage the error message

seb

@heut I will have a closer look at your code now but in the mean time, a few things that often catch users out:

Make sure both LoPy modules are running the same firmware
Please set the frequency to ensure it is the same on both

One thing I have already noticed is you have set tx_iq on both the node and the gateway, it should be rx_iq=True for the gateway and tx_iq=True on the nodes.

heut

@seb Thank you so much, that was the point! Setting rx_iq instead of tx_iq True on the Gateway solved the issue. And frequency is set as well.

Archana Kadam

@roberto
I have used the above code and the communication between the one node and gateway is generated, but the device id and package. What is the procedure to get the exact connected device.Please update as soon as possible your help will be appreciable.

Thank You

Explore Pybytes | Official Documentation | Report a Firmware Bug/Issue | GitHub

LoPy Nano-Gateway Extended (Timeout and Retry)

Pycom on Twitter