TTN ABP from LoPy4 loses many packets despite low RSSI



  • I have several LoPy4 and a Laird TTN gateway in the same room. I've noticed that out of hundreds of uplinked packets listed on TTN frame counter, only a handful actually show. I have simple 4 byte payloads being sent every 5 minutes. The data changes a small amount each time as it's measuring dust particulate in Parts per Billion. Is that still too frequent to trigger airtime limits? It says Estimated Airtime 30.976ms
    Doing a similar project with a Heltec ESP32 Lora board written in C, we were getting very high success rates. My friend who suggested I try Pycom is having similar issues.
    Here's one of the received packets shown on the TTN site
    {
    "time": "2019-01-20T02:09:37.924262461Z",
    "frequency": 905.1,
    "modulation": "LORA",
    "data_rate": "SF7BW125",
    "coding_rate": "4/5",
    "gateways": [
    {
    "gtw_id": "eui-c0ee40ffff29477e",
    "gtw_trusted": true,
    "timestamp": 1571763780,
    "time": "",
    "channel": 6,
    "rssi": -33,
    "snr": 10.75,
    "rf_chain": 1,
    "latitude": 33.539913,
    "longitude": -86.7531
    }
    ]
    }
    And my current code

    # main.py -- put your code here!
    from network import LoRa
    import socket
    import ubinascii
    import struct
    import time
    import os
    import math
    import abpkeys #local py file with the abp keys stored for easy code reuse across devices
    from machine import UART
    
    uart = UART(1, 9600)                         # init with given baudrate
    uart.init(9600, bits=8, parity=None, stop=1) # init with given parameters
    
    # Initialise LoRa in LORAWAN mode.
    # Please pick the region that matches where you are using the device:
    # Asia = LoRa.AS923
    # Australia = LoRa.AU915
    # Europe = LoRa.EU868
    # United States = LoRa.US915
    lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.US915)
    
    # create an ABP authentication params
    dev_addr=abpkeys.dev_addr
    nwk_swkey=abpkeys.nwk_swkey
    app_swkey=abpkeys.app_swkey
    #dev_addr = struct.unpack(">l", ubinascii.unhexlify('xxxxxxxx'))[0]
    #nwk_swkey = ubinascii.unhexlify('xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')
    #app_swkey = ubinascii.unhexlify('xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')
    print('restoring LoRa NVRAM')
    lora.nvram_restore()
    print('Joining LoRa via ABP')
    # join a network using ABP (Activation By Personalization)
    lora.join(activation=LoRa.ABP, auth=(dev_addr, nwk_swkey, app_swkey))
    
    # create a LoRa socket
    s = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
    
    # set the LoRaWAN data rate
    s.setsockopt(socket.SOL_LORA, socket.SO_DR, 3)
    
    while (1):
        readytosend=False
        print('polling Dust Sensor')
        data = uart.readall()# get everythitng from the serial buffer
        if data is not None:#make sure it'st not an empty buffer
            bytestream=data.split(b'BM')[1][0:31]# HPMA sends 32 byte packets that start with \x42\x4D ('BM')
            if len(bytestream)==30:#check tthat we actually got 30 bytes (excluding the data we split out)
                if bytestream[0:2]==b'\x00\x1c':#HPMA packet is always 28 data bytes(excluding start header and checksum)
                    checksum=143#since split command removed the \x42\x4D, add them back to get correct checksum
                    validated=0
                    for i in range(0,28):
                        checksum+=bytestream[i]#cycle thru the data values and add them up
                    validated=int(bytestream[28])*256+int(bytestream[29])#read checksum bytes from packet and make them a 2-byte integer
                    if checksum==validated:# both values should match, else we have a corrupt packet
                        pm25=int(bytestream[4])*256+int(bytestream[5])#these two bytes represent the particulate detected that's 2.5um large
                        pm10=int(bytestream[6])*256+int(bytestream[7])#these two bytes represent the particulate detected that's 10um large
                        print('pm2.5 = ',pm25)
                        print('pm10 = ',pm10)
                        readytosend=True
                    else:
                        print('checksum failed')
                else:
                    print('invalid packet length')
            else:
                print('incomplete packet')
        if readytosend==True:
            # make the socket blocking
            # (waits for the data to be sent and for the 2 receive windows to expire)
            s.setblocking(True)
            print("Sending data!")
            # send some data
            # s.send(bytes([0x01, 0x02, 0x03]))
            s.send(bytes([bytestream[4], bytestream[5], bytestream[6],bytestream[7]]))
            pybytes.send_virtual_pin_value(False, 0, (bytestream[4]<<8)+bytestream[5])
            time.sleep(1)
            pybytes.send_virtual_pin_value(False, 1, (bytestream[6]<<8)+bytestream[7])
            time.sleep(1)
            #pybytes.send_virtual_pin_value(False,2,100)
            #time.sleep(10)
    
        print('saving LoRa NVRAM')
        lora.nvram_save()
        # make the socket non-blocking
        # (because if there's no data received it will block forever...)
        s.setblocking(False)
        print("receiving data!")
        # get any data received (if any...)
        data = s.recv(64)
        print(data)
        print("Go to sleep 5 minutes!")
        time.sleep(300)```


  • @jcaron Thanks for recommending the range adjustment. I've added that to my code.



  • @misterlisty
    I still need to build a TTN utility to visualize the data captured, but based on when I have my TTN page open looking at data received, it looks like I'm getting greater than 95% packets at my close range. The web page doesn't seem to like being open for more than 45 minutes or so before I get some page error, but during that period, I seem to see packets at the 5 minute intervals set in my software. So it's definitely worth the effort to capture back any info you can about what channels are used and delete all the unused ones for a particular location..
    Now, if only I can find an automated way to do this... (not have to hard-code the channels but let the code figure them out..)



  • @dnear1 There are a lot of possible uplink channels in the US915 band (a total of 72). Most gateways (including the Multitech gateway) can only listen on a small subset of those channels at a given time.

    The Multitech gateway has a setting to select the "sub-band", which is a set of 8 consecutive channels. Apparently you're on the second sub-band (don't know if the sub-band numbering starts at 0 or 1, so it's either sub-band 1 or 2), which covers channels 8 to 15.

    Note: the range operator in python does not include the upper bound, so you want range(0,72), not range(0,71).



  • @misterlisty
    It's early to tell, what with 5 minute delays between packets, but after sniffing packets on TTN all day to see what channels were received, I was able to figure out which 8 channels to add and get 4 packets in succession thru.

    from network import LoRa
    import socket
    import ubinascii
    import struct
    import time
    import os
    import math
    import abpkeys
    from machine import UART
    
    uart = UART(1, 9600)                         # init with given baudrate
    uart.init(9600, bits=8, parity=None, stop=1) # init with given parameters
    
    # Initialise LoRa in LORAWAN mode.
    # Please pick the region that matches where you are using the device:
    # Asia = LoRa.AS923
    # Australia = LoRa.AU915
    # Europe = LoRa.EU868
    # United States = LoRa.US915
    lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.US915)
    
    # create an ABP authentication params
    #my keys are stored in abpkeys.py then linked here so I have the same source running 
    #on multiple devices
    dev_addr=abpkeys.dev_addr
    nwk_swkey=abpkeys.nwk_swkey
    app_swkey=abpkeys.app_swkey
    #dev_addr = struct.unpack(">l", ubinascii.unhexlify('xxxxxxxx'))[0]
    #nwk_swkey = ubinascii.unhexlify('xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')
    #app_swkey = ubinascii.unhexlify('xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')
    for i in range(0, 72):
        lora.remove_channel(i)
    print('Removed default channels')
    time.sleep(1)
        
        # Set US ISM 915 channel plan for TTN US
    lora.add_channel(0, frequency=903900000, dr_min=0, dr_max=3)
    lora.add_channel(1, frequency=904100000, dr_min=0, dr_max=3)
    lora.add_channel(2, frequency=904300000, dr_min=0, dr_max=3)
    lora.add_channel(3, frequency=904500000, dr_min=0, dr_max=3)
    lora.add_channel(4, frequency=904700000, dr_min=0, dr_max=3)
    lora.add_channel(5, frequency=904900000, dr_min=0, dr_max=3)
    Lora.add_channel(6, frequency=905100000, dr_min=0, dr_max=3)
    lora.add_channel(7, frequency=905300000, dr_min=0, dr_max=3)
    #channel	8:	903900000	hz	min_dr	0	max_dr	3				
    #channel	9:	904100000	hz	min_dr	0	max_dr	3				
    #channel	10:	904300000	hz	min_dr	0	max_dr	3				
    #channel	11:	904500000	hz	min_dr	0	max_dr	3				
    #channel	12:	904700000	hz	min_dr	0	max_dr	3				
    #channel	13:	904900000	hz	min_dr	0	max_dr	3				
    #channel	14:	905100000	hz	min_dr	0	max_dr	3				
    #channel	15:	905300000	hz	min_dr	0	max_dr	3				
    
    print('US channels set')
    time.sleep(1)
    
    
    print('restoring LoRa NVRAM')
    lora.nvram_restore()
    print('Joining LoRa via ABP')
    # join a network using ABP (Activation By Personalization)
    lora.join(activation=LoRa.ABP, auth=(dev_addr, nwk_swkey, app_swkey))
    
    # create a LoRa socket
    s = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
    
    # set the LoRaWAN data rate
    s.setsockopt(socket.SOL_LORA, socket.SO_DR, 3)
    #The first 5 seconds of dust data always seems irregular, so let's delay and clear the buffer
    data = uart.readall()
    time.sleep(10)
    data= uart.readall()
    time.sleep(1)# then give it another second for the next good byte to come in.
    while (1):
        readytosend=False
        print('polling Dust Sensor')
        data = uart.readall()# get everythitng from the serial buffer
        if data is not None:#make sure it'st not an empty buffer
            bytestream=data.split(b'BM')[1][0:31]# HPMA sends 32 byte packets that start with \x42\x4D ('BM')
            if len(bytestream)==30:#check tthat we actually got 30 bytes (excluding the data we split out)
                if bytestream[0:2]==b'\x00\x1c':#HPMA packet is always 28 data bytes(excluding start header and checksum)
                    checksum=143#since split command removed the \x42\x4D, add them back to get correct checksum
                    validated=0
                    for i in range(0,28):
                        checksum+=bytestream[i]#cycle thru the data values and add them up
                    validated=int(bytestream[28])*256+int(bytestream[29])#read checksum bytes from packet and make them a 2-byte integer
                    if checksum==validated:# both values should match, else we have a corrupt packet
                        pm25=int(bytestream[4])*256+int(bytestream[5])#these two bytes represent the particulate detected that's 2.5um large
                        pm10=int(bytestream[6])*256+int(bytestream[7])#these two bytes represent the particulate detected that's 10um large
                        print('pm2.5 = ',pm25)
                        print('pm10 = ',pm10)
                        readytosend=True
                    else:
                        print('checksum failed')
                else:
                    print('invalid packet length')
            else:
                print('incomplete packet')
        if readytosend==True:
            # make the socket blocking
            # (waits for the data to be sent and for the 2 receive windows to expire)
            s.setblocking(True)
            print("Sending data!")
            # send some data
            # s.send(bytes([0x01, 0x02, 0x03]))
            s.send(bytes([bytestream[4], bytestream[5], bytestream[6],bytestream[7]]))
            pybytes.send_virtual_pin_value(False, 0, (bytestream[4]<<8)+bytestream[5])
            time.sleep(1)
            pybytes.send_virtual_pin_value(False, 1, (bytestream[6]<<8)+bytestream[7])
            time.sleep(1)
            #pybytes.send_virtual_pin_value(False,2,100)
            #time.sleep(10)
    
        print('saving LoRa NVRAM')
        lora.nvram_save()
        # make the socket non-blocking
        # (because if there's no data received it will block forever...)
        s.setblocking(False)
        print("receiving data!")
        # get any data received (if any...)
        data = s.recv(64)
        print(data)
        print("Go to sleep 5 minutes!")
        time.sleep(300)```


  • I'm find a similar issue with Multitech Gateway and pycom but was assuming that Lorawan operates this way.

    I have a Multitech lora gateway in the same room as pycom and i get about 60% success rate!


Log in to reply
 

Pycom on Twitter