Pycom FiPy ADR not working on AS923 (TTN and Chirpstack)

[Gijs]

I would love to help you out there, but I do not really have a clue there unfortunately..
Maybe somebody else on the forum has a better idea on what is happening here, or you could try the Chirpstack forum (I believe you mention its Chirpstack related?)

[bcwong71]

@Gijs
Hi Gijs, I have been evaluating the ADR performance on the Chirpstack server. I found that the lorawan stack failed to change the data rate even when adr request was transmitted by the gateway. See the debug prints below. I have this feeling that there is still some kind of error somewhere. What is your advice?

[LoRaMac] adr set enable: 1
Joined
b''
datarate out 2, txpowout 0
[modlora] TASK_LoRa: taking semaphore 1
datarate out 2, txpowout 0
[LoRaMac] PrepareFrame: adrNext[LoRaMac] ScheduleTx: next channel 0, RX channel DR: 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[LoRaMac] SendFrameOnChannel: channel 0
[LoRaMac] SendFrameOnChannel: uplink frequency 923200000
[RegionAS923] RegionAS923RxConfig: RX config channel 0, frequency 923200000, datarate 2
[RegionAS923] LinkAdrReq: status variables drOut: 5, txPowOut:1, nbRepOut 1, nbBytesParsed:5
[modlora] McpsConfirm: Giving semaphore 4
Trying to send uplink
b''
datarate out 2, txpowout 0
[modlora] TASK_LoRa: taking semaphore 1
datarate out 2, txpowout 0
[LoRaMac] PrepareFrame: adrNext[LoRaMac] ScheduleTx: next channel 1, RX channel DR: 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 0
[LoRaMac] SendFrameOnChannel: channel 1
[LoRaMac] SendFrameOnChannel: uplink frequency 923400000
[RegionAS923] RegionAS923RxConfig: RX config channel 1, frequency 923400000, datarate 2
[RegionAS923] LinkAdrReq: status variables drOut: 5, txPowOut:2, nbRepOut 1, nbBytesParsed:5
[modlora] McpsConfirm: Giving semaphore 4
b''
datarate out 2, txpowout 0
[modlora] TASK_LoRa: taking semaphore 1
datarate out 2, txpowout 0
[LoRaMac] PrepareFrame: adrNext[LoRaMac] ScheduleTx: next channel 1, RX channel DR: 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 0
[LoRaMac] SendFrameOnChannel: channel 1
[LoRaMac] SendFrameOnChannel: uplink frequency 923400000
[RegionAS923] RegionAS923RxConfig: RX config channel 1, frequency 923400000, datarate 2
[RegionAS923] LinkAdrReq: status variables drOut: 5, txPowOut:2, nbRepOut 1, nbBytesParsed:5
[modlora] McpsConfirm: Giving semaphore 4```

No issue on TTN so far.

[bcwong71]

@Gijs
Hi, good news on my side as well. Using exactly your sample source employing either Activation via OTAA or ABP on Fipy flashed with the firmware your provided, I have successfully got ADR to work on both TTN and Chirpstack Server. On TTN server, the LinkAdrReq was transmitted by the gateway after every 65 packets (Exactly as what you shared). On Chirpstack server, the LinkAdrReq was transmitted by the gateway after every packet it had received. I would say that we have successfully solved the problem I was facing. However, can you please highlight again the changes that you have made to the source code of the firmware? Maybe these changes can be introduced in the next firmware version. Thank you very much and I appreciate your help greatly.

[Gijs]

Some more data points for you
This is the view from TTN node when we receive a ADR REQ
. There is no metadata in the packet visible.

This is the view from TTN Gateway, when we send a ADR REQ:

And this is the content of the packet: (not sure if that means anything to you)
60EA680126A5000003507F000136141F26

It seems like this happens every 65 packets on my side now, I just got another one at 129. I have the gateway setup to ttn-router-jp on AS923-1

Another edit, I just used the regular firmware, and can agree that I do not get a downlink message listed anywhere to change the datarate in AS923. It does however work for EU868 at 65 packet-intervals, both again on TTN.

[bcwong71]

Alright. Give me some time to work out the possible difference. The only obvious difference I can see now is the different activation modes. Mine is ABP and yours is OTAA. I will come back to you with more information.

[Gijs]

For me it took a while (like 100 packets while). Im using this script to test the setup on TTN, AS923: (very similar to yours)

from network import LoRa
import socket
import time
import ubinascii

# Initialise LoRa in LORAWAN mode.
# Please pick the region that matches where you are using the device:
# Asia = LoRa.AS923
# Australia = LoRa.AU915
# Europe = LoRa.EU868
# United States = LoRa.US915
lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.AS923, adr=True)

# create an OTAA authentication parameters, change them to the provided credentials
app_eui = ubinascii.unhexlify('')
app_key = ubinascii.unhexlify('')
#uncomment to use LoRaWAN application provided dev_eui
dev_eui = ubinascii.unhexlify('')
# join a network using OTAA (Over the Air Activation)
#uncomment below to use LoRaWAN application provided dev_eui
#lora.join(activation=LoRa.OTAA, auth=(app_eui, app_key), timeout=0)
lora.join(activation=LoRa.OTAA, auth=(dev_eui, app_eui, app_key), timeout=0)

# wait until the module has joined the network
while not lora.has_joined():
    time.sleep(2.5)
    print('Not yet joined...')

print('Joined')
# create a LoRa socket
s = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
# set the LoRaWAN data rate
s.setsockopt(socket.SOL_LORA, socket.SO_DR, 5)
while True:


    # make the socket blocking
    # (waits for the data to be sent and for the 2 receive windows to expire)
    s.setblocking(False)

    # send some data
    s.send(bytes([0x01, 0x02, 0x03]))
    # make the socket non-blocking
    # (because if there's no data received it will block forever...)
    s.setblocking(False)

    # get any data received (if any...)
    data = s.recv(64)
    print(data)
    time.sleep(5)

[bcwong71]

@Gijs
Hi Gijs, I flashed my FiPy with the firmware you provided and tried AS923 on TTN. See the debug prints below:

[modlora] lora_init_helper: Giving semaphore 3
[LoRaMac] adr set enable: 1
datarate out 2, txpowout 0
[modlora] TASK_LoRa: taking semaphore 1
datarate out 2, txpowout 0
[LoRaMac] PrepareFrame: adrNext[LoRaMac] ScheduleTx: next channel 0, RX channel DR: 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[LoRaMac] SendFrameOnChannel: channel 0
[LoRaMac] SendFrameOnChannel: uplink frequency 923200000
[RegionAS923] RegionAS923RxConfig: RX config channel 0, frequency 923200000, datarate 2
[RegionAS923] RegionAS923RxConfig: RX config channel 0, frequency 923200000, datarate 2
[modlora] McpsConfirm: Giving semaphore 4
datarate out 2, txpowout 0
[modlora] TASK_LoRa: taking semaphore 1
datarate out 2, txpowout 0
[LoRaMac] PrepareFrame: adrNext[LoRaMac] ScheduleTx: next channel 1, RX channel DR: 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[LoRaMac] SendFrameOnChannel: channel 1
[LoRaMac] SendFrameOnChannel: uplink frequency 923400000
[RegionAS923] RegionAS923RxConfig: RX config channel 1, frequency 923400000, datarate 2
[RegionAS923] RegionAS923RxConfig: RX config channel 1, frequency 923200000, datarate 2
[modlora] McpsConfirm: Giving semaphore 4
datarate out 2, txpowout 0
[modlora] TASK_LoRa: taking semaphore 1
datarate out 2, txpowout 0
[LoRaMac] PrepareFrame: adrNext[LoRaMac] ScheduleTx: next channel 0, RX channel DR: 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[LoRaMac] SendFrameOnChannel: channel 0
[LoRaMac] SendFrameOnChannel: uplink frequency 923200000
[RegionAS923] RegionAS923RxConfig: RX config channel 0, frequency 923200000, datarate 2
[RegionAS923] RegionAS923RxConfig: RX config channel 0, frequency 923200000, datarate 2
[modlora] McpsConfirm: Giving semaphore 4

I could not find any prints showing LinkAdrReq. Could it be a problem with the configurations on my TTN account?

Can you please check if my micropython source code is fine?

lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.AS923, adr=True  )
dev_addr = struct.unpack(">l", ubinascii.unhexlify(""))[0];
nwk_swkey = ubinascii.unhexlify("");
app_swkey = ubinascii.unhexlify("");
lora.join(activation=LoRa.ABP, auth=(dev_addr, nwk_swkey, app_swkey), dr=2)
s = socket.socket(socket.AF_LORA, socket.SOCK_RAW);   
s.setsockopt(socket.SOL_LORA, socket.SO_DR, 6 )
#s.setsockopt(socket.SOL_LORA, socket.SO_CONFIRMED, True);
while( True ):
    try:
        # make the socket blocking
        # (waits for the data to be sent and for the 2 receive windows to expire)
        s.setblocking(True)

        # send some data
        s.send(bytes([0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 ]))

        # make the socket non-blocking
        # (because if there's no data received it will block forever...)
        s.setblocking(False)

        # get any data received (if any...)
        data = s.recv(512)
        pycom.rgbled(0x00FF00)
        time.sleep(1)
        pycom.rgbled(0x000000)  # Green
        time.sleep(29)
    except Exception as e:
        print( e )

Thanks.

Wong

[Gijs]

The spreading factor in lora.init() is only meant for the RAW lora connection, as the LoRaMAC uses the Datarate parameter in the socket to determine the spreading factor and bandwidth combination

[bcwong71]

@Gijs
How about if I set the spreading factor directly at lora.init() ? That should be an absolute hold right?

[Gijs]

If you set the spreading factor in lora.join(), it only holds for the join requests. The datarate set in the s.setsockopt(socket.SOL_LORA, socket.SO_DR, 2 ) should hold though, if adr is disabled, but here DR2 = SF10BW125, you should be able to see that in the debug prints on the firmware I attached above.

Edit: It worked for me this time!

datarate out 2, txpowout 0
[modlora] TASK_LoRa: taking semaphore 1 
datarate out 2, txpowout 0
[LoRaMac] PrepareFrame: adrNext[LoRaMac] ScheduleTx: next channel 6, RX channel DR: 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[LoRaMac] SendFrameOnChannel: channel 6
[LoRaMac] SendFrameOnChannel: uplink frequency 923000000
[RegionAS923] RegionAS923RxConfig: RX config channel 6, frequency 923000000, datarate 2
[RegionAS923] RegionAS923RxConfig: RX config channel 6, frequency 923200000, datarate 2
[RegionAS923] LinkAdrReq: status variables drOut: 5, txPowOut:0, nbRepOut 1, nbBytesParsed:5 <-- see this line!
[modlora] McpsConfirm: Giving semaphore 4
b''
datarate out 5, txpowout 0
[modlora] TASK_LoRa: taking semaphore 1 
datarate out 5, txpowout 0
[LoRaMac] PrepareFrame: adrNext[LoRaMac] ScheduleTx: next channel 5, RX channel DR: 5
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 5
[RegionAS923] RegionAS923ComputeRxWindowParameters: datarate 2
[LoRaMac] SendFrameOnChannel: channel 5
[LoRaMac] SendFrameOnChannel: uplink frequency 922800000
[RegionAS923] RegionAS923RxConfig: RX config channel 5, frequency 922800000, datarate 5
[RegionAS923] RegionAS923RxConfig: RX config channel 5, frequency 923200000, datarate 2
[modlora] McpsConfirm: Giving semaphore 4
b''

[bcwong71]

@Gijs
Hi Gijs. Thank you so much. I will try it now. By the way, do you have advice on why the the spreading factor was constant at 10 even when I have manually set the spreading factor to another value and have the adr turned off when on AS923?

Thanks.

Wong

[Gijs]

No worries! I have compiled the firmware for you here. It provides additional debug statements about the whole TX/RX and ADR process. You can upload the firmware in the pycom firmware updater tool, by selecting a file

Let me know what you get!
Note that I still was not able to get it working on A923 in TTN, but the platform gives me limited feedback about the ADR settings, on 868, it did work..

[bcwong71]

@Gijs
Many thanks in advance for the effort you are about to take. Great to know that we are looking at the same picture over this matter. Yes, the ADR code for both EU868 and AS923 should be exactly similar apart from the regional data rate limit and the dwell times. Shall wait for your update.

Words cannot express how much I appreciate your help. Thanks.

[Gijs]

Thanks for the great explanation. On Monday, I'll look at this again and might test it with a Chirpstack environment as well. I was not aware of the dwelltime workings.
I took my data from table 68 and 69 there from the same paper and the sourcecode, but I might have missed the dwelltime factor.

RegionAS923AdrNext function which constantly returns datarate = 2

This is what I got as well, though I never received any updates from TTN. On 868, this function did return different parameters when we received the LinkAdrReq.

What I can also do for you, is straight up copy the ADR code from EU868 over to AS923 (as it should work the same right?) and change the limits to reflect the correct limits in AS923. It could be possible we're both missing a typo. Next to that, I can compile in additional debug statements so you can test this along with me, lets get to the bottom of this

Gijs

[bcwong71]

@Gijs

Hello Gijs,

Thank you so much again. I can confirm on my end that FiPy did not respond to the adr request on AS923 despite the fact that the LinkADRReq was actually sent out by the Chirpstack server. I am really not too sure about the ADR behavior on TTN for AS923 but I did attempt AS923 on TTN and the data rate remained constant at dr=2. However, when I was back in London, I tried AS868 on TTN and the ADR worked albeit not perfectly. From my understanding, TTN always has an issue with ADR, probably because of the limited number of downlinks it is able to send per day due to fair access policy. I remember the ADR on EU868 on TTN server was not working as efficiently as the ADR on EU868 on the Chirpstack Server. Probably because Chirpstack is a development platform and therefore the number of downlinks are not limited. Have you got any opportunity to try the Chirpstack server?

However, on the Chirpstack Server, I can acknowledge that the server transmitted the LinkADRReq correctly. The data packet is as follows: 03 52 07 00 01 05 00 d2 ad 84 08 01, where 03 says change the ADR and 05 says the data rate. This downlink packet has been transmitted continuously by the gateway but somehow FiPy did not respond to the request and just kept on transmitting at DR=2.

Also, I am not too sure if the minimum data rate is AS923 is DR=0 since if dwelltime exists, the minimum DR is 2. See 2.10.6 at: https://lora-alliance.org/wp-content/uploads/2020/11/RP_2-1.0.2.pdf. And also, from my testing, I found that FiPy is only able to transmit at dr=2 even when I set the spreading factor value in: lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.AS923, sf=7, adr=False ) to 7 (dr=5) and the adr bit to False. Also, I found that the function lora.join(activation=LoRa.ABP, auth=(dev_addr, nwk_swkey, app_swkey), dr=2, timeout = 0 only accepts 2 as the value for dr with or without adr. Has there been any hard code or bug during the initialisation in the source code that has caused the dr to stay fixed at 2 on AS923?

Another culprit could be the RegionAS923AdrNext function which constantly returns datarate = 2 or maybe adrAckReq=False. I am not sure about this but I guess the main culprit is still why the lora.init and lora.join function.

I hope you have not been frustrated by my overwhelming interest and concern in this topic. I just want this matter to be resolved so that FiPy is able to function properly in Singapore and other Asian countries.

Thank you so much.

Wong

[Gijs]

I can imagine!
What i could find, is for AS923, the mininum datarate is (should be set to) DR0, and the max is DR6, while we're (both) stuck on DR2.
The code is here for AS923, where it decides the next ADR: https://github.com/pycom/pycom-micropython-sigfox/blob/d574024b715990fa3323073f9eb1fe327987be81/lib/lora/mac/region/RegionAS923.c#L500 (and line 682 for AdrReq)
and here for EU868: https://github.com/pycom/pycom-micropython-sigfox/blob/d574024b715990fa3323073f9eb1fe327987be81/lib/lora/mac/region/RegionEU868.c#L482 (and line 653 for AdrReq)
I did some side-by-side comparisons to check it out any differences that could indicate the different behaviour.
I only noticed that on AS923, I never received the LinkAdrReq from TTN, while it finally did work for EU868.

From what I can see in the rest of the source is that the differentiation happens in Regions.c, meaning that before that, all LoRa code is uniform between regions.

[bcwong71]

Hi Gijs,

Thanks for your reply. Ideally, ADR will allow FiPy to settle to an optimal data rate. Normally, it should be a fixed data rate after some oscillations at the beginning if the radio environment remains essentially the same. Can you pinpoint to me the source code that determines the maximum settings for the ADR code?

From my understanding, on the EU868 Channel, the min data rate is 0 and the max data rate is 5. Whereas, for the AS923 channel, the min data rate is 2 and the max data rate is 5. I did not have the chance to try AS923 on the TTN server but when I worked on EU868 previously, ADR was working on TTN. On the Chirpstack server, I saw the LinkAdrReq for both EU868 and AS923. However, FiPy only responded during EU868 by increasing the data rate from the initial join data rate=0 to data rate = 4 or 5 since the distance between the FiPy and gateway is relatively near. FiPy did not respond to the LinkAdrReq during AS923 and the data rate stayed fixed at the initial data rate = 2.

This is so frustrating because I would like to have ADR so very much on AS923.

Thanks.

Wong

[Gijs]

Hi,
Ive spent a lot of time trying to find the cause of your issue, but my experience with adaptive datarate is quite limited. Im not really sure what to expect (other than a varying datarate (SF/BW) and TX power). and cannot seem to reproduce the varying datarate in the first place on EU868, nor on AS923. From the sourcecode point of view, There really is no difference between the adaptive datarate code between the regions, other than the different maximum settings, that I could find. Could it be related to the TTN server settings perhaps? From what I read, the adaptive datarate is controlled from there: https://www.thethingsnetwork.org/docs/lorawan/adaptive-data-rate.html, or perhaps you could shed some more light onto the subject, so I can reproduce it on my side!
Let me know,
Gijs

Edit: I just saw one of my LinkAdrReq status updates fly by on EU868, after letting it run for a while. Meaning that TTN requested to update the datarate. I have not seen such a thing yet on AS923..

[RegionEU868] LinkAdrReq: status variables drOut: 5, txPowOut:1, nbRepOut 1, nbBytesParsed:5