LoPy communication with RFM95 modules

Clemens

We have here some (Arduino-)nodes with RFM95 modules operated by the RadioHead lib http://www.airspayce.com/mikem/arduino/RadioHead/ Is it and how is it possible to use the LoPy as a gateway for this nodes. The LoPy is only a local gateway and should receive packages from the Arduino/RFM95 nodes. After receiving the LoPy should "forward" the payload via HTTP or MQTT over WiFi to a server. So I'm talking about a local gateway only not a WAN gateway.

constantinos

It is possible. I have done it.
Here are the steps:

on Arduino:
Edit library RH_RF95.cpp
In function RH_RF95::init() change the following code at the end of the function (maybe it is ok in your case)
setModemConfig(Bw125Cr45Sf128);
setPreambleLength(8);
setFrequency(868.0);

on pyCom:
initialize Code:
LoRa(mode=LoRa.LORA, frequency=868000000, bandwidth=LoRa.BW_125KHZ , sf=7, preamble=8, coding_rate=LoRa.CODING_4_5 , rx_iq=True, power_mode=LoRa.ALWAYS_ON )

Keep in mind that in pyCom you need to struct.unpack the byte array of your message

Clemens

Hi Constantinos, many thanks for your reply and the "proofed" confirmation from you that this is possible and working. I had some initial problems getting LoRa working on the LoPy (see https://forum.pycom.io/topic/770/lora-receiving-only-b/5) but now it's running and I'm starting to take the next step RRM95 and LoPy.

I had a look at the RadioHead lib right now and setModemConfig as well as setPreambleLength have the same values default in the RFM95 lib as you write in your post. But really strange is that:

setFrequency(434.0);

It's the file \libraries\RadioHead\RF95.cpp so my impression would be it's for the RFM95 chip and this chip is for 868 MHz only. Because of this I would expect a default value of 868 - and for the RFM96 - the 433 MHz version - the default value above.

Anyway, there s a variable you can change in your sketch also so there seems to be no need for changing it in the library's *.cpp file. You can do it in the *.ino file directly:
https://learn.adafruit.com/adafruit-rfm69hcw-and-rfm96-rfm95-rfm98-lora-packet-padio-breakouts/rfm9x-test#frequency

// Change to 434.0 or other frequency, must match RX's freq!
#define RF95_FREQ 868.0

Hmm struct.unpack ok, I had not planed to send structs via RMF95 because different RFM95 nodes will have different payloads in the length (number and type of variables) but perhaps struct means her e.g. nodeid,payload and not nodeid,var1,var2,var3 ... I dealed with structs in payloads with the RFM69 lowpowerlib https://github.com/LowPowerLab/RFM69 only till now.

constantinos

@Clemens Thanks for your reply

The reason I changed the FREQ value inside the library is to make it default for my region. You are absolutely right saying that it can also been changed in the sketch.

The "struck.unpack issue" is not an issue actually. In my case I use the first 2 bytes as indicator of message type. Thank means I can have 65536 different messages type. Then with a large case statement I do the **struct.unpack" to the appropriate structure.

Another option (option 2) is to use json strings from node to pyCom. But all the messages must be like this {msgtype:1,bla:2,blabla:3}. In that case you always ask for msgtype to do content base message routing. The catch is that your messages have more bytes (field descriptors). The good thing is that is easy for us the humans.

The third option is not to process that node data and pass them as raw to your web service layer. In that case you have more processing power to decode the message using (.NET, php, nodejs). Especially in .NET you can async parse the message, in order to respond quickly to http(s) request of your pyCom. This option is good because you don't need to updates to pyCom device. I managed to overcome this with automatic *.py downloading solution at my platform.

Clemens

@constantinos Thanks for your comments. Atm I have a RFM69 setup where I sent a header the first time the node is installed in the network (it's in the Arduino code in the setup so to be more precise it is sent every time the node is reset). That's a comma separated payload like
Date/Time,Weight,Out Temp,Out Humid,B1 Temp,B2 Temp,B3 Temp,Voltage

I had to shortening this line on the RFM69 side because pf payload limitations, but this should now went better with the RFM95!

After this initial and first payload I sent the data in CSV also
2016/05/16 04:33:08, 1.946, 7.3, 73.7, 7.2, 7.1, 7.1, 4.23

Matching--data to variable-- was done on the server side. It would be possible to send json objects also but due to the reduced payload size I did it not in the past.

constantinos

@Clemens I appreciate your comments.
You have excellent analysis on your project.

The only suggestion I have based on my experience is: avoid floating point data. I always use nominal values!

Clemens

@constantinos In our first steps on radio transmission we thought about a still human readable but also compressed format that not relays on structs. So we used a Bencode based protocol BERadio https://hiveeyes.org/docs/beradio/beradio.html . it relinquishs of floats in the payload. It's really flexible but with a growing payload size and the possibility to send a dataset in more than one packet it's getting more and more obsolete.

Clemens

@constantinos It's hard to get the combination of LoRa+RFM95 running. I tried an Adafruit test sketch for the Feather M0 https://learn.adafruit.com/adafruit-feather-m0-radio-with-lora-radio-module/using-the-rfm-9x-radio#transmitter-example-code and a basic LoPy code as gateway. I got sometimes a packages received, perhaps 10% but also only one packages of 100. Do you have a simple example sketch pair that is working on your side?

constantinos

@Clemens It was not easy and there is a drawback when combining two different tranceivers.

To be honest I tried sending messages with values like 1,2,3,4,5,6,7 in the fields of the structure. Then I located them in the byte array the LoPy represented to me.
So let's see what I managed to do:

Arduino + RFM95 library from https://learn.adafruit.com/adafruit-rfm69hcw-and-rfm96-rfm95-rfm98-lora-packet-padio-breakouts/rfm9x-test
Arduino Code

#include <SPI.h>
#include <RH_RF95.h>

#define FREQUENCY  868.0

struct tpayload {
   uint32_t deviceid;   //4
   byte seq;            //1
   byte indicator1;     //1
   byte indicator2;     //1
   byte stype;          //1
   long svalue;         //4
   byte check;          //1
};

tpayload payload;
RH_RF95 rf95(10,2);    // Chip Select=D10, INT Pin = D2

void setup() {
  Serial.begin(57600);
  if (!rf95.init()) {
     Serial.println("init failed");
  }
  else { 
    Serial.print("init OK - "); Serial.print(FREQUENCY); Serial.println("mhz"); 
  }

  payload.seq = 0;
  payload.control = 0;
  payload.deviceid = 8512;
  payload.indicator1 = 1;
  payload.indicator1 = 2;
  payload.stype = 2;
  payload.svalue = -10;
}

void loop() {
  payload.seq++;
  if (payload.seq == 0) payload.seq++;  // to avoid division by zero at the next line
  payload.check = payload.deviceid % payload.seq;  // you need this to discard corrupted messages
  uint16_t now = micros();
  rf95.send( (const void*)&payload, sizeof(payload));
  delay(5000);
}

LoPy Code

import struct
import socket
from network import LoRa
import time
import gc

LORA_MSG_FORMAT = "LLBBBBlB" 


class incoming_message:
    def __init__(self, data):
       self.control, self.deviceID, self.Seq, self.indicator1,  self.indicator2,  self.stype,  self.svalue,  self.check = struct.unpack(LORA_MSG_FORMAT, data)
#self.control is something added from RFM95. Just discard it. Your message starts from the 5th byte

class server:
    def __init__(self):
        cfg_mode = LoRa.LORA
        cfg_freq = 868000000
        cfg_bandwidth=LoRa.BW_125KHZ        # LoRa.BW_125KHZ or LoRa.BW_250KHZ
        cfg_sf = 7                          # Accepts values between 6 and 12.
                                            #  6 =   64  Chips/Symbol
                                            #  7 =  128  Chips/Symbol
                                            #  8 =  256  Chips/Symbol
                                            #  9 =  512  Chips/Symbol
                                            # 10 = 1024  Chips/Symbol
                                            # 11 = 2048  Chips/Symbol
                                            # 12 = 4096  Chips/Symbol
                                            
        cfg_preamble=8                      #configures the number of pre-amble symbols. The default value is 8
        cfg_coding_rate=LoRa.CODING_4_5     #LoRa.CODING_4_5, LoRa.CODING_4_6, LoRa.CODING_4_7 or LoRa.CODING_4_8
        cfg_power_mode=LoRa.ALWAYS_ON       #LoRa.ALWAYS_ON, LoRa.TX_ONLY or LoRa.SLEEP
        
        self.lora = LoRa(mode=cfg_mode, frequency=cfg_freq, bandwidth=cfg_bandwidth, sf=cfg_sf, preamble=cfg_preamble, coding_rate=cfg_coding_rate,  rx_iq=True, power_mode=cfg_power_mode)
        self.lora.init(mode=LoRa.LORA,
          frequency=868000000,
          tx_power=14,
          bandwidth=LoRa.BW_125KHZ,
          sf=7,
          preamble=8,
          coding_rate=LoRa.CODING_4_5,
          power_mode=LoRa.ALWAYS_ON,
          tx_iq=False,
          rx_iq=False)
        self.lora_sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW)
        self.lora_sock.setblocking(False)
    def update(self):
        raw_data = self.lora_sock.recv(61)
        
        if raw_data:
            try:
                msg = incoming_message(raw_data)
                if msg.deviceID % msg.Seq == msg.check:
                    print("msg received. devid:", msg.deviceID,  " seq:", msg.Seq)
                else:
                    print("msg check failed")    
            except:
                print("msg failed to parse")
        else:
            pass

def test_server():
    srv = server()
    while True:
        srv.update()
        
        
test_server()

Clemens

@constantinos Many thanks. Unfortunately it's not working out of the box. I got some compiling errors on the Arduino side, perhaps the Arduino1.8.1 is too new. But my old 1.6.6 Arduino IDE did not work with the Feather M0 properly. And on the LoPy the firmware version is also 1.6.6 (what a chance :-)

I will check tomorrow your configuration parameters and code with my sketches so we will see.

constantinos

@Clemens Maybe I made a mistake.

Here is the code that combiles with Arduino 1.6.8

#include <SPI.h>
#include <RH_RF95.h>

#define FREQUENCY  868.0

struct tpayload {
   uint32_t deviceid;   //4
   byte seq;            //1
   byte indicator1;     //1
   byte indicator2;     //1
   byte stype;          //1
   long svalue;         //4
   byte check;          //1
};

tpayload payload;
RH_RF95 rf95(10,2);    // Chip Select=D10, INT Pin = D2

void setup() {
  Serial.begin(57600);
  if (!rf95.init()) {
     Serial.println("init failed");
  }
  else { 
    Serial.print("init OK - "); Serial.print(FREQUENCY); Serial.println("mhz"); 
  }

  payload.seq = 0;
  payload.deviceid = 8512;
  payload.indicator1 = 1;
  payload.indicator1 = 2;
  payload.stype = 2;
  payload.svalue = -10;
}

void loop() {
  payload.seq++;
  if (payload.seq == 0) payload.seq++;  // to avoid division by zero at the next line
  payload.check = payload.deviceid % payload.seq;  // you need this to discard corrupted messages
  uint16_t now = micros();
  rf95.send( (const void*)&payload, sizeof(payload));
  delay(5000);
}

Clemens

I got a compiling error and changed
rf95.send( (const uint8_t*)&payload, sizeof(payload));

I did some other changes on the rf95.send line and received 3 packages. but unfortunately I cohld not replicate it.

constantinos

@Clemens Further update about this issue.

I forgot to say that the experiment was done with firmware version 1.2.x.
Now I updated two LoPys with the latest firmware and I'm proud to announce that we have the same results: Nothing!

I will keep searching...

constantinos

@Clemens Daniel gave us the solution again!

just add public=False to Lora Initialization

        cfg_mode = LoRa.LORA
        cfg_freq = 869000000
        cfg_bandwidth=LoRa.BW_125KHZ        # LoRa.BW_125KHZ or LoRa.BW_250KHZ
        cfg_sf = 7                          # Accepts values between 6 and 12.
                                            #  6 =   64  Chips/Symbol
                                            #  7 =  128  Chips/Symbol
                                            #  8 =  256  Chips/Symbol
                                            #  9 =  512  Chips/Symbol
                                            # 10 = 1024  Chips/Symbol
                                            # 11 = 2048  Chips/Symbol
                                            # 12 = 4096  Chips/Symbol
                                            
        cfg_preamble=8                      #configures the number of pre-amble symbols. The default value is 8
        cfg_coding_rate=LoRa.CODING_4_5     #LoRa.CODING_4_5, LoRa.CODING_4_6, LoRa.CODING_4_7 or LoRa.CODING_4_8
        cfg_power_mode=LoRa.ALWAYS_ON       #LoRa.ALWAYS_ON, LoRa.TX_ONLY or LoRa.SLEEP
        
        self.lora = LoRa(mode=cfg_mode,
          frequency=cfg_freq,
          tx_power=14,
          bandwidth=cfg_bandwidth,
          sf=cfg_sf,
          preamble=cfg_preamble,
          coding_rate=cfg_coding_rate,
          power_mode=cfg_power_mode,
          tx_iq=False,
          rx_iq=False, 
          public=False)
        
        self.lora.init(mode=cfg_mode,
          frequency=cfg_freq,
          tx_power=14,
          bandwidth=cfg_bandwidth,
          sf=cfg_sf,
          preamble=cfg_preamble,
          coding_rate=cfg_coding_rate,
          power_mode=cfg_power_mode,
          tx_iq=False,
          rx_iq=False, 
          public=False)

Clemens

@constantinos Sorry for destroying your setup! ;-) (aka never change a running system) but I have seen your investigations in the thread "New firmware release 1.6.7.b1" Many thanks!

constantinos

@Clemens Please do not apologise! My setup is not live at the moment!

Clemens

@constantinos Yes! With the additional parameter public=False it is running really smoothly. A small downside I noticed. Beside the RFM95 a second Lopy as "node" can now not connect with the LoPy Gateway in parallel to the RFM. No problem for me, because I will not use this scenario productive--just as a quick side note.

constantinos

@Clemens I think that It can be done. You may notice some additional bytes arriving from RFM95 to Lopy. These bytes are stored in msg.control field. If you want a lopy to send a message acting as an RFM95 add these "control" bytes to your message.

I'm sure about this because I was able to do a LoPy to RFM95 transmission.

constantinos

@Clemens Try this

import struct
from network import LoRa
LORA_MSG_FORMAT = "LLBBBBlB" 

class outgoing_message:
    def __init__(self, control, deviceID,  Seq,  indicator1,  indicator2, stype,  svalue):
        global LORA_MSG_FORMAT
        self.data = struct.pack(LORA_MSG_FORMAT,control,  deviceID, Seq, indicator1, indicator2, stype, svalue )
        
        
        
#sending code acting as RFM95
#lora = initialize your lora object
#lora_sock = your lora socket object
lora.power_mode(LoRa.TX_ONLY)
msg = outgoing_message( <control_bytes:use the same RFM95 sends>,  <deviceID>, <Seq>, <indicator1>,  <indicator2>,  <stype>,  <svalue>)
lora_sock.send(msg.data)

andreas

Dear @constantinos,

thanks a bunch for providing these examples about how to decode the LoRa frame using struct.unpack! We built upon that and gave a shot on a LoRa-to-HTTP gateway (gateway_lora_http_csv.py), which is forwarding data in CSV format to a HTTP endpoint, as this perfectly fits @Clemens' requirements. Maybe this can also spark your interest.

For more details, please have a look at:

Please be aware these are only drafts and haven't been run on any real Pycom iron yet. However, the simple demo programs run successfully in a »MicroPython for Unix« sandbox. As we don't know about the coverage of the installation requirements in the Pycom MicroPython environment, we would be happy to receive any feedback about this.

There are also other examples e.g. for transmitting data using MQTT/JSON or by other means. A very basic MQTT example can be found at node_mqtt_json_basic.py. The intention and motto of this little telemetry framework is:

Using micropython-uterkin, you can save a few lines of code and do simple things as well as more advanced ones using the very same telemetry library API, independent of transport protocol, acquisition channel address or serialization format.

With kind regards,
Andreas.

Explore Pybytes | Official Documentation | Report a Firmware Bug/Issue | GitHub

LoPy communication with RFM95 modules

Pycom on Twitter