1. Home
  2. Getting Started
  3. FiPy
  4. Tracing MQTT over CAT_M1 in Wireshark

Explore Pybytes | Official Documentation | Report a Firmware Bug/Issue | GitHub

Tracing MQTT over CAT_M1 in Wireshark

  • securigy

    Is anyone here tried that successfully? Why don't I see anything in Wireshark when I filter it with target MQTT Broker address? Or do I need to install any kind of special packet driver? (in addition to what installed by Wireshark installer)

    0 J K 2 Replies
  • J

    @securigy Did you check the Pybytes thing? I believe it's enabled by default in some firmware versions, and it wants to do its own thing spontaneously, which could explain things working for a few seconds and then stopping.

    Next thing I would try is to see if switching to Wi-Fi changes anything.

    But activating logs would most definitely be the best source of information.

    0
  • securigy

    @kjm Obviously you need to do ADC initialization only once....

    0
  • K

    @securigy So you only do the adc setup the once not after every MQTT message? I mean if you did it after every message it would be like having too many sockets open when you don't do an r.close() & the whole thing grinds to a halt.

    0 securigy 1 Reply
  • securigy

    @kjm BTW, I saw some people using this:

    lte.send_at_cmd('AT+CGDCONT=1,"IP","telstra.m2m"')
    0
  • securigy

    @kjm Here is the code I use for connecting LTE. I used to use the AT-Based stuff but eventually during my experimentation I got rid of it since somewhere on this side I read from Pycom VP that AT! commands are deprecated and should not be used.
    You can see it commented out except RESET. I actually got it from Telstra example somewhere. I think you should try band 28.

    def getLTE():

    # If already used, the lte device will have an active connection.
# If not, need to set up a new connection.
if lte.isconnected():
    return lte

 # Modem does not connect successfully without first being reset.
print("Resetting LTE modem ... ", end="")
lte.send_at_cmd('AT^RESET')
print("OK")
time.sleep(1)
#print(".\n")

# From Pycom Documentation
# https://docs.pycom.io/firmwareapi/pycom/network/lte/
#
# Fipy/GPy v1.0 ==> supports 6 bands only (3, 4, 12, 13, 20, 28)
# Fipy/GPy v1.2 with Sequans modem Firmware (41xxx) ==> Supports Full range of 17 bands (1, 2, 3, 4, 5, 8, 12, 13, 14, 17, 18, 19, 20, 25, 26, 28, 66)
# Fipy/GPy v1.2 with Sequans older modem Firmwares (39xxx)==> Supports 8 Bands (3, 4, 5, 8, 12, 13, 20, 28)
#Fipy/GPy v1.2 with Sequans old modem Firmwares < (39xxx)==> Supports 6 Bands (3, 4, 12, 13, 20, 28)

# # While the configuration of the CGDCONT register survives resets,
# # the other configurations don't. So just set them all up every time.
# print("Configuring LTE ", end='')
# lte.send_at_cmd('AT+CGDCONT=1,"IP","hologram"')  # Changed this from origninal
# print(".", end='')
# lte.send_at_cmd('AT!="RRC::addscanfreq band=4 dl-earfcn=9410"') # changed band from 28 to 4. I dont know what earfcn=9410 is;
# print(".", end='')
# lte.send_at_cmd('AT+CFUN=1')
# print(" OK")

# If correctly configured for carrier network, attach() should succeed.
if not lte.isattached():
    print("Attaching to LTE network ", end='')
    lte.attach()
    while(True):
        if lte.isattached():
            print(" OK")
            break
        print('.', end='')
        time.sleep(1)

# Once attached, connect() should succeed.
if not lte.isconnected():
    print("Connecting to LTE network ", end='')
    try:
        lte.connect()
    except Exception as ex:
        print("Connecting to LTE-M network failed: {}".format(ex.args[0]))

    while(True):
        if lte.isconnected():
            print(" OK")
            break
        print('.', end='')
        time.sleep(1)

# Once connect() succeeds, any call requiring Internet access will
# use the active LTE connection.

return lte

    Now, the disconnect code already has this check and still it throws exception on lte.disconnect().
    If I don't initiate ADC I can send thousand of messages without any problems - unless I set it very high transmission rate - I hit the MQTT Broker limit that set to 10 Kbps and 500 messages/sec (free account). This also stops transmission and hangs there indefinitely...
    BTW, I do close MQTT client before calling endLTE(). Besides, if I do not initiate ADC everything behaves normally and lte.disconnect() does not throw exception.
    ...As you can see from the code above Hologram equivalent to your command is

    lte.send_at_cmd('AT+CGDCONT=1,"IP","hologram"')

    # Clean disconnection of the LTE network is required for future
# successful connections without a complete power cycle between.
def endLTE():

    print("Disonnecting LTE ... ", end='')
    if lte.isconnected():
        lte.disconnect()
    print("OK")
    time.sleep(1)
    print("Detaching LTE ... ", end='')
    if lte.isattached():
        lte.dettach()
    print("OK")
    pycom.heartbeat(True)

    Back to ADC: I also tried just now ADC(bits=10) - same result... So now I commented out the line:

        adc_c = adc.channel(pin=analogInputPin, attn=ADC.ATTN_11DB) # •	Output Voltage: 0 - 3.3V

    and the messages seems going through (usually it stopped at 6th message and I am not over 50 and it keeps going...
    So may be somebody who reads this can enlighten me...The 'analogInputPin is P18 and I call this function like this:

    adc_channel = initECG('P18', 460)
    0
  • K

    @securigy That's bizarre. I'm not using the ADC either & have no setup code for it so my modem gets into zombi mode some other way. I'd be interested to know if you have a zombi modem when your transmission choke happens. Any chance you could do a

    try: 
    if lte.isconnected(): lte.disconnect()
except: print('zombi modem')

    to see if it is? If it's not a zombi modem, maybe you are getting an overflow like happens with requests if you don't do an r.close() after each one. I know you're using MQTT but maybe it has an r.close() equivalent you need to do each time?

    Lastly, you mentioned a while back that you're using a hologram sim. I can't figure out what the hologram equivalent of lte.send_at_cmd('AT+CGDCONT=1,"IP","telstra.internet"') should be. Do you know?

    0 securigy 2 Replies
  • securigy

    @kjm Well, I never heard about it so maybe in my scenario it gets into zombie mode. However, last night I discovered why it is happening. If I initiate the sensor connection with the following code then the transmission chokes after a number of messages (depending on transmission frequency and payload size). Without this code everything works fine.

        adc = ADC(bits=12)
    print("adc: {}".format(adc))

    # CALIBRATION
    # Currently the ESP32’s ADC is not calibrated from the factory. 
    # This means it must be calibrated each time you wish to use it. 
    # To do this you must firstly measure the internal voltage reference. 
    # The following code will connect the 1.1v reference to P22

    # Output Vref of P22
    #adc.vref_to_pin('P22')
    #print("set vref to pin 22")

    # Now that the voltage reference is externally accessible you should measure 
    # it with the most accurate voltmeter you have access to. Note down the reading in millivolts, 
    # e.g. 1120. To disconnect the 1.1v reference from P22 please reset your module. 
    # You can now calibrate the ADC by telling it the true value of the internal reference. 
    # You should then check your calibration by connecting the ADC to a known voltage source.

    # Set calibration - see note above
    #adc.vref(voltref)#1100)
    #print("set adc ref value to 460")

    # Check calibration by reading a known voltage - P16 is the battery
    # ADC.ATTN_0DB — the full range voltage: 1.2V
    # ADC.ATTN_2_5DB — the full range voltage: 1.5V
    # ADC.ATTN_6DB — the full range voltage: 2.0V
    # ADC.ATTN_11DB — the full range voltage: 3.3V

    # Analogog Input Pin on Expansion Board 3.1 should be P13, P14, P17, P18 (G4, G5, G30, G31)
    adc_c = adc.channel(pin=analogInputPin, attn=ADC.ATTN_11DB) # •	Output Voltage: 0 - 3.3V
    print("got adc channel on P18")

    The interesting part is that I do not use and neither read the data from the sensor - in fact it is not even connected...
    So something in this code causes the chaos...

    0 K 1 Reply
  • K

    @securigy Random latchups where you can no longer send & recieve over lte are down to a problem with the modem. As mentioned in https://forum.pycom.io/topic/5498/gpy-sequans-catch-22, the lte modem can get into a zombie mode where it thinks it's connected but can't run any data. It's my most common cause of OSError 118 on the GPY. Zombi mode is most easily detected by asking the modem if it's connected then sending an lte.disconnect() to it if it says it is. If it's in zombi mode it will throw an OSError. I can't find a way around it, the only cure seems to be a machine.reset()

    0 securigy 1 Reply
  • securigy

    @jcaron I do send it to CloudMQTT and can see there exact messages that I sent. It stops there exactly where it stops in FiPy - I print debugging messages on FiPy. So it does not show who's to blame: MQTT.py (doubtful), firmware (which is a long story for me to start playing with), CAT-M1 connection (already tried different settings and different carriers at no avail, both with AT commands and without). It seems that there is no simple way to isolate the problem, but to me majors suspects is Firmware and LTE chip. In general, I find developing any Pycom based projects are very difficult (poor or misleading documentation, almost non-existent tech support, and products that for sure did not go through thorough QA...
    I am very close to abandoning this combo of Pycom+MicroPython... I am just hanging there to see if I get any enlightenment in the next few days...

    I really don't know if anybody succeeded here to make FiPy (the buggy of them all) to successfully send data over time...

    0 J 1 Reply
  • J

    @securigy You could start by using your own MQTT server and using tcpdump there to see what is received or not, though I doubt this would really tell you much (but it's easy to do).

    Next, I would probably compile my own version of the firmware with LTE_LOG_BUFF set to see what happens. Don't have a FiPy or GPy so can't test that, but it should give you quite a bit of information on what is going on from what I get from a quick glance in the sources.

    Also, did you make sure Pybytes in not getting in the way? It seems prone to wanting to do all sorts of stuff spontaneously while you are trying to do something else.

    0 securigy 1 Reply
  • securigy

    @jcaron I was hoping for some kind of trick/tool that allows to see TCPIP traffic...
    The reason I asked is that I am banging my head for weeks now with a problem of continuously sending data with FiPy over MQTT. I tried all kind of code reduction/optimization, and variety of combinations of payload size and sending frequencies...with pretty much the same result: after sending a few hundred bytes (for high frequency with very small payload) or 4K-5K for big payloads every second... it stops sending and just hangs until the time out, after which I get error 118.
    I have very detailed thread about it (talking to myself):
    here

    0 J K 2 Replies
  • J

    @securigy I'm not quire sure I understand what you are trying to do? Where are you running Wireshark? On what interface?

    Obviously if you are running it on your local computer while the FiPy is talking to a server over CAT-M1, you won't see anything, and I don't think there's any way for you to see anything.

    The only place you could see anything would be on the server, or if you use a separate modem and have Wireshark "in between" the *Py module and the modem (though I have no idea how this would be done exactly).

    Or did I miss something?

    0 securigy 1 Reply
Log in to reply
 

Pycom on Twitter