How to reset a serial/uart connection?

[nathanh]

Hi All,

I am having an issue communicating with a ublox GPS module with my FiPy which I seem to be able to fix if I reset the uart port after connecting to the device by issuing a machine.reset(). I was hoping there was another way to "reset" the uart connection as uart.deinit() and then recreating does not have the desired effect.

For background, I am wanting to communicate with the ublox GPS device using its own binary (ubx) protocol. The first command I issue on power on to the GPS is to set the uart port speed and enable binary protocol communication. After doing this I have found that when setting the buardrate on the GPS module I get a weird out of sync issue when reading serial data. Essentially, all the responses I get back seem to have some 'leftover' data from the previous call appended to it, the start of the response I am expecting is shifted right and is incomplete. Issuing a second call to force another response will send the rest of the response I am waiting for.

To help illustrate the below is what I would see if I were to issue 3 commands for the device to respond to (note this not real data, just for illustrative purposes):

command1: 0x11 0x11 0x11
response1 (expecting 0x22 0x33 0x44 0x55): 0xAA 0xAA 0x22 0x33

command2: 0x11 0x11 0x11
response2  (expecting 0x66 0x77 0x88 0x99): 0x44 0x55 0x66 0x77

command3: 0x11 0x11 0x11
response3 (expecting 0x22 0x33 0x44 0x55): 0x88 0x99 0x22 0x33

Now at first I thought maybe I was not reading all the data but have set loops to wait for over 5 seconds to make sure there is not data, and uart.any() / uart.read() always returns nothing available to read. Also, regardless of the command I send and the response I expect, the start of the correct data is always at the same index. That index does however change if I power cycle the GPS module and my FiPy.

After a couple of days of playing around and trying to work this issue out (including posting on the ublox forums), I found that after I initialise my uart connection

gps = UART(1, baudrate=9600, bits=8, parity=None, stop=1, timeout_chars=10, pins=('P3', 'P4'))

If I then do a machine reset (noting the GPS does not/not lose power during this) and reinitialise the uart connection

machine.reset()
gps = UART(1, baudrate=115200, bits=8, parity=None, stop=1, timeout_chars=10, pins=('P3', 'P4'))

Communication with the ublox GPS module is perfect and as expected. I have tested this out for hours and every time I do a machine.reset() everything works fine. However, I find it odd that I would need to reset the pycom device just to get the serial connection to "re-sync".

However, if I try the following, the port does not seem to "reset" and I still get the corrupted/out of sync responses:

gps = UART(1, baudrate=9600, bits=8, parity=None, stop=1, timeout_chars=10, pins=('P3', 'P4'))
gps.write(b'\x11\x11\x11') # set config/baudrate
gps.deinit()
gps = None
gps = UART(1, baudrate=115200, bits=8, parity=None, stop=1, timeout_chars=10, pins=('P3', 'P4')) 

I did find though, if I recreate the UART connection on another bus, I fix all the issues and get communication working as expected:

gps = UART(1, baudrate=9600, bits=8, parity=None, stop=1, timeout_chars=10, pins=('P3', 'P4'))
gps.write(b'\x11\x11\x11') # set config/baudrate
gps.deinit()
gps = UART(2, baudrate=115200, bits=8, parity=None, stop=1, timeout_chars=10, pins=('P3', 'P4'))

And this would be an adequate solution except for the fact that the Uart Bus 2 on the FiPy is used for celluar communication and seeing as uart.deinit() does not seem to completely close down the connection I would be hesitant to rely on using Bus 2 then switching to Bus 1.

So back the my question, is there some way to completely reset a Uart/serial connection without having to perform a machine.reset() or am I doing something wrong?

[daniel]

Hello guys,

The issue fixed in the IDF is definitely affecting us. We will merge that patch and will be available for our next release this week.

Cheers,
Daniel

[nathanh]

@incognico said in How to reset a serial/uart connection?:

@nathanh Yeah I reproduced this with the GPS and then with a controlled test setup sending configurable amounts of data to the UART on command. It definitely appears to be a bug with the UART RX ring buffer. Raised the bug report here. Hopefully we get some response!

Ahh excellent, well not with the bug but at least it was not me doing something stupid.

For the time being I have switched to using i2c to communicate with the GPS and it is working perfectly.

Thanks again for confirming that

[incognico]

@nathanh Yeah I reproduced this with the GPS and then with a controlled test setup sending configurable amounts of data to the UART on command. It definitely appears to be a bug with the UART RX ring buffer. Raised the bug report here. Hopefully we get some response!

[nathanh]

So, just further on this, I split the TX line out of the ublox GPS to go both into my FiPy and a separate USB-Serial board to tap and listen to the data coming from the GPS into my FiPy. I can confirm that the binary data coming from the ublox device is as expected. So the problem does indeed seem to be with how the FiPy is interpreting the data and presenting it back.

Additionally, something I did not mention is that even when receiving "bad" data, any command I send to the GPS unit is sent correctly and the GPS responds appropriately, so the issue is solely with the pycom uart receive end from what I can see.

[nathanh]

@incognico said in How to reset a serial/uart connection?:

I coincidentally have a ublox SAM-M8Q GNSS module

I have the exact same one haha!

So my first command is to set baudrate to 9600 (default) and turn of all messages except ublox (I normally set the speed to 115200 but was testing with 9600 to see if I kept the rate the same would it stop the issue)

cmd: b'\xB5\x62\x06\x00\x14\x00\x01\x00\x00\x00\xC0\x08\x00\x00\x80\x25\x00\x00\x03\x00\x01\x00\x00\x00\x00\x00\x8c\x89'

I should then get an ACK response in the form of the following, however if you are changing the baudrate it would be understandable to miss it if the rates are not synced at the right time but sticking with defaults of 9600 it should be fine.

resp: b'\xb5\x62x\x05\x01\x02\x00\x06\x07\x<checksum1>\x<checksum2> 
(0xb5 0x62 0x05 0x01 0x02 0x00 0x06 0x00 <checksum1>  <checksum2>) <-- formatted for easier viewing

I then typically issue the following poll command to the config to make sure the changes went through

cmd: b'\xb5\x62\x06\x00\x01\x00\x01\x08\x22'
resp: b'\xb5\x62\x06\x00....data.....<checksum1><checksum2>\xb5\x62\x05\x01\x02\x00\x06\x00\x<checksum1>\x<checksum2>

So the response should be the current config, plus an ack response on the end.

Another command I try is to get the actual gps data (lat, lon, date, time, velocity etc

cmd: b'\xb5\x62\x01\x07\x00\x00\x08\x19'
resp: b'\xb5\x62\x01\x07\x5c\x00...payload....<checksum1><checksum2>'

I guess the easy test it after the first config command, all responses from the ublox should always start with 0xb5 0x62 and end with 2 one-byte checksums

Thanks for helping confirm the issue or my inability to code :)

[incognico]

A FIFO buffer in between the two devices would likely work but is certainly a huge overkill. If this does turn out to be an issue with the Pycom UART buffer then it should be fixable with a firmware patch.

I coincidentally have a ublox SAM-M8Q GNSS module on my desk at the moment. If you feel like posting up the actual commands/responses I'm happy to try and reproduce the symptoms. If we can demonstrate a simple test-case that causes the issue it should be easy enough to get fixed.

[nathanh]

@incognico said in How to reset a serial/uart connection?:

Is it possible the default NMEA output is filling up the FIFO buffer and triggering this bug

I expect so given the Ublox sends a wall of text on a power on start. The main reason my first command on my GPS devices is to turn off NMEA strings and increase the baudrate is to, in fact, limit/handle the amount of traffic coming over the wire. I already experienced other buffer issues with our GPS when the NMEA strings flooded the buffer at 9600 baud.

I am still rather new to microcontrollers so would I be correct in saying the ESP32 UART is a software UART and potentially getting a hardware UART/serial connection might fix these kind of issues?

[incognico]

I see a number of reports of the ESP32 RX UART fifo buffer getting "out of sync" after a buffer overflow, causing symptoms exactly like this (eg correct number of chars available but previously received chars in buffer, hardware reset required to fix). I do not know if the same issue is present with the Pycom devices/codebase, but it certainly seems very coincidental.

There was a ESP32/IDF fix published recently here - I don't know what that means or how/if it relates to the Pycom firmware.

Is it possible the default NMEA output is filling up the FIFO buffer and triggering this bug (if it even exists on our platform) before you switch the NMEA stream off? Then the rx buffer could be corrupt for all subsequent UBX comms until it is reset(). Just a thought.