uart.readline() not working reliably for long strings?

[SciWax]

If I try the other method from the user vruizvil, I run into the problem, that the function wants to decode chars of the NoneType class, which doesn't work:

Traceback (most recent call last):
  File "<stdin>", line 44, in <module>
  File "<stdin>", line 29, in serialrecv
AttributeError: 'NoneType' object has no attribute 'decode'

I've changed

length_str += char.decode()

to

length_str += char.decode("utf-8")

def serialrecv(uartport):
  #read the length of the data, letter by letter until we reach EOL
  length_str = ''
  char = uartport.read(1)
  while char.decode() != '\r':
    length_str += char.decode("utf-8")
    char = uartport.read(1) 
  return length_str

Can I put an identifier somewhere, so that the function doesn't decode chars, which are belong to the NoneType class. Maybe with an if statement?

The string I'm interested in starts with {. Can I use that somehow to start from there decoding my chars?

[robert-hh]

@SciWax UART read is not blocking. It returns immediately, and if there is nothing to read, it returns None. So you have to test for the retirn value, or call read only if uart.any returns, zhst sufficiently many characters are present.

[SciWax]

@robert-hh

thanks for your suggestions. I have a hard time at the moment figuring out how to make the testing work, when I have to test for single bytes. For the code in my first post, it's pretty easy.

Basically at the end, the GPy is only there for listening to the Arduino and sending the data. I kinda have to tell the GPy, if uart.read(1) is None, jump back to the beginning of the function till uart.read(1) finally reads something, which isn't None.

I've tried to realize it like that, but no success:

def serialrecv(uartport):
  #read the length of the data, letter by letter until we reach EOL
  length_str = ''
  char = uart.read(1)
  if char is not None:
    while char.decode() != '\r':
      length_str += char.decode("utf-8")
      char = uartport.read(1)
  else:
    return print("Waiting for datastring") 
  return length_str

The output for this is this:

None
Waiting for datastring
None
Waiting for datastring
None
Waiting for datastring
None

[SciWax]

I think I'm making some progress:

def serialrecv(uartport):
  #read the length of the data, letter by letter until we reach EOL
  length_str = ""
  char = uart1.read(1)
  if char is not None:
    while char.decode() != "\r":
      print("inside the while loop")
      length_str += char.decode("utf-8")
      char = uart1.read(1)
  else:
    return print("Waiting for datastring") 
  return length_str

while True:
  dataRec = serialrecv(uart1)
  print(dataRec)

The output:

None
Waiting for datastring
None
Waiting for datastring
None
inside the while loop
inside the while loop
inside the while loop
.....
inside the while loop
inside the while loop
inside the while loop
{"Id":131,"Y":2148,"M":12,"D":31,"Hr":14,"Mi":59,"B1":4.2,"B2":4.3,"T1":13.33,"H1":13.33,"T2":27.36,"H2":44.45,"O1":100.00,"O1g":200.25,"O1t":27.00,"O1h":56.00,"N1":111.00,"N1g":222.25,"N1t":27.00,"N1h":56.00,"C1":10000.00,"C1g":20.25,"C1t":27.00,"C1h":56.00,"O2":24300,"N2":26000,"C2":21000,"C3":22000,"P1":100.11,"P2":130.11,"P4":160.11,"P10":200.11}
inside the while loop
Traceback (most recent call last):
  File "<stdin>", line 49, in <module>
  File "<stdin>", line 32, in serialrecv
AttributeError: 'NoneType' object has no attribute 'decode'

I don't really get the error. Something should only be decoded, if a char is not None. How does a char with None gets into my while loop?

[SciWax]

I think I've got it now:

def serialrecv(uartport):
  #read the data, byte by byte and decode it
  length_str = ""
  char = uart1.read(1)
  if char is not None:
    while char is not None:
      print("inside the while loop")
      length_str += char.decode("utf-8")
      char = uart1.read(1)
  else:
    return print("Waiting for datastring") 
  return length_str

Didn't throw any errors so far. I will test my new code and will give feedback about my experiences with the change.

[robert-hh]

@SciWax the UART object can also be created with a timeout_chars=x parameter. In that case it will wait at least x character times for a value to arrive. The default is 0, which means: do not wait.
see: https://docs.pycom.io/firmwareapi/pycom/machine/uart/#app

[SciWax]

@robert-hh

Thank you for that information. Do you would recommend to me using that? I have alot of characters so in the worst case, it could take ages if anything goes wrong. For instance reading byte by byte, with 200 bytes to read and a timeout of 1 second, in the worst case maybe I would have to wait 200 seconds. Please correct me, if I'm wrong.

[robert-hh]

@SciWax If you set the timeout to 1 second and call read(1), if the uart buffer has at least one character, then it will return immediately and not wait for a timeout.

[SciWax]

@robert-hh

I added your suggestion to my code. Thanks!

Do you think this would help with uart.readline()? Last night, I've tested my old script with uart.readline() for 4 hours reading a string sent by my arduino every 10 seconds via a serial connection between my arduino and GPy. 10 times in 4 hours, uart.readline() didn't read the full json-formatted string from the arduino completely.

[robert-hh]

@robert-hh with uart.readline() you have to be sure, that the receive buffer is sufficiently large. You can increase that when creating the UART object with the parameter rx_buffer_size=nnn. Since the sample JSON object of your post is 352 bytes, you could try to use a buffer size of 1024 or even larger, which will cope with delays during forwarding the content.
Edit: The default size is 512.