Receive data on lopy through serial

[livius]

@monersss
The last question is reall surprising ;-)
If you know how to recive data throught UART(i see that you succesfully retrive it) then how can you assume that e.g. jpg will be stored somwhere automatically?
Especially that you retrive "data" from UART and reciving side do not know what data mean.
You must "tell" what to do with data and especially what data really is :)

[monersss]

@livius @robert-hh i thought this is getting started forum so suprising questions are allowed.. I was searching the forum to find out how to store received data on lopy but did not find the answer.

[livius]

@monersss
Yes, are allowed :)
look at documentation about uart
https://docs.pycom.io/chapter/firmwareapi/pycom/machine/UART.html

especially you can be interested in:

uart.readall()
uart.readline()
uart.read([nbytes])

then look at python docs about working with files:
https://docs.python.org/3/tutorial/inputoutput.html

for jpg data you should use r+b mode in open method of the file

e.g. (i wrote this sample without any testing if it is working)

uart = UART(1, baudrate=9600)
with open('/flash/myfile.dat', 'r+b') as f:
	for i in range(0, 1000): #while True:
		d = uart.readall()
		if d:
			f.write(d)

you must of course have some protocol implementing to know when to start writing to the file and when to finish it.

[robert-hh]

@monersss Micropython supports the usual file operations. So you can open a file and write the received data to that file. Like:

with open("mydata", "wb") as f:
    # receive data from uart
    f.write(data)

[robert-hh]

@monersss You might also check if you can use ftp for your purpose, beause the pycom devices have a built-in ftp server, and Matlab may be able to send data through ftp.

[monersss]

@robert-hh That is not possible. Lopy will be connected to our custom built device in remote location collecting different data through sensors. Thus we can connect it only through serial to the collecting system and later use WiFi to send the data through lopy to another device where they will be stored.

[monersss]

@livius Thanks for help, I will try to implement it over this weekend.

[monersss]

@livius Hello there, I have tried to write the data on several ways, but i am doing something wrong:
from machine import UART
import pycom
import time

uart = UART(1, baudrate=9600, timeout_chars=2000)
while True:
data = uart.readall()
print(data)
if data != None:
with open('/flash/'mydatittaa','wb') as f:
f.write(data)

what i am getting back is invalid syntax data in line 12.

What am I doing wrong? data is assigned to uartread, and the message that is send to lopy is a string in binary format..

[robert-hh]

@monersss That's strange, especially since there is no line 12. Besides that, I would rearrange the lines, otherwise the content of the file would always be overwritten. I have not xxPy deviec here at the moment and cannot try these few lines.

from machine import UART
import pycom
import time

uart = UART(1, baudrate=9600, timeout_chars=2000)
with open('/flash/'mydatittaa','wb') as f:
    while True:
        data = uart.readall()
        print(data)
        if data is not None:
            f.write(data)

P.S.: Iy you include text in lines conatining three backquotes (```), they will be shown as code.

[monersss]

@robert-hh problem solved, since i am sending to lopy a string in binary format i changed code a bit:

uart = UART(1, baudrate=9600, timeout_chars=2000)
while True:
data = uart.readall()
print(data)
if data != None:
with open('/flash/mydatitt' ,'w') as f:
#print(data)
f.write(str(data))
f.close()
one code line: f.write(str(data)).

What modification to the code I should do to be able to send and save jpg and txt files?
PS Just to be 100% sure - UART uses binary format for the data transfer right? I did not find wb option in open module, so i have used w, is that correct or should i use wb or w +b ?

[livius]

@monersss said in Receive data on lopy through serial:

I did not find wb option in open module

Where have you looked for?
For storing binary file you must use 'wb' or 'r+b' mode.
'wb' for only writing or 'r+b' for writing and reading.

and converting to str is not good concept here

[robert-hh]

@monersss The file operations in MicroPython follow the Python standard. So Option "wb" just tells to perform not translation of special bytes, like \n. The option w+b opens a file in binary mode for both reading and writing, just using "w" opens and trunctates a file.

I'm a little bit surprised that by your changes the code now works. They should not make a difference, besides the data format.

UART should exchange binary data, and that is what you need for jpg files.

[monersss]

@robert-hh corrected and now i am storing data on flash, now I want to store it on SD, i have mounted SD card:

, but when using:
with open('/SD/mydat.txt','wb') as f:

I am getting ENODEV error, how can I store data on SD?

[crumble]

write sd in lowercase and don't forget to mount your sd card

[monersss]

@crumble heres the code:

from machine import UART
import pycom
import time
from machine import SD
import os

sd = SD()
os.mount(sd, '/sd')

uart = UART(1, baudrate=9600, timeout_chars=2000)
while True:
data = uart.readall()
print(data)
if data != None:
with open('/sd/mydat','wb') as f:
#print(data)
f.write(data)
f.close()

i am getting out OSErroR: Errno 5 EIO

[crumble]

@monersss

try to post your code with ``` ín the linbe above. I will never get it why people love languages with whitespace as block marks.

I have no *Py to test your stuff at moment. I supose you get this error because you double close the file. The with statement shall close the file automatically at the end of the inner block. So you don't need the explicit close statement.

Beside there are a few things which you better change:

• Don't open and close your file inside the loop. This will be a huge performance issue. Open and close it outside the loop. Use flush() instead, if you fear too loose data.
• Your file will contain only the data of the latest loop cycle. You open the file with 'wb' which will clear the file. Add an a, so the data will be appended at the end of the file
• don't use readall() use a fix buffer size, otherwise you will sooner or later run into an out of memory exception.

[monersss]

@crumble i will update my code. Meanwhile i am getting an error at open line, when i open file on flash everything works well, thus i believe it has to be something refering to SD card and writing in binary mode that brings an error

[robert-hh]

@monersss The SD card does not care about binary or not. For the card, everything is binary. It is related to something else. And, as @crumble said and I also had given in my example: Do not open and close the file all the time. Put the with ... clause outside of the receive loop.
Once you have done that, you need a proper way to tell, when you do not expect more data. Typically protocols have a kind of signaling mechanism for that purpose. If you transfer binary data as it is, that is a little bit tricky, because any byte is allowed in the data. I think I said that in my first response, that once the basic communication is working, you need a kind of protocol, such that both sides know what to do.

[monersss]

@crumble I have updated my code:

import pycom
import time
from machine import SD
import os

uart = UART(1, baudrate=11520, timeout_chars=2000)
while True:
    data = uart.read(30000)
    
if data != None:
    with open('/flash/mydati', 'wb') as f:
        
        f.write(data)
        f.close()

I did not change anythng in 'with open..' as for now i am only testing how the tramsfer goes so i do not care about overwriting the data.
i am sending 23 KB data to lopy and i am arriving at an error:
memory allocation failed, allocating 30001 bytes.
Wheres my mistake?

[robert-hh]

@monersss First of all, there is a typo in the uart = UART(... statement. It should be 115200, not 11520.
But, if you are using a LoPy 1, the total available heap space is about 50k, which may be fragemented. It is highly unlikely, that you can allocate a 30k buffer. Try using a smaller buffer, like 2k bytes and write these to SD (or just count the number of received bytes).