Receive data on lopy through serial

[livius]

@monersss said in Receive data on lopy through serial:

I did not find wb option in open module

Where have you looked for?
For storing binary file you must use 'wb' or 'r+b' mode.
'wb' for only writing or 'r+b' for writing and reading.

and converting to str is not good concept here

[robert-hh]

@monersss The file operations in MicroPython follow the Python standard. So Option "wb" just tells to perform not translation of special bytes, like \n. The option w+b opens a file in binary mode for both reading and writing, just using "w" opens and trunctates a file.

I'm a little bit surprised that by your changes the code now works. They should not make a difference, besides the data format.

UART should exchange binary data, and that is what you need for jpg files.

[monersss]

@robert-hh corrected and now i am storing data on flash, now I want to store it on SD, i have mounted SD card:

, but when using:
with open('/SD/mydat.txt','wb') as f:

I am getting ENODEV error, how can I store data on SD?

[crumble]

write sd in lowercase and don't forget to mount your sd card

[monersss]

@crumble heres the code:

from machine import UART
import pycom
import time
from machine import SD
import os

sd = SD()
os.mount(sd, '/sd')

uart = UART(1, baudrate=9600, timeout_chars=2000)
while True:
data = uart.readall()
print(data)
if data != None:
with open('/sd/mydat','wb') as f:
#print(data)
f.write(data)
f.close()

i am getting out OSErroR: Errno 5 EIO

[crumble]

@monersss

try to post your code with ``` ín the linbe above. I will never get it why people love languages with whitespace as block marks.

I have no *Py to test your stuff at moment. I supose you get this error because you double close the file. The with statement shall close the file automatically at the end of the inner block. So you don't need the explicit close statement.

Beside there are a few things which you better change:

• Don't open and close your file inside the loop. This will be a huge performance issue. Open and close it outside the loop. Use flush() instead, if you fear too loose data.
• Your file will contain only the data of the latest loop cycle. You open the file with 'wb' which will clear the file. Add an a, so the data will be appended at the end of the file
• don't use readall() use a fix buffer size, otherwise you will sooner or later run into an out of memory exception.

[monersss]

@crumble i will update my code. Meanwhile i am getting an error at open line, when i open file on flash everything works well, thus i believe it has to be something refering to SD card and writing in binary mode that brings an error

[robert-hh]

@monersss The SD card does not care about binary or not. For the card, everything is binary. It is related to something else. And, as @crumble said and I also had given in my example: Do not open and close the file all the time. Put the with ... clause outside of the receive loop.
Once you have done that, you need a proper way to tell, when you do not expect more data. Typically protocols have a kind of signaling mechanism for that purpose. If you transfer binary data as it is, that is a little bit tricky, because any byte is allowed in the data. I think I said that in my first response, that once the basic communication is working, you need a kind of protocol, such that both sides know what to do.

[monersss]

@crumble I have updated my code:

import pycom
import time
from machine import SD
import os

uart = UART(1, baudrate=11520, timeout_chars=2000)
while True:
    data = uart.read(30000)
    
if data != None:
    with open('/flash/mydati', 'wb') as f:
        
        f.write(data)
        f.close()

I did not change anythng in 'with open..' as for now i am only testing how the tramsfer goes so i do not care about overwriting the data.
i am sending 23 KB data to lopy and i am arriving at an error:
memory allocation failed, allocating 30001 bytes.
Wheres my mistake?

[robert-hh]

@monersss First of all, there is a typo in the uart = UART(... statement. It should be 115200, not 11520.
But, if you are using a LoPy 1, the total available heap space is about 50k, which may be fragemented. It is highly unlikely, that you can allocate a 30k buffer. Try using a smaller buffer, like 2k bytes and write these to SD (or just count the number of received bytes).

[monersss]

@robert-hh sorry for the typo, i cannot save the file on SD, the card is empty after running the code:

import pycom
import time
from machine import SD
import os

sd = SD()
os.mount(sd, '/sd')

uart = UART(1, baudrate=9600, timeout_chars=2000)
while True:
    data = uart.read(2000)
    print(data)
if data != None:
    with open('/sd/mydat','wb') as f:
        f.write(data)
        f.close()

[crumble]

@monersss

There will be multiple errors:

1. You will never write your data to the sd card. You read it only in the while loop. The other part will never be reached.
2. After it booted your LoPy will have around 30-64Kb of free memory. But not in one chunk. It will be scattered in small parts, depending on what you have done before. So you have either

• to split up your image into smaller chunks
• request a fix data block in the booting process as soon as possible. Than you read the from the uart into this fix buffer
• use a *Py with 3MB memory

At best you send the image in smaller chunks, so your LoPy can handle the amount of data. Therefore you will need some sort of protocol around it. Like:
start sending image 'FileName' 23123bytes:
So your read at first a line with a string. You parse the name of the file and its length.
You create the file. Read 90times a small chunk of 255bytes and append it to this file. At last you do this for the remaining 173 bytes and close the file.

At the beginning it will be easier to send the controling line as 3 lines. something like first line "send image" followed by two lines for file name and file length. So you work around string parsing and you will need only read, compare and convert to integer.

[crumble]

@monersss said in Receive data on lopy through serial:

@robert-hh sorry for the typo, i cannot save the file on SD, the card is empty after running the code:

while True:
data = uart.read(2000)
print(data)
if data != None:
with open('/sd/mydat','wb') as f:
f.write(data)
f.close()

Yes, because all you do inside the loop is reading from the uart and printing the content. Because your loop will never end, you will not reach the part where you write it to disk.

[robert-hh]

@monersss That is most likely because the indentation is wrong. If you see data in the print statement, then it shoud work with:

import pycom
import time
from machine import SD
import os

sd = SD()
os.mount(sd, '/sd')

uart = UART(1, baudrate=9600, timeout_chars=2000)
while True:
    data = uart.read(2000)
    print(data)
    if data != None:
        with open('/sd/mydat','ab') as f:
            f.write(data)
            f.close()

Using 'ab' causes data to be appended to the file.

[monersss]

@robert-hh I have updated my code and i am sending a jpg over chunks of data to my lopy, transmission works if i do it to flash, unfortunately SD card write does not work, heres the code:

import pycom
import time
from machine import SD
import os

sd = SD()
os.mount(sd,'/sd')
os.listdir('/sd')


uart = UART(1, baudrate=9600, timeout_chars=2000)
while True:
    data = uart.read(20)
    print(data)
    if data != None:
        with open('/sd/mydatas','ab') as f:
            f.write('data')
            f.close()

I have updated my firmware and atom release hoping that it will solve the problem, but that is not the case. When i open the file in 'ab' on sd using console everything is ok. Are you aware of any bugs that can cause this?

[crumble]

@monersss said in Receive data on lopy through serial:

        f.write('data')

remove the ' around data. You write the string 'data' multiple times in your file. You shall see this, if you open the file in a text editor. Remove the ' and you will write the content of your data variable.

[monersss]

@crumble no this is not a problem, i was actually trying to write anything to sd and it does not work, even when i try to write (data) i am getting 20 bytes once and than OSError Erro 5 EIO, so i am unable to write anything to my sd running script from Atom

[robert-hh]

@monersss Try to get Atom out of the way, since it may introduce problems of its own. Use a simple a terminal emulator like putty or screen, copy your script to the target device usiong ftp and try it there.

[crumble]

@monersss

1. Have you switched it off and on againt? Or better have you checked that SD card is formatted in FAT 16 or 32 and can be written by other devices?
2. have a look on the pinout PDF. UART 1 and SD card handling seems to share the same pins. Maybe you cannot use UART 1 and SD card at the same time. Try to close the UART before writing to the SD card.

Uh, I hope 2) is not the problem. Otherwise I will run with P10 into the same trap.

[robert-hh]

@crumble If Pin conflict is the issue, one can reassign the UART Tx/Rx to other Pins, usign the uart.init() method https://docs.pycom.io/chapter/firmwareapi/pycom/machine/UART.html