Receive data on lopy through serial

[monersss]

@crumble i will update my code. Meanwhile i am getting an error at open line, when i open file on flash everything works well, thus i believe it has to be something refering to SD card and writing in binary mode that brings an error

[robert-hh]

@monersss The SD card does not care about binary or not. For the card, everything is binary. It is related to something else. And, as @crumble said and I also had given in my example: Do not open and close the file all the time. Put the with ... clause outside of the receive loop.
Once you have done that, you need a proper way to tell, when you do not expect more data. Typically protocols have a kind of signaling mechanism for that purpose. If you transfer binary data as it is, that is a little bit tricky, because any byte is allowed in the data. I think I said that in my first response, that once the basic communication is working, you need a kind of protocol, such that both sides know what to do.

[monersss]

@crumble I have updated my code:

import pycom
import time
from machine import SD
import os

uart = UART(1, baudrate=11520, timeout_chars=2000)
while True:
    data = uart.read(30000)
    
if data != None:
    with open('/flash/mydati', 'wb') as f:
        
        f.write(data)
        f.close()

I did not change anythng in 'with open..' as for now i am only testing how the tramsfer goes so i do not care about overwriting the data.
i am sending 23 KB data to lopy and i am arriving at an error:
memory allocation failed, allocating 30001 bytes.
Wheres my mistake?

[robert-hh]

@monersss First of all, there is a typo in the uart = UART(... statement. It should be 115200, not 11520.
But, if you are using a LoPy 1, the total available heap space is about 50k, which may be fragemented. It is highly unlikely, that you can allocate a 30k buffer. Try using a smaller buffer, like 2k bytes and write these to SD (or just count the number of received bytes).

[monersss]

@robert-hh sorry for the typo, i cannot save the file on SD, the card is empty after running the code:

import pycom
import time
from machine import SD
import os

sd = SD()
os.mount(sd, '/sd')

uart = UART(1, baudrate=9600, timeout_chars=2000)
while True:
    data = uart.read(2000)
    print(data)
if data != None:
    with open('/sd/mydat','wb') as f:
        f.write(data)
        f.close()

[crumble]

@monersss

There will be multiple errors:

1. You will never write your data to the sd card. You read it only in the while loop. The other part will never be reached.
2. After it booted your LoPy will have around 30-64Kb of free memory. But not in one chunk. It will be scattered in small parts, depending on what you have done before. So you have either

• to split up your image into smaller chunks
• request a fix data block in the booting process as soon as possible. Than you read the from the uart into this fix buffer
• use a *Py with 3MB memory

At best you send the image in smaller chunks, so your LoPy can handle the amount of data. Therefore you will need some sort of protocol around it. Like:
start sending image 'FileName' 23123bytes:
So your read at first a line with a string. You parse the name of the file and its length.
You create the file. Read 90times a small chunk of 255bytes and append it to this file. At last you do this for the remaining 173 bytes and close the file.

At the beginning it will be easier to send the controling line as 3 lines. something like first line "send image" followed by two lines for file name and file length. So you work around string parsing and you will need only read, compare and convert to integer.

[crumble]

@monersss said in Receive data on lopy through serial:

@robert-hh sorry for the typo, i cannot save the file on SD, the card is empty after running the code:

while True:
data = uart.read(2000)
print(data)
if data != None:
with open('/sd/mydat','wb') as f:
f.write(data)
f.close()

Yes, because all you do inside the loop is reading from the uart and printing the content. Because your loop will never end, you will not reach the part where you write it to disk.

[robert-hh]

@monersss That is most likely because the indentation is wrong. If you see data in the print statement, then it shoud work with:

import pycom
import time
from machine import SD
import os

sd = SD()
os.mount(sd, '/sd')

uart = UART(1, baudrate=9600, timeout_chars=2000)
while True:
    data = uart.read(2000)
    print(data)
    if data != None:
        with open('/sd/mydat','ab') as f:
            f.write(data)
            f.close()

Using 'ab' causes data to be appended to the file.

[monersss]

@robert-hh I have updated my code and i am sending a jpg over chunks of data to my lopy, transmission works if i do it to flash, unfortunately SD card write does not work, heres the code:

import pycom
import time
from machine import SD
import os

sd = SD()
os.mount(sd,'/sd')
os.listdir('/sd')


uart = UART(1, baudrate=9600, timeout_chars=2000)
while True:
    data = uart.read(20)
    print(data)
    if data != None:
        with open('/sd/mydatas','ab') as f:
            f.write('data')
            f.close()

I have updated my firmware and atom release hoping that it will solve the problem, but that is not the case. When i open the file in 'ab' on sd using console everything is ok. Are you aware of any bugs that can cause this?

[crumble]

@monersss said in Receive data on lopy through serial:

        f.write('data')

remove the ' around data. You write the string 'data' multiple times in your file. You shall see this, if you open the file in a text editor. Remove the ' and you will write the content of your data variable.

[monersss]

@crumble no this is not a problem, i was actually trying to write anything to sd and it does not work, even when i try to write (data) i am getting 20 bytes once and than OSError Erro 5 EIO, so i am unable to write anything to my sd running script from Atom

[robert-hh]

@monersss Try to get Atom out of the way, since it may introduce problems of its own. Use a simple a terminal emulator like putty or screen, copy your script to the target device usiong ftp and try it there.

[crumble]

@monersss

1. Have you switched it off and on againt? Or better have you checked that SD card is formatted in FAT 16 or 32 and can be written by other devices?
2. have a look on the pinout PDF. UART 1 and SD card handling seems to share the same pins. Maybe you cannot use UART 1 and SD card at the same time. Try to close the UART before writing to the SD card.

Uh, I hope 2) is not the problem. Otherwise I will run with P10 into the same trap.

[robert-hh]

@crumble If Pin conflict is the issue, one can reassign the UART Tx/Rx to other Pins, usign the uart.init() method https://docs.pycom.io/chapter/firmwareapi/pycom/machine/UART.html

[monersss]

@crumble unfortunately it was a pin problem, when connected to pins 21 and 20 as serial TXD and RXD data is stored on SD without any problem

[monersss]

@robert-hh I have managed to read and write a jpg file using lopy and store it on SD after assigning different pins for uart.
When i try to do the same with the same jpg and store it on flash the file is being corrupted, number of bytes is different and somewhere in the middle of a file binary data are totally different then those of original file, i am using this code:

from machine import UART
import pycom
import time
from machine import SD
import os

#sd = SD()
#os.mount(sd,'/sd')
#os.listdir('/sd')


uart = UART(1, baudrate=115200, pins = ('P21', 'P20'), timeout_chars=2000)
while True:
    data = uart.read(25)
    print(data)
    if data != None:
        with open('/flash/pliki/mon','ab') as f:
            f.write(data)
            f.close()

What may be causing file corruption?

[robert-hh]

@monersss Flash is way slower than the SD, so jou are simply loosing data, because there is no flow control. For SD that works, because the firmware buffers some bytes, and that grants a time window for storing. AFAIK, the receive buffer is 256 bytes. Besides that, you code is highly ineffcient, because you open and close the file for every chunk of data. You could try it this way around:

from machine import UART
import pycom
import time
from machine import SD
import os

#sd = SD()
#os.mount(sd,'/sd')
#os.listdir('/sd')


uart = UART(1, baudrate=115200, pins = ('P21', 'P20'), timeout_chars=2000)
with open('/flash/pliki/mon','ab') as f:
    while True:
        data = uart.read(25)
        print(".")
        if data is not None:
            f.write(data)
        else:
            break
f.close()

The f.close() is still there, but not needed due to the with... clause. The file system has an internal 4k buffer, unless that is full or the fiel is closed, nothing will be written. You should also not print the data, because that also consumes time. If that does not work, you'll need a handshake protocol between sender and receiver.

[livius]

@monersss
You also can decrease speed of transmision e.g 9600
but the best is what we suggest you at start
you must of course have some protocol implementing.
especially you can send info with e.g. md5 hash of portion of data back and if this is ok send more if not then repeat...

[monersss]

@livius tried with lower speed, result is the same. I have never written a protocol So i am not really sure how to do it..
Another thing is That i would like my lopy to stop reading data when the whole for example 20 kB file was received, how can i do that??

[livius]

@monersss by protocol implementing we mean simple thing

1. you send command to destination "START_JPG"
2. you send back e.g. "OK" or "UNKNOW"
3. you send e.g 200Bytes of jpg + CRC(or MD5 or what you need to verify}
4. you read that data and calculate CRC and compare and you send back "OK" or "retry" or "cancel"
   ....

all in your hand (look in google for e.g. handshake protocol)