Does pulses_get function have a 200 bit limitation when reading?

[Pesien]

Hello, I'm a new programmer and I'm working on a library to comunicate with SDI-12 sensors using a GPIO pin in a WiPy 3.

In order to read the pulses I get from the sensor I use the pulses_get function, but it seems to have a limitation on 200 bits even though it is not mentioned in the documentation.

After sending the bits ,the code for reading is:

pin=Pin('P11', mode=Pin.IN)
pulses=pycom.pulses_get(pin, 10000)

And the output I get:
[(1, 4146), (0, 1659), (1, 1656), (0, 835), (1, 827), (0, 830), (1, 2487), (0, 1661), (1, 828), (0, 1663), (1, 829), (0, 1658), (1, 1657), (0, 1659), (1, 1657), (0, 833), (1, 2486), (0, 831), (1, 2487), (0, 828), (1, 828), (0, 834), (1, 828), (0, 830), (1, 828), (0, 830), (1, 2487), (0, 2494), (1, 828), (0, 1661), (1, 3315), (0, 2494), (1, 829), (0, 830), (1, 4146), (0, 830), (1, 828), (0, 834), (1, 828), (0, 2490), (1, 2488), (0, 830), (1, 829), (0, 833), (1, 830), (0, 3318), (1, 1658), (0, 2492), (1, 1658), (0, 2489), (1, 1657), (0, 830), (1, 828), (0, 834), (1, 4974), (0, 831), (1, 829), (0, 1662), (1, 829), (0, 829), (1, 828), (0, 1660), (1, 1657), (0, 830), (1, 828), (0, 835), (1, 4146), (0, 829), (1, 829), (0, 831), (1, 828), (0, 834), (1, 828), (0, 1659), (1, 1657), (0, 830), (1, 829), (0, 830), (1, 828), (0, 834), (1, 828), (0, 830), (1, 828), (0, 1659), (1, 829), (0, 830), (1, 1657), (0, 834), (1, 1657), (0, 1659), (1, 828), (0, 1659), (1, 1657), (0, 834), (1, 4975), (0, 829), (1, 829), (0, 1663), (1, 828), (0, 1659), (1, 1658), (0, 1658), (1, 1658), (0,834), (1, 3316), (0, 2489), (1, 829), (0, 1663), (1, 1657), (0, 1659), (1, 829), (0, 1658), (1, 1658), (0, 834), (1, 3317), (0, 2489), (1, 828), (0, 1663), (1, 3317), (0, 2489), (1, 829), (0, 1663), (1, 830), (0, 829), (1, 1658), (0, 2489), (1, 1656), (0, 834)]

Translated it is:
013DECAGON MPS-6 386889
I know the reading is incomplete because it should end with \r\n

I tried it with multiple sensors and commands but I always get the same bit limit.
Does pulses_get have a limit? If it has, why is it not mentioned in the documentation?

[Martinnn]

@Pesien said in Does pulses_get function have a 200 bit limitation when reading?:

pycom.pulses_get

Here's the code: https://github.com/pycom/pycom-micropython-sigfox/blob/master/esp32/mods/modpycom.c

[Pesien]

@Martinnn Thank you for answering, but as I said I'm new to programming (it's for college) and can't identify the issue there :(

[robert-hh]

@Pesien pulses_get has no limitation in the number of pulses beside the RAM, which is used to store the results. You cannot even enforce a maximum number of pulses to be received. It stops receiving when a timeout happens. SO maybe after 200 pulses you have a short delay, which is detected as the timeout.
I wonder why you uses pulses_get instead of UART receive?

And yes, the documentations is wrong.

[Pesien]

@robert-hh Thank you, but I don't see what kind of delay it's happening there. The sensor keeps sending bits and the function simply stops reading at 200 bits. Since it happens with different sensors and commands, and the timeout is set at 10000 us I can't see where that delay comes from, and how is it bigger than 10 ms.

I'm using this function because my project is to comunicate with SDI-12 sensors using only one GPIO pin. My teacher has it working with the UART and two pins, but reducing it to one GPIO pin only has several advantages as you might imagine.
Also, the pulses_get function is way more stable and precise than the interrupts and timers I was trying to use (with no results).

[robert-hh]

@Pesien OK. Looking at the code, the RMT collects pulses into a ringbuffer until a timeout occurs, and then the number of pulses which have been collected in the ringbuffer are returned. I could not really tell the size of the ringbuffer, but 200 looks strange. The example in you post has 128 entries, which looks more like a plausible number.

[robert-hh]

@Pesien Digging a little bit further I found the place. It is indeed a limited number events, defined by line 59 of modpycom.c
rmt_driver_install(RMT_CHANNEL_0, 1000, 0);
1000 is the size of the ringbuffer, with 4 bytes per event. So if you want more, increase that number and rebuild the firmware.
The hint for that is here: https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/peripherals/rmt.html
Edit: Testing with a square wave generator returns 128 pulses, which is less than expected from the explanation above, but the same number you have seen.
Edit2: Increasing the number above does not change anything.

[Pesien]

@robert-hh Thank you so much for the help, really. I'll try to understand how I can fix it tomorrow when I have some time.
Edit: Even though it was not a fix, thank you for your help. Also you confirmed me it was not my code only.

[robert-hh]

@Pesien See below: It was'nt a fix.

[Pesien]

@robert-hh Any new ideas on why is this happening?

[robert-hh]

@Pesien I have to check further this evening. The 128 events match the 64 * 4 byte RAM buffer page dedicated to each RMT channel. That should be copied to the ringbuffer to avoid overrung of the RMT RAM, but somehow that does not happen. Trying to assign more than one RMT Buffer page to a channel failed.

[crumble]

@Pesien said in Does pulses_get function have a 200 bit limitation when reading?:

@robert-hh Any new ideas on why is this happening?

According to the link from @robert-hh this function is part of an API whicht deals with infrared remote controls. They send/receive usualy only short messages.

[robert-hh]

@Pesien No luck with simple hacks. Also machine.RMT offers no better choices. But in documentation there the 128 entry limit is mentioned. I do not know whether this is a limitation of the underlying RTOS or the way the pycom firmware uses it.

[Pesien]

@robert-hh Then I guess I'll have to try again with interrupts and timers...
Thank you for your help again, and if you know any better solutions for pulse reading i'd love to hear them.

[jcaron]

@Pesien According to the documentation, you can configure the UART to use only one pin if you are only receiving. Just state None as the TXD pin and you should be all set.

[robert-hh]

@jcaron Well, but@Pesien wnat to send AND receive in hald-duplex mode. But it may be possible to switch between send & receive. The only problem would be to find the right time, because the send() returns before the data actually has been transmitted.

[jcaron]

@robert-hh Ah indeed, had only seen discussion about receiving data, but I now see that the protocol implies data in both directions.

One may try to give the same pin for both RXD and TXD, maybe? Not sure if the ESP32 would support that. There would probably be "echoing" of sent data if it were the case, though.

Otherwise the timing issue should be pretty simple. 1200 bits/s, 7 bits, parity, probably one start, one stop, that means 120 characters per second.

[robert-hh]

@jcaron I made a little bit of testing. It seems that you can a single Pin between TX and RX, by re-doing the uart init or instantiation between send and receive with the option pins=(pin, None) and pins=(None, pin) for send and receive. That switch takes 600-800 µs, about 1 bit duration. There are some hiccups involved.

• The other side must not send any characters before the switch from TX to RX has happened. Otherwise, the receiver will lock up and will not detect anything for about 3 seconds (maybe 32 characters, t.d.a.).
• The Pycom device must wait with switching until all characters have been sent. Otherwise the message will not be sent.

So its a little bit of tricky timing, which works best if the sensor is a slow responder. But I read that in SDI-12 the sensor must respond within 15 ms. uart.wait_tx_done() may be appropriate for the wait.

[jcaron]

@robert-hh did you by any chance try to use the same pin for RX and TX? I would be surprised if that worked, but you never know...

[robert-hh]

@jcaron Yes. That was indeed the intention of the test - using the same pin for send and receive. To some extent it worked, with a strange glitch:
If in the process of switching the pin from send to receive characters for receive arrive too early, the receiver locks up for a while. I could not determine with my simple set-up, whether it locks up for a certain time (2-3 seconds or ~250 character times at 1200 baud) or for some number of characters, which are dropped. My gut feeling says it's the latter, and has something to do with the receive ring buffer. But I did not look into the code yet, and it is surely tricky to make a proper test set-up.