OneWire.py problem

kjm · Oct 28, 2019, 1:05 AM

I've tried a couple of different flavours of the OneWire library with a single DS18B20 temperature sender. Both routines, the one at the bottom of the page at https://docs.pycom.io/tutorials/all/owd/, and a more resent version at https://github.com/pycom/pycom-libraries/blob/master/lib/onewire/onewire.py will chug away happily in a loop printing temperatures at 2s intervals forever.

However if I try them in a program that deepsleeps they often fail to give a temperature. If I'm running the older library, read_temp_async gives an IndexError(most likely indicating it can't find the DS18B20). While the newer version returns None (indicating a busy DS18B20). I feel this difference is indicative of what the root cause of the problem is, but I don't understand the library well enough to figure it out.

My problem is the only way to fix this seems to be to reboot the gpy. Putting the temperature read in a loop, or having multiple reads scattered through the main program do no good. Once the library stops working it stays not working till I reset the GPY.

Can anyone suggest something I might try other than a reboot? The DS18B20 is continuously powered & works fine with other devices so I don't think it is the problem. Something the OneWire library uses is presumably unavailable after random deepsleep wakeups & a reboot is the only thing currently that seems to fix it.

andreas · Nov 18, 2019, 4:29 PM

@robert-hh said in OneWire.py problem:

The [refurbished] driver is here: https://github.com/robert-hh/Onewire_DS18X20. It follows the API of the native MicroPython driver.

Taking this chance, we want to thank @robert-hh so much for taking care of all the details regarding this driver. It was a real pleasure to work together with you getting this a real home and - to top it all off - big thanks that you also made its API match the one of the native driver, so people can swap them in and out. We have been doing this through [1] already and are very happy about it.

[1] https://github.com/hiveeyes/hiveeyes-micropython-firmware/commit/2a23af47

robert-hh · Nov 18, 2019, 4:29 PM

@John-Baird Parasitic power is not easy, because it has to be supported by the driver. For using it, the data line has to switched to a string pull-up aka Vcc when the temp conversion takes place. This strong pullup has to be set within 10µs after sending the start conversion command. That is not possible at app level, but must be done at the bit level driver. I know it because I just have implemented it yesterday.
And it does not solve the problem that sometimes the sensors do not respond. A full powercycle helps, including pulling the data line to 0. If the data line stays high, then the devices stay powered, and that could be the problem here.
b.t.w.: The driver is here: https://github.com/robert-hh/Onewire_DS18X20. It follows the API of the micropython.org version.

John Baird · Nov 18, 2019, 11:22 AM

Jumping in a bit late, but FWIIW - I've been running DS18B20 sensors in many devices that regularly go to deepsleep for months now. They work very well, with few (if any) missed reads. I think the difference here is that our application is very power sensitive, so the DS180B20 is powered from a supply that turns off every time the module goes into deepsleep. Perhaps you could try parasitic power? That would also turn off the sensor during deepsleep.

andreas · Nov 17, 2019, 3:29 AM

Dear @kjm, @crumble and @robert-hh,

@andreas said in OneWire.py problem:

I was too quick on that detail. After getting into the MicroPython code, I see that porting/enabling the modonewire.c from Genuine MicroPython on Pycom MicroPython will require more efforts than I will be able to dedicate to this topic for now.

Finally, we have been able to add support for the native 1-Wire/DS18X20 driver on Pycom MicroPython and published respective firmware images called 1.20.1.r1-0.7.0-vanilla-dragonfly-onewire. See also Inofficial firmware bakery for Pycom/ESP32 devices.

Please be aware that you will have to use a set of different Python drivers to access the lowlevel API available through https://github.com/micropython/micropython/tree/master/drivers/onewire.

If you are willing to test this, we will be happy to hear about the outcome for you.

With kind regards,
Andreas.

crumble · Nov 6, 2019, 7:45 AM

@kjm

Pycom ran into this damn ESP 32 trap.

Don't expect that Itell you, that the actual firmware solved my problems. I did that too many times and a few hours later everything crashed. But it seems that I2C is now as reliable as the old one core version of the firmware.

kjm · Nov 6, 2019, 7:45 AM

@crumble It's a real shame. The idea of a python capable micro with integrated 4G looked so good in the advert.

crumble · Nov 5, 2019, 7:51 AM

@kjm

as long as the hard-/software on which the pycom devices base has timing problems, you have to make your software fault tolerant. Playing around with timings is playing tic tac toe. You cannot win as long as you do not control the exact state of hardware and OS.

You have to deal that the device is only 99% reliable and how it can get back into a good state after failure. The high rate of failure makes the ESP 32 a good learning system for this kind of software ;)

kjm · Nov 5, 2019, 1:40 AM

@andreas All of us reinventing the wheel Anrdeas.

andreas · Nov 5, 2019, 1:40 AM

@kjm said in OneWire.py problem:

I made a little mod to def scan

if devices!=[]: return devices
self.reset()

Thanks! It looks like we already did that to be better safe than sorry. Good to know we have not been dreaming here.

# The 1-Wire bus sometimes needs a fix when coming back from deep sleep.
self.adapter.reset()

-- https://github.com/hiveeyes/hiveeyes-micropython-firmware/blob/0.6.0/terkin/sensor/core.py#L212-L216

kjm · Nov 4, 2019, 10:01 PM

@kjm I made a little mod to def scan

def scan(self):
        """
        Return a list of ROMs for all attached devices.
        Each ROM is returned as a bytes object of 8 bytes.
        """
        devices = []
        diff = 65
        rom = False
        for i in range(2):
            for i in range(0xff):
                rom, diff = self._search_rom(rom, diff)
                if rom:
                    devices += [rom]
                if diff == 0:
                    break
            if devices!=[]: return devices
            self.reset()

and got 100% on an overnight run, looks like the DS18B20 sometimes needs a reset after a deepsleep.

andreas · Nov 4, 2019, 8:47 PM

@robert-hh said in OneWire.py problem:

Sorry no. It was my mistake. I did not notice that I had onewire.py on that device. Notably the one which had modified. But that one seems stable.

Thank you very much for this. We are also running that and are quite happy with it. However, we see occasional bumps where the CRC check probably hits the fan. Seeing this happen, it is way better than getting any misreadings out of it.

@robert-hh said in OneWire.py problem:

Pycom does not really react to Pull Requests. Especially PR's for the library are completely ignored. And also for the firmware there seems no on to be in charge of that.

It's a bit sad yes. We understand the core team must be pretty busy and are more than happy to see MicroPython 1.11 finally came to Pycom MicroPython with the most recent release. Kudos to all who made that happen!

Saying this, we would be more than happy if someone would pick up the work from [1]. Maybe someone capable and lucky enough to tackle that problem will be the one we could salute here?

[1] https://github.com/pycom/pycom-micropython-sigfox/pull/356

robert-hh · Nov 4, 2019, 8:30 PM

@andreas Sorry no. It was my mistake. I did not notice that I had onewire.py on that device. Notably the one which had modified. But that one seems stable. And I remember having that running in a test of several sensors for about a week.
Edit: Pycom does not really react to Pull Requests. Especially PR's for the library are completely ignored. And also for the firmware there seems no on to be in charge of that.

andreas · Nov 4, 2019, 8:26 PM

Dear Robert,

@robert-hh said in OneWire.py problem:

I do not understand you problem. The onewire driver is built-in into the machines code.

Unfortunately, this does not happen to be for us. Are you really sure about that detail?

The only files we can find inside the repository are

offgrid:pycom-micropython-sigfox amo$ find . -iname 'onewire.py'
./drivers/onewire/onewire.py
./tests/esp32/onewire.py
./ports/esp8266/modules/onewire.py
./ports/stm32/modules/onewire.py

and the first guy comes from [1] - which is actually the 1-Wire driver for vanilla MicroPython which also has a different module layout.

The driver you are most probably referring to is coming from [2] and is not included by default, unless we are looking in a totally wrong direction.

However, after including that driver manually into our datalogger program, we received many glitches/misreadings from the sensors. After looking at the code in more detail, we found that the crc8 method is not invoked at all, so it's not surprising that things will go south.

After some hair pulling, we finally found the contribution [3] coming from your pen and are reasonably happy with it. Results from @kjm also show that this works better than the vanilla pure-Python driver shipped through pycom-libraries.

Saying this, we feel a bit sad that Pycom hasn't been able to include the native driver from Vanilla MicroPython yet. We humbly started working on [4] yesterday night but will have to back away from that now.

Please correct me if I am wrong on any of these details. Actually, I would be happy to learn where I missed some important detail.

With kind regards,
Andreas.

[1] https://github.com/pycom/pycom-micropython-sigfox/tree/Dev/drivers/onewire
[2] https://github.com/pycom/pycom-libraries/tree/master/lib/onewire
[3] https://github.com/pycom/pycom-libraries/pull/62
[4] https://github.com/pycom/pycom-micropython-sigfox/pull/356

robert-hh · Nov 4, 2019, 8:03 PM

This post is deleted!

kjm · Nov 4, 2019, 11:49 AM

@kjm I tried an ow.scan() after the ow.reset() in start_conversion & read_temp_async, those occasions when I get a None response instead of a temperature are when the scan returns [] instead of [bytearray(b'(\xff\xd3\xe1E\x16\x03>')] I'm really having trouble understanding the logic of 'if self.is busy' in read_temp_async now, because a null response from the scan means it couldn't find the DS18B20 not that the DS18B20 was busy doing a conversion surely?

kjm · Nov 4, 2019, 11:14 AM

@andreas I note Rob has an unused function in there def scan(self), does anybody have an example of that being used?

andreas · Nov 4, 2019, 3:51 AM

Clarification

@kjm said in OneWire.py problem:

Does this mean 1.20.1.r1 is still not quite right?

It definitively is right, no worries.

Future research

@kjm said in OneWire.py problem:

It needs to be redone with a different driver before 100% success can be achieved when reading the DS18B20?

I was too quick on that detail. After getting into the MicroPython code, I see that porting/enabling the modonewire.c from Genuine MicroPython on Pycom MicroPython will require more efforts than I will be able to dedicate to this topic for now.

We failed on porting this to Pycom MicroPython appropriately. While we have come quite far making it compile, it panics at runtime right away when trying to reset the onewire bus.

-- https://community.hiveeyes.org/t/untersuchung-und-verbesserung-des-timings-bei-der-ansteuerung-der-ds18b20-sensoren-unter-micropython/2309/94

Conclusion

So, we believe the onewire.py driver referenced above tweaked by @robert-hh is currently the best you can get for getting the best results out of running it on the Pycom devices.

kjm · Nov 4, 2019, 3:37 AM

@andreas said in OneWire.py problem:

@andreas said in OneWire.py problem:

Taking a fresh perspective on that, I finally recognize the same thing is also in Pycom MicroPython already [3]. So, Pycom most probably missed to update their repositories by removing the old pure-Python driver from [4] to reflect that situation, also misleading us. Bummer!

Saying this, we will try to also move into the right direction now, abandoning the pure-Python driver altogether.

Does this mean 1.20.1.r1 is still not quite right? It needs to be redone with a different driver before https://github.com/pycom/pycom-libraries/blob/86fa8e75916837200a608168a8340093691e8654/examples/DS18X20/onewire.py can achieve 100% success with the DS18B20?

kjm · Nov 4, 2019, 1:07 AM

@kjm What exasperates me is if I run an accelerated test, wakeup-get temperature-deepsleep for 1s the success rate (sample size 50) is 100%. But in my actual boot.py program with lte/rtc/adc/wdt/urequests/10min deepsleeps the success rate (over ~ 8hrs for the same sample size) drops to 92%. It troubles me greatly.

Explore Pybytes | Official Documentation | Report a Firmware Bug/Issue | GitHub

OneWire.py problem

Clarification

Future research

Conclusion

Pycom on Twitter