OneWire.py problem

[kjm]

I've tried a couple of different flavours of the OneWire library with a single DS18B20 temperature sender. Both routines, the one at the bottom of the page at https://docs.pycom.io/tutorials/all/owd/, and a more resent version at https://github.com/pycom/pycom-libraries/blob/master/lib/onewire/onewire.py will chug away happily in a loop printing temperatures at 2s intervals forever.

However if I try them in a program that deepsleeps they often fail to give a temperature. If I'm running the older library, read_temp_async gives an IndexError(most likely indicating it can't find the DS18B20). While the newer version returns None (indicating a busy DS18B20). I feel this difference is indicative of what the root cause of the problem is, but I don't understand the library well enough to figure it out.

My problem is the only way to fix this seems to be to reboot the gpy. Putting the temperature read in a loop, or having multiple reads scattered through the main program do no good. Once the library stops working it stays not working till I reset the GPY.

Can anyone suggest something I might try other than a reboot? The DS18B20 is continuously powered & works fine with other devices so I don't think it is the problem. Something the OneWire library uses is presumably unavailable after random deepsleep wakeups & a reboot is the only thing currently that seems to fix it.

[andreas]

Dear @kjm,

we are successfully using the onewire.py [1] by @robert-hh (thanks again for your countless excellent contributions!) within our datalogger program [2]. It is attributed with Optimize timing, enable CRC check and slim the code as outlined through [3].

With kind regards,
Andreas.

[1] https://github.com/pycom/pycom-libraries/blob/dabce8d9cc8cf3b9849446000d811a39c53b6093/examples/DS18X20/onewire.py
[2] https://github.com/hiveeyes/hiveeyes-micropython-firmware
[3] https://github.com/pycom/pycom-libraries/pull/62

[kjm]

I uploaded the new library with crc lookup at https://github.com/pycom/pycom- As if to mock me the very first cycle returned a busy result for the DS18B20. I'm going to have to swap to a TMP36 or some other alternative now.

[kjm]

@andreas said in OneWire.py problem:

Dear @kjm,

we are successfully using the onewire.py [1] by @robert-hh (thanks again for your countless excellent contributions!) within our datalogger program [2]. It is attributed with Optimize timing, enable CRC check and slim the code as outlined through [3].

I'm using the latest onewire https://github.com/pycom/pycom-libraries/blob/86fa8e75916837200a608168a8340093691e8654/examples/DS18X20/onewire.py driver I could find Andreas but only 75% of attempts to read the DS18B20 return a value, the rest return None. I've got 2 different DS18B20s here (curiously one draws 50uA and the other <0.1uA) but they both average the same 75% success rate.

[andreas]

Dear @kjm,

We already spent many hours on that topic [1] before finding the onewire.py in the examples folder contributed by @robert-hh which saved us.

@kjm said in OneWire.py problem:

only 75% of attempts to read the DS18B20 return a value, the rest return None

We are sharing your observations.

Given the circumstances, we consider this to be a good thing (TM). Before using that onewire.py driver by @robert-hh (thanks again!), we got lots of glitches with readings of just plain wrong values.

So, we see the CRC-check is working here which makes me happy from the perspective of a software guy. I can't tell if the 25% of None values are related to faulty hardware, missing bus termination or something along the lines.

Most probably, the timing is still not quite right in some cases but at least the CRC mechanism saves us from further troubles of having to compensate any misreadings.

With kind regards,
Andreas.

P.S.: By the way, you will also have to turn off the garbage collector while reading digital sensors like outlined within [2]. Otherwise, things will probably also go south.

[1] https://community.hiveeyes.org/t/untersuchung-und-verbesserung-des-timings-bei-der-ansteuerung-der-ds18b20-sensoren-unter-micropython/2309
[2] https://github.com/hiveeyes/hiveeyes-micropython-firmware/blob/0.6.0/terkin/datalogger.py#L320-L323

[andreas]

@andreas said in OneWire.py problem:

So, we see the CRC-check is working here which makes me happy from the perspective of a software guy.

From a more general perspective, one should just never to timing-critical stuff from within MicroPython.

Within the codebase of recent Genuine MicroPython, you can see that the {readbit,readbyte,writebit,writebyte} methods are calling into the C-level MicroPython core [2].

Taking a fresh perspective on that, I finally recognize the same thing is also in Pycom MicroPython already [3]. So, Pycom most probably missed to update their repositories by removing the old pure-Python driver from [4] to reflect that situation, also misleading us. Bummer!

Saying this, we will try to also move into the right direction now, abandoning the pure-Python driver altogether.

With kind regards,
Andreas.

[1] https://github.com/micropython/micropython/blob/v1.11/drivers/onewire/onewire.py
[2] https://github.com/micropython/micropython/blob/v1.11/extmod/modonewire.c
[3] https://github.com/pycom/pycom-micropython-sigfox/blob/Dev/extmod/modonewire.c
[4] https://github.com/pycom/pycom-libraries

[kjm]

@andreas said in OneWire.py problem:

Most probably, the timing is still not quite right in some cases but at least the CRC mechanism saves us from further troubles of having to compensate any misreadings.

Mea Culpa, I got my OneWire.py & my onewire.py mixed up. The driver at https://github.com/pycom/pycom-libraries/blob/86fa8e75916837200a608168a8340093691e8654/examples/DS18X20/onewire.py actually has a 93% success rate with the DS18B20 so ~20% better than the earlier versions.

[kjm]

@kjm What exasperates me is if I run an accelerated test, wakeup-get temperature-deepsleep for 1s the success rate (sample size 50) is 100%. But in my actual boot.py program with lte/rtc/adc/wdt/urequests/10min deepsleeps the success rate (over ~ 8hrs for the same sample size) drops to 92%. It troubles me greatly.

[kjm]

@andreas said in OneWire.py problem:

@andreas said in OneWire.py problem:

Taking a fresh perspective on that, I finally recognize the same thing is also in Pycom MicroPython already [3]. So, Pycom most probably missed to update their repositories by removing the old pure-Python driver from [4] to reflect that situation, also misleading us. Bummer!

Saying this, we will try to also move into the right direction now, abandoning the pure-Python driver altogether.

Does this mean 1.20.1.r1 is still not quite right? It needs to be redone with a different driver before https://github.com/pycom/pycom-libraries/blob/86fa8e75916837200a608168a8340093691e8654/examples/DS18X20/onewire.py can achieve 100% success with the DS18B20?

[andreas]

Clarification

@kjm said in OneWire.py problem:

Does this mean 1.20.1.r1 is still not quite right?

It definitively is right, no worries.

Future research

@kjm said in OneWire.py problem:

It needs to be redone with a different driver before 100% success can be achieved when reading the DS18B20?

I was too quick on that detail. After getting into the MicroPython code, I see that porting/enabling the modonewire.c from Genuine MicroPython on Pycom MicroPython will require more efforts than I will be able to dedicate to this topic for now.

We failed on porting this to Pycom MicroPython appropriately. While we have come quite far making it compile, it panics at runtime right away when trying to reset the onewire bus.

-- https://community.hiveeyes.org/t/untersuchung-und-verbesserung-des-timings-bei-der-ansteuerung-der-ds18b20-sensoren-unter-micropython/2309/94

Conclusion

So, we believe the onewire.py driver referenced above tweaked by @robert-hh is currently the best you can get for getting the best results out of running it on the Pycom devices.

[kjm]

@andreas I note Rob has an unused function in there def scan(self), does anybody have an example of that being used?

[kjm]

@kjm I tried an ow.scan() after the ow.reset() in start_conversion & read_temp_async, those occasions when I get a None response instead of a temperature are when the scan returns [] instead of [bytearray(b'(\xff\xd3\xe1E\x16\x03>')] I'm really having trouble understanding the logic of 'if self.is busy' in read_temp_async now, because a null response from the scan means it couldn't find the DS18B20 not that the DS18B20 was busy doing a conversion surely?

[robert-hh]

This post is deleted!

[andreas]

Dear Robert,

@robert-hh said in OneWire.py problem:

I do not understand you problem. The onewire driver is built-in into the machines code.

Unfortunately, this does not happen to be for us. Are you really sure about that detail?

The only files we can find inside the repository are

offgrid:pycom-micropython-sigfox amo$ find . -iname 'onewire.py'
./drivers/onewire/onewire.py
./tests/esp32/onewire.py
./ports/esp8266/modules/onewire.py
./ports/stm32/modules/onewire.py

and the first guy comes from [1] - which is actually the 1-Wire driver for vanilla MicroPython which also has a different module layout.

The driver you are most probably referring to is coming from [2] and is not included by default, unless we are looking in a totally wrong direction.

However, after including that driver manually into our datalogger program, we received many glitches/misreadings from the sensors. After looking at the code in more detail, we found that the crc8 method is not invoked at all, so it's not surprising that things will go south.

After some hair pulling, we finally found the contribution [3] coming from your pen and are reasonably happy with it. Results from @kjm also show that this works better than the vanilla pure-Python driver shipped through pycom-libraries.

Saying this, we feel a bit sad that Pycom hasn't been able to include the native driver from Vanilla MicroPython yet. We humbly started working on [4] yesterday night but will have to back away from that now.

Please correct me if I am wrong on any of these details. Actually, I would be happy to learn where I missed some important detail.

With kind regards,
Andreas.

[1] https://github.com/pycom/pycom-micropython-sigfox/tree/Dev/drivers/onewire
[2] https://github.com/pycom/pycom-libraries/tree/master/lib/onewire
[3] https://github.com/pycom/pycom-libraries/pull/62
[4] https://github.com/pycom/pycom-micropython-sigfox/pull/356

[robert-hh]

@andreas Sorry no. It was my mistake. I did not notice that I had onewire.py on that device. Notably the one which had modified. But that one seems stable. And I remember having that running in a test of several sensors for about a week.
Edit: Pycom does not really react to Pull Requests. Especially PR's for the library are completely ignored. And also for the firmware there seems no on to be in charge of that.

[andreas]

@robert-hh said in OneWire.py problem:

Sorry no. It was my mistake. I did not notice that I had onewire.py on that device. Notably the one which had modified. But that one seems stable.

Thank you very much for this. We are also running that and are quite happy with it. However, we see occasional bumps where the CRC check probably hits the fan. Seeing this happen, it is way better than getting any misreadings out of it.

@robert-hh said in OneWire.py problem:

Pycom does not really react to Pull Requests. Especially PR's for the library are completely ignored. And also for the firmware there seems no on to be in charge of that.

It's a bit sad yes. We understand the core team must be pretty busy and are more than happy to see MicroPython 1.11 finally came to Pycom MicroPython with the most recent release. Kudos to all who made that happen!

---

Saying this, we would be more than happy if someone would pick up the work from [1]. Maybe someone capable and lucky enough to tackle that problem will be the one we could salute here?

[1] https://github.com/pycom/pycom-micropython-sigfox/pull/356

[kjm]

@kjm I made a little mod to def scan

def scan(self):
        """
        Return a list of ROMs for all attached devices.
        Each ROM is returned as a bytes object of 8 bytes.
        """
        devices = []
        diff = 65
        rom = False
        for i in range(2):
            for i in range(0xff):
                rom, diff = self._search_rom(rom, diff)
                if rom:
                    devices += [rom]
                if diff == 0:
                    break
            if devices!=[]: return devices
            self.reset()

and got 100% on an overnight run, looks like the DS18B20 sometimes needs a reset after a deepsleep.

[andreas]

@kjm said in OneWire.py problem:

I made a little mod to def scan

if devices!=[]: return devices
self.reset()

Thanks! It looks like we already did that to be better safe than sorry. Good to know we have not been dreaming here.

# The 1-Wire bus sometimes needs a fix when coming back from deep sleep.
self.adapter.reset()

-- https://github.com/hiveeyes/hiveeyes-micropython-firmware/blob/0.6.0/terkin/sensor/core.py#L212-L216

[kjm]

@andreas All of us reinventing the wheel Anrdeas.

[crumble]

@kjm

as long as the hard-/software on which the pycom devices base has timing problems, you have to make your software fault tolerant. Playing around with timings is playing tic tac toe. You cannot win as long as you do not control the exact state of hardware and OS.

You have to deal that the device is only 99% reliable and how it can get back into a good state after failure. The high rate of failure makes the ESP 32 a good learning system for this kind of software ;)