OneWire.py problem

andreas

Dear @kjm,

We already spent many hours on that topic [1] before finding the onewire.py in the examples folder contributed by @robert-hh which saved us.

@kjm said in OneWire.py problem:

only 75% of attempts to read the DS18B20 return a value, the rest return None

We are sharing your observations.

Given the circumstances, we consider this to be a good thing (TM). Before using that onewire.py driver by @robert-hh (thanks again!), we got lots of glitches with readings of just plain wrong values.

So, we see the CRC-check is working here which makes me happy from the perspective of a software guy. I can't tell if the 25% of None values are related to faulty hardware, missing bus termination or something along the lines.

Most probably, the timing is still not quite right in some cases but at least the CRC mechanism saves us from further troubles of having to compensate any misreadings.

With kind regards,
Andreas.

P.S.: By the way, you will also have to turn off the garbage collector while reading digital sensors like outlined within [2]. Otherwise, things will probably also go south.

[1] https://community.hiveeyes.org/t/untersuchung-und-verbesserung-des-timings-bei-der-ansteuerung-der-ds18b20-sensoren-unter-micropython/2309
[2] https://github.com/hiveeyes/hiveeyes-micropython-firmware/blob/0.6.0/terkin/datalogger.py#L320-L323

andreas

@andreas said in OneWire.py problem:

So, we see the CRC-check is working here which makes me happy from the perspective of a software guy.

From a more general perspective, one should just never to timing-critical stuff from within MicroPython.

Within the codebase of recent Genuine MicroPython, you can see that the {readbit,readbyte,writebit,writebyte} methods are calling into the C-level MicroPython core [2].

Taking a fresh perspective on that, I finally recognize the same thing is also in Pycom MicroPython already [3]. So, Pycom most probably missed to update their repositories by removing the old pure-Python driver from [4] to reflect that situation, also misleading us. Bummer!

Saying this, we will try to also move into the right direction now, abandoning the pure-Python driver altogether.

With kind regards,
Andreas.

[1] https://github.com/micropython/micropython/blob/v1.11/drivers/onewire/onewire.py
[2] https://github.com/micropython/micropython/blob/v1.11/extmod/modonewire.c
[3] https://github.com/pycom/pycom-micropython-sigfox/blob/Dev/extmod/modonewire.c
[4] https://github.com/pycom/pycom-libraries

kjm

@andreas said in OneWire.py problem:

Most probably, the timing is still not quite right in some cases but at least the CRC mechanism saves us from further troubles of having to compensate any misreadings.

Mea Culpa, I got my OneWire.py & my onewire.py mixed up. The driver at https://github.com/pycom/pycom-libraries/blob/86fa8e75916837200a608168a8340093691e8654/examples/DS18X20/onewire.py actually has a 93% success rate with the DS18B20 so ~20% better than the earlier versions.

kjm

@kjm What exasperates me is if I run an accelerated test, wakeup-get temperature-deepsleep for 1s the success rate (sample size 50) is 100%. But in my actual boot.py program with lte/rtc/adc/wdt/urequests/10min deepsleeps the success rate (over ~ 8hrs for the same sample size) drops to 92%. It troubles me greatly.

kjm

@andreas said in OneWire.py problem:

@andreas said in OneWire.py problem:

Taking a fresh perspective on that, I finally recognize the same thing is also in Pycom MicroPython already [3]. So, Pycom most probably missed to update their repositories by removing the old pure-Python driver from [4] to reflect that situation, also misleading us. Bummer!

Saying this, we will try to also move into the right direction now, abandoning the pure-Python driver altogether.

Does this mean 1.20.1.r1 is still not quite right? It needs to be redone with a different driver before https://github.com/pycom/pycom-libraries/blob/86fa8e75916837200a608168a8340093691e8654/examples/DS18X20/onewire.py can achieve 100% success with the DS18B20?

andreas

Clarification

@kjm said in OneWire.py problem:

Does this mean 1.20.1.r1 is still not quite right?

It definitively is right, no worries.

Future research

@kjm said in OneWire.py problem:

It needs to be redone with a different driver before 100% success can be achieved when reading the DS18B20?

I was too quick on that detail. After getting into the MicroPython code, I see that porting/enabling the modonewire.c from Genuine MicroPython on Pycom MicroPython will require more efforts than I will be able to dedicate to this topic for now.

We failed on porting this to Pycom MicroPython appropriately. While we have come quite far making it compile, it panics at runtime right away when trying to reset the onewire bus.

-- https://community.hiveeyes.org/t/untersuchung-und-verbesserung-des-timings-bei-der-ansteuerung-der-ds18b20-sensoren-unter-micropython/2309/94

Conclusion

So, we believe the onewire.py driver referenced above tweaked by @robert-hh is currently the best you can get for getting the best results out of running it on the Pycom devices.

kjm

@andreas I note Rob has an unused function in there def scan(self), does anybody have an example of that being used?

kjm

@kjm I tried an ow.scan() after the ow.reset() in start_conversion & read_temp_async, those occasions when I get a None response instead of a temperature are when the scan returns [] instead of [bytearray(b'(\xff\xd3\xe1E\x16\x03>')] I'm really having trouble understanding the logic of 'if self.is busy' in read_temp_async now, because a null response from the scan means it couldn't find the DS18B20 not that the DS18B20 was busy doing a conversion surely?

robert-hh

This post is deleted!

andreas

Dear Robert,

@robert-hh said in OneWire.py problem:

I do not understand you problem. The onewire driver is built-in into the machines code.

Unfortunately, this does not happen to be for us. Are you really sure about that detail?

The only files we can find inside the repository are

offgrid:pycom-micropython-sigfox amo$ find . -iname 'onewire.py'
./drivers/onewire/onewire.py
./tests/esp32/onewire.py
./ports/esp8266/modules/onewire.py
./ports/stm32/modules/onewire.py

and the first guy comes from [1] - which is actually the 1-Wire driver for vanilla MicroPython which also has a different module layout.

The driver you are most probably referring to is coming from [2] and is not included by default, unless we are looking in a totally wrong direction.

However, after including that driver manually into our datalogger program, we received many glitches/misreadings from the sensors. After looking at the code in more detail, we found that the crc8 method is not invoked at all, so it's not surprising that things will go south.

After some hair pulling, we finally found the contribution [3] coming from your pen and are reasonably happy with it. Results from @kjm also show that this works better than the vanilla pure-Python driver shipped through pycom-libraries.

Saying this, we feel a bit sad that Pycom hasn't been able to include the native driver from Vanilla MicroPython yet. We humbly started working on [4] yesterday night but will have to back away from that now.

Please correct me if I am wrong on any of these details. Actually, I would be happy to learn where I missed some important detail.

With kind regards,
Andreas.

[1] https://github.com/pycom/pycom-micropython-sigfox/tree/Dev/drivers/onewire
[2] https://github.com/pycom/pycom-libraries/tree/master/lib/onewire
[3] https://github.com/pycom/pycom-libraries/pull/62
[4] https://github.com/pycom/pycom-micropython-sigfox/pull/356

robert-hh

@andreas Sorry no. It was my mistake. I did not notice that I had onewire.py on that device. Notably the one which had modified. But that one seems stable. And I remember having that running in a test of several sensors for about a week.
Edit: Pycom does not really react to Pull Requests. Especially PR's for the library are completely ignored. And also for the firmware there seems no on to be in charge of that.

andreas

@robert-hh said in OneWire.py problem:

Sorry no. It was my mistake. I did not notice that I had onewire.py on that device. Notably the one which had modified. But that one seems stable.

Thank you very much for this. We are also running that and are quite happy with it. However, we see occasional bumps where the CRC check probably hits the fan. Seeing this happen, it is way better than getting any misreadings out of it.

@robert-hh said in OneWire.py problem:

Pycom does not really react to Pull Requests. Especially PR's for the library are completely ignored. And also for the firmware there seems no on to be in charge of that.

It's a bit sad yes. We understand the core team must be pretty busy and are more than happy to see MicroPython 1.11 finally came to Pycom MicroPython with the most recent release. Kudos to all who made that happen!

Saying this, we would be more than happy if someone would pick up the work from [1]. Maybe someone capable and lucky enough to tackle that problem will be the one we could salute here?

[1] https://github.com/pycom/pycom-micropython-sigfox/pull/356

kjm

@kjm I made a little mod to def scan

def scan(self):
        """
        Return a list of ROMs for all attached devices.
        Each ROM is returned as a bytes object of 8 bytes.
        """
        devices = []
        diff = 65
        rom = False
        for i in range(2):
            for i in range(0xff):
                rom, diff = self._search_rom(rom, diff)
                if rom:
                    devices += [rom]
                if diff == 0:
                    break
            if devices!=[]: return devices
            self.reset()

and got 100% on an overnight run, looks like the DS18B20 sometimes needs a reset after a deepsleep.

andreas

@kjm said in OneWire.py problem:

I made a little mod to def scan

if devices!=[]: return devices
self.reset()

Thanks! It looks like we already did that to be better safe than sorry. Good to know we have not been dreaming here.

# The 1-Wire bus sometimes needs a fix when coming back from deep sleep.
self.adapter.reset()

-- https://github.com/hiveeyes/hiveeyes-micropython-firmware/blob/0.6.0/terkin/sensor/core.py#L212-L216

kjm

@andreas All of us reinventing the wheel Anrdeas.

crumble

@kjm

as long as the hard-/software on which the pycom devices base has timing problems, you have to make your software fault tolerant. Playing around with timings is playing tic tac toe. You cannot win as long as you do not control the exact state of hardware and OS.

You have to deal that the device is only 99% reliable and how it can get back into a good state after failure. The high rate of failure makes the ESP 32 a good learning system for this kind of software ;)

kjm

@crumble It's a real shame. The idea of a python capable micro with integrated 4G looked so good in the advert.

crumble

@kjm

Pycom ran into this damn ESP 32 trap.

Don't expect that Itell you, that the actual firmware solved my problems. I did that too many times and a few hours later everything crashed. But it seems that I2C is now as reliable as the old one core version of the firmware.

andreas

Dear @kjm, @crumble and @robert-hh,

@andreas said in OneWire.py problem:

I was too quick on that detail. After getting into the MicroPython code, I see that porting/enabling the modonewire.c from Genuine MicroPython on Pycom MicroPython will require more efforts than I will be able to dedicate to this topic for now.

Finally, we have been able to add support for the native 1-Wire/DS18X20 driver on Pycom MicroPython and published respective firmware images called 1.20.1.r1-0.7.0-vanilla-dragonfly-onewire. See also Inofficial firmware bakery for Pycom/ESP32 devices.

Please be aware that you will have to use a set of different Python drivers to access the lowlevel API available through https://github.com/micropython/micropython/tree/master/drivers/onewire.

If you are willing to test this, we will be happy to hear about the outcome for you.

With kind regards,
Andreas.

John Baird

Jumping in a bit late, but FWIIW - I've been running DS18B20 sensors in many devices that regularly go to deepsleep for months now. They work very well, with few (if any) missed reads. I think the difference here is that our application is very power sensitive, so the DS180B20 is powered from a supply that turns off every time the module goes into deepsleep. Perhaps you could try parasitic power? That would also turn off the sensor during deepsleep.

Explore Pybytes | Official Documentation | Report a Firmware Bug/Issue | GitHub

OneWire.py problem

Clarification

Future research

Conclusion

Pycom on Twitter