LoPy stops detecting BLE advertisements
-
@juansezoh That's simply how it works, but if you call the function fast enough you'll get all "pings" anyway. None will be returned if there are no pings available.
To avoid detection to stop altogether after a while I do this:
while True: if not bluetooth.isscanning(): print("Restarting scan") bluetooth.deinit() bluetooth.init() bluetooth.start_scan(10) time.sleep(1)Here's a complete example that works in a stable way. I ignore UUID in this case. I use rssi - power as a coarse way of determining distance.
from network import Bluetooth import pycom import time import gc pycom.heartbeat(False) timeSleep = 0.01 bluetooth = Bluetooth() def new_adv_event(event): if event.events() == Bluetooth.NEW_ADV_EVENT: anydata = True while anydata: device = bluetooth.get_adv() if device != None: rssi = device.rssi data = [b for b in device.data] if data[5] == 0x4c and data[6] == 0x00 and data[7] == 0x02 and data[8] == 0x15: majorid = data[25] * 256 + data[26] minorid = data[27] * 256 + data[28] power = data[29] if power > 127: power -= 256 pycom.rgbled(0x000800) print("iBeacon: " + str(rssi) + '/' + str(power) + ' (' + str(rssi - power) + ') ' + str(majorid) + '/' + str(minorid)) else: pycom.rgbled(0x000008) time.sleep(timeSleep) pycom.rgbled(0x000000) else: anydata = False bluetooth.callback(trigger = Bluetooth.NEW_ADV_EVENT, handler = new_adv_event) while True: if not bluetooth.isscanning(): print("Restarting scan") bluetooth.deinit() bluetooth.init() bluetooth.start_scan(10) time.sleep(1)
-
@papasmurph @RichardTXD Do you have any progress you can share from reading beacons? Can you explain why the advertisment feature only gives me one beacon at a time ?. I hope you can help me, thank you very much.
-
@RichardTXD Great. I'll try that approach. Thanks. Update: So far stable, after 10 or so hours. I'll keep it going.
This is the code I use now. I ignore iBeacon UUID at the moment, as all of them use my UUID.
from network import Bluetooth import pycom import time import gc pycom.heartbeat(False) timeSleep = 0.01 bluetooth = Bluetooth() def new_adv_event(event): if event.events() == Bluetooth.NEW_ADV_EVENT: anydata = True while anydata: device = bluetooth.get_adv() if device != None: rssi = device.rssi data = [b for b in device.data] if data[5] == 0x4c and data[6] == 0x00 and data[7] == 0x02 and data[8] == 0x15: majorid = data[25] * 256 + data[26] minorid = data[27] * 256 + data[28] power = data[29] if power > 127: power -= 256 pycom.rgbled(0x000800) print("iBeacon: " + str(rssi) + '/' + str(power) + ' (' + str(rssi - power) + ') ' + str(majorid) + '/' + str(minorid)) else: pycom.rgbled(0x000008) time.sleep(timeSleep) pycom.rgbled(0x000000) else: anydata = False bluetooth.callback(trigger = Bluetooth.NEW_ADV_EVENT, handler = new_adv_event) while True: if not bluetooth.isscanning(): print("Restarting scan") bluetooth.deinit() bluetooth.init() bluetooth.start_scan(10) time.sleep(1)
-
@stef
It has been running for about 18.5hours now and received 960,000 packets.
That dumb bit of code seems to keep it alive - at the expense of turning off bluetooth and turning it on periodically - at least with very simple code. I'll update the code till it breaks again...
-
@stef
I'm trying scanning for just 10 seconds, then de-init the bluetooth, re-init it and scan for another 10 seconds. I get all the packets from the queue every 0.1 second. It's been running for just over 3 hours now. Read 168,000 packets and over 1100 resets. This simple snippet of codeif bluetooth.isscanning() != True: bluetoothResets += 1 print(' Turning on bluetooth. Resets = ' + str(bluetoothResets)) bluetooth.deinit() bluetooth.init() bluetooth.start_scan(10) LogRecord(LOG_startScanning)I would be interested to hear how your packet counting goes with the WiFi disabled & the server turned off.
-
Yes , my use of LoPy is only based on detection , not data transfer ( as we are doing location ) . If it's help , i can test a long time BT loop directly with ESP32 Framork ( if that's working it's a python problem , if not we just have to wait for IDF update ).
-
@stef
@papasmurph
@daniel
'Stef' I'm thinking your testing shows that the Pycom software sits on top of the ESP32 firmware, same as the ESP32_core_Board_V2 as both have very similar performance.
From reading various posts it is apparent that Expressif are still working on their firmware.
I would feel disappointed at the results also. It seems somewhat poor imho - I would have expected better in an RF clean environment. Getting close to 100% is probably a big ask.
Maybe try shutting down the WiFi and in particular the server - that may have an impact (if it is sending out SSIDs I would have thought so). Interested to hear how you go on that.
For myself, we are just wanting to receive the packets. We are not ranging, just want to know we are nearby so losing a few isn't an issue.What is a killer for us is the way it fails. Silently. We can't survive bugs like that. It is a show stopper and I've been looking at options with other products last night and today. I don't mind if it fails occasionally, so long as I can know quickly and take corrective action.
For now I'm (dis)satisfied that the BLE is both very insensitive "deaf" and also buggy. I've probably spent 2 days and not been able to make it not crash after a few hours at most.
I'll monitor when updates come out and give them a try. Even the most simple code to read all BLE advertisements over time fails. It is simple to test but it does take time. About 3.6 hours is my record so far.I haven't tried turning off the server either - might do that now and report if there is any major change.
I'll be focussing on testing the 24/7 capability of the product (WiPy) in other areas, clock, timers, I2C and WiFi. If it won't do that then there is probably little point proceeding.
-
@stef I moved away all other wireless devices 5 or so meters away and kept only one beacon. I got 31 pings in 180 seconds, with ping interval of 1 second, so 83% loss. WiFi was active on the LoPy. BLE Scanner has a loss of < 5-10% at the same distance.
-
@papasmurph
I start a simple test on Lopy and an generic board : Scan for 60 secondes a Kontact Beacon ( only one - 350ms message freq ) , the beacon is put on the Lopy .
Then i just count how many time this beacon appear for 60sTOTAL CAST : 60000/350 =170
TOTAL for python test on Lopy ( 4x - reset after each ) :100-103-130-128
TOTAL for C test (4x) on ESP32_core_Board_V2 ( esp32-idf ) :99-110-120-115so , in the two cases , we have min. 30% packet lost.
Someone can test with other beacons or hardware ?
Stéphane A.
-
@RichardTXD I have an iOS/Android app (CliqTags Spotter) for detecting beacons and geofences, and it has a hidden page/pane that shows all beacon activity in a stable list (once a beacon is found it's stays in its place, unless it goes away for a longer time, so it's easy to see what happens with any specific beacon), and it shows many more pings than my Python app, so there's a difference. I also tried with BLE Scanner and got similar results. In a setting with only one or two beacons near by the app detects almost all pings. I was not aware of the use case. I use beacons mostly as pure proximity markers, not for absolute location, even though that's no doubt an interesting field that I'd like to get into, and I understand the relevance of a higher frequency for that use. Regarding accuracy: Try holding a beacon between you and the detecting device. Water and metal are beacon killers. But the main issue here is of course that it stops.
-
@papasmurph
I mostly agree - and strongly agree on battery.
To a small degree I'd be surprised with 3 beacons if you collected all advertizing packets every time for a period or more than a few seconds.
Problem/advantage is that shorter intervals give better geolocation. iDevices seem to do an averaging and lots of packets make it more stable iirc. They work via RSSI and it is a real struggle. Android seems to report everything (could be wrong on that).
We use 500mS as we need better battery life - and accept the lesser accuracy.
The marketing material I see that claims 1 metre and better accuracy makes me smile. In the office standing still it is very variable with 500mS.
Problem with 1 second and low power, Level-0, is that it is very possible to move past it and not get a reading in an iDevice as it doesn't get enough hits. Battery life is great tho.
I suppose it depends on the application & tradeoffs.
Packet collisions... that is my theory as to what is happening. I may be wrong - happy to be adviced by those more knowledgable but it would seem logical.
Multipath is also an issue we see in the field. Reflections of the same packet arriving time shifted as a result of a reflecting surface. Seen this in the field with just one beacon and small dead spots in a zone where it is working all around. Same effect as 'ghosting' all those years ago on television but leads to broken packets.
Agree, a longer sample time would be good.
-
@stef
Hi Stef. We use Kontakt beacons also. I have something like 90 to 100 beacons in the office with me, set on Level-2 power and 500mS interval.
I notice a speed difference in scanning depending on code. I can get close to 1,000 packets a minute or down below 600 depending on the code. One run I got 103,000 packets in about 2 hours. A different set of code with different garbage collection I got about 90,000 packets in 3.5 hours approx. Quite a difference.
I'm not that worried about lost packets; the possibility of collisions and broken packets is very high - and then there are my 2 or 3 WiFi devices in the office also pushing out RF. Broken and lost packets are a given imho. There is no way they won't collide.
With 3 beacons and 250mS gap there would be less collisions but I would not rule it out. Maybe set a longer interval in the beacons and increase the scanning window.
Latest test-tweak of the code has run for 1hr 20m approx. Nothing special. I would have thought 24 to 48 hours 'good' but not great if it falls over then. The problem is that it falls over and seems to die.No error message. No recovery.
-
@stef You can reconfigure any beacon (including Kontakt's) to ping less often. 0.25 is usually unnecessarily short and drains the battery fast, unless they are USB-attached of course. Try 1 second. I recommend that you scan for 10 seconds and collect all detected beacons in a list, send the list (in some clever format) and then activate sleep. I don't understand why you miss any pings with that high frequency. The Pycom should detect all and queue them (up to 8) that should all be visible via get_adv if you just call it often enough (at least every 50 milliseconds or so), or better, use a callback with an internal loop for "strays", the way I do it now. I've noticed not all pings show up, but I have assumed that's a bug.
-
@RichardTXD
yes , the only workarround i found for this problem is to use deepsleep ( as it act like a reset).
Another problem i try to solve is to avoid all the 'miss' on a short scan time.
I use Kontack Beacons , they push a message every 250ms.
With a 2 secondes scan , on 3 Beacons surrounding , one tag is missing on 1 / 4 loop.any hint ?
-
@papasmurph
Yes. I truly hope so and soon. It is unstable. Same for WiPy 2.0Recap : My code is basically yours with extra diagnositcs for number of beacons read, heap space, number of items in the queue.
When it fails, I press control-C and each time it says the last line was "while device != None" (I pull all packets out of the queue when it wakes up).
Is this a hint to anyone?
I've spent the day runing tests on it. Best is about 3.6 hours. Worst about 10 minutes. Putting in garbage collection, taking it out. Changing "sleep" for "idle()", adding diagnostic messages. No closer to nailing down what it is.
Tempted to run the program without any bluetooth to see if it is the underlying system that is flaky. Maybe tomorrow.
Rather frustrating.
-
@RichardTXD As this is discussed in Issues&Bugs, let's hope Pycom acts on it.
-
@papasmurph
I am Running WiPy 2.0 with latest firmware flashed yesterday.
Windows 7 and Atom. I'm using an expansion board.
I set my PC to not shutdown/go to sleep etc.Hi. There is also a concern with callback. I tried it also. Then I went back to a non-call back program. The callback was still triggering.
It would appear that unless you press reset of the board, the callback lingers in memory.
I would not be surprised at a difference. I'm guessing memory is deallocated when the callback is terminated and that may be at the heart of it all.@stef
Hi. You are quite correct. It is unstable.
I've seen different crashes depending on if garbage collection is done. If I don't garbage collect it seems to crash quicker - I will go back to testing that later.I'm still trying to get a feel for how and where it crashes but so far it seems random - a few hours to tens of minutes.
I modded the program to tell me how many beacons were received. I have a lot of beacons in the office. I empty the queue on each iteration.
I did 3 tests late yesterday.
I set it to scan for -1 seconds = infinitely
There is a garbage collection before each read of bluetooth into its variable.
I press reset between each program run
Test-1. It received 103,139 beacons and failed after about 2.5 hours
Test-2. It received 17,094 beacons and failed - not all that long.
Test-3. It received 31,945 beacons and failed.
Each fail was the code hanging up. Stopped.
I just now put in an elapsed seconds on the end of each result.
The outer while loop repeats till it is not scanning. An inner loop empties the queue. I assume it was still scanning, yet nothing was received.Until we get past this issue, I can't ship product of course, same as others I suppose. I'm trying to think of a workaround.
-
@papasmurph ,
I confirm that , still stoping after time . nothing to do but reset.
New firmware doesn't matter .
-
@RichardTXD With callback to an event handler it can work for an hour or so, but still eventually stops. It's time Pycom takes a look at it.
-
@papasmurph
Hi.
I've been working on this, following your progress and the great community support - thanks. This is a show stopper for me if it can't be made to scan reliably 24/7.Have you made progress?
I found that it just stops scanning early without an error message. I told it to scan for "-1" seconds = forever. I thought it was broken packets but I (currently) think that is wrong.
I've been tinkering with your code. I found it sensitive to putting in a gc.collect() just before fetching a packet (device = bluetooth.get etc) - that made the difference between scanning for approx 1 minute to many minutes (still going after 10 and I must leave the office).
Much more testing required.
