New firmware release, version 1.6.0.b1

daniel

Hello everyone,

A new firmware release is available. We spend quite some upgrading to the new IDF which has solved several bugs around WiFi and Bluetooth stability. We have focused mainly on bug fixes and stability improvements. These are the release notes:

Add parameter to enable/disable WLAN power save mode. Bump version to 1.6.0.b1.
Update all libraries to match the new IDF.
Update to the new IDF. Move most SX1272 callbacks out of IRAM.
Add descriptor to characteristics for notifications.
Fix failing US915 tests related to channels mask MAC commands.
Remove bind method for Sigfox sockets.
Remove unused async parameter from the Sigfox cmd structure.
Correct LoRa US915 remaining channel mask and use MP_ERROR consistently.
Fix the channel counting algorithm for US-915 channels. Thanks to @cmsedore for his contribution!

If you don't have the latest updater tool, please make sure to get it from here:

https://www.pycom.io/resources/ (Under Firmware Updates)

Cheers,
Daniel

Pete

Thanks guys - fixes to WiFi stability sound very welcome :-)

Pete

livius

@daniel
some progress
faster wifi connection(join) and faster connect
AP mode is really stable - but not STA

OSError: [Errno 104] ECONNRESET :(

rst:0x1 (POWERON_RESET),boot:0x3b (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0x00
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:QIO, clock div:2
load:0x3fff9010,len:8
load:0x3fff9018,len:248
load:0x40078000,len:4056
load:0x4009fc00,len:920
entry 0x4009fde4
I (2307) wifi: frc2_timer_task_hdl:3ffd4a00, prio:22, stack:2048
I (2312) wifi: Init lldesc rx mblock:10
I (2312) wifi: Init lldesc rx ampdu len mblock:7
I (2312) wifi: Init lldesc rx ampdu entry mblock:4
I (2315) wifi: pp_task_hdl : 3ffdbb3c, prio:23, stack:8192
I (2320) wifi: sleep disable
I (3309) wifi: mode : softAP (24:0a:c4:00:6e:c3)
16.09167 102101.2 133.7686
wifi_sta
I (4058) wifi: sleep disable
I (4059) wifi: mode : sta (24:0a:c4:00:6e:c2)
livius
UPC77A39D9
TP-LINK_9A6D1E
UPC Wi-Free
UPC Wi-Free
UPC0838976
2.4G-vnet-1C0A58
DIRECT-y6-BRAVIA
UPC248017814
2.4G-Vectra-WiFi-B7C11C
keke
UPC8578324
vnet-A6EA06
UPC Wi-Free
vnet-EC0D8E
2.4G-vnet-08CC4C
UPC1334070
Dobra Jama
UPC Wi-Free
vnet-35A54A
UPC Wi-Free
XY
UPCB243C94
UPC7321003
I (7611) wifi: n:9 2, o:6 0, ap:255 255, sta:9 2, prof:6
I (8599) wifi: state: init -> auth (b0)
I (8601) wifi: state: auth -> assoc (0)
I (8607) wifi: state: assoc -> run (10)
I (8623) wifi: connected with livius, channel 9
WLAN connection to the router succeeded!
(1970, 1, 1, 0, 0, 6, 425460, None)
(1970, 1, 1, 0, 0, 6, 426040, None)
connected to socket
[UID=24:0a:c4:00:6e:c2] None, 1, 35, 10
Traceback (most recent call last):
  File "main.py", line 14, in <module>
  File "wifi_sta_router.py", line 159, in <module>
OSError: [Errno 104] ECONNRESET
MicroPython v1.8.6-442-gccd949ca on 2017-02-14; WiPy with ESP32
Type "help()" for more information.
>>> I (18608) wifi: pm start, type:0

and problem with ntp - try e.g.:

from socket import AF_INET, SOCK_DGRAM
from machine import RTC
def getNTPTime(host = "pool.ntp.org"):
    port = 123
    buf = 1024
    address = socket.getaddrinfo(host,  port)[0][-1]
    msg = '\x1b' + 47 * '\0'
    msg = msg.encode()
    TIME1970 = 2208988800 # 1970-01-01 00:00:00

    # connect to server
    client = socket.socket(AF_INET, SOCK_DGRAM)
    client.sendto(msg, address)
    msg, address = client.recvfrom(buf)
    t = ustruct.unpack("!12I", msg)[10]
    t -= TIME1970
    tuple_time = utime.localtime(t)
    rtc.init((tuple_time))
    client.close()

rtc = RTC()
print(rtc.now())
getNTPTime()
print(rtc.now())

it never print second line :(
no error no print

daniel

@livius can you share a minimal code example that reproduces your problem? Thanks!

livius

@daniel
i have minimized code but it work without sensors and do not raise OSError
i will try extending it step by step but it take time
whole code work ok without any change in 1.5.0

in the meantime you can test another issue
test this to see what happened inside
code never finished and never bring error
file ntest.py - i execute it as execfile('ntest.py')

print('start')
import machine
import socket
from socket import AF_INET, SOCK_DGRAM
from machine import RTC
from network import WLAN

print('def')
def getNTPTime(host = "pool.ntp.org"):
    port = 123
    buf = 1024
    address = socket.getaddrinfo(host,  port)[0][-1]
    msg = '\x1b' + 47 * '\0'
    msg = msg.encode()
    TIME1970 = 2208988800 # 1970-01-01 00:00:00

    # connect to server
    client = socket.socket(AF_INET, SOCK_DGRAM)
    client.sendto(msg, address)
    msg, address = client.recvfrom(buf)
    t = ustruct.unpack("!12I", msg)[10]
    t -= TIME1970
    tuple_time = utime.localtime(t)
    rtc.init((tuple_time))
    client.close()
print('adef')

wlan = WLAN(mode=WLAN.STA)
nets = wlan.scan()
#time.sleep_ms(100) 
for net in nets:
   print(str(net.ssid))

if not wlan.isconnected():
	wlan.ifconfig(config=('192.168.1.110', '255.255.255.0', '192.168.1.1', '192.168.1.1'))	
	wlan.connect('livius', auth=(WLAN.WPA2, 'secretpass'), timeout=5000)
	while not wlan.isconnected():
		machine.idle() #time.sleep_ms(10) #pass #machine.idle() # save power while waiting		
	print('WLAN connection to the router succeeded!')

rtc = RTC()
print(rtc.now())
getNTPTime()
print(rtc.now())

PCA

@daniel good job ! Bluetooth stack seems in progress : my Lopy with GATTService is now visible with Bluegiga debug tools and read/write characteristics works.
BUT the read characteristic callback never fires (worked in 1.5.1-b1)
Pascal

crankshaft

@daniel - just updated, now the project that i've been working on for the last month is broken, have not had chance to troubleshoot yet but it is unable to run the onewire / ds1820b code any longer :-(

Also, does this update address the posts that nobody has responded to regarding guru mediation crashes and the instability of the threads, interrupts and sockets as these are driving me crazy ?

What is the parameter you added mentioned here "Add parameter to enable/disable WLAN power save mode."

Can we please get a proper bug reporting / tracking system in place as I am getting very frustrated with issues that are show stoppers for me and that nobody seems interested in responding to or addressing ?

crankshaft

@daniel - I realise that this is similar to another post I made a few weeks ago, but I feel I need to vent this again.

For IOT, I committed to PYCOM about a month ago and purchased a handful of wipy2 / lopy devices, based on the understanding that the following advertised features which are essenital (and very attractive) were available right now and were not 'planned specs' that would be available 'some time in the future':

14ma with WIFI enabled
Dual Core
Threading & Interrupts

However it is now clear that this is not the case, that many are only partially working, what is working is not very stable, and that PYCOM devices are most certainly not ready for any production environment by a long way.

Unhandled Guru mediation crashes, unresponsive and dead sockets and intermittent wifi connections alone are a problem, but when you have all of them together, it is just too much, and it does seem that fixes or for these are a lesser priority than releasing new hardware ?!

It is extremely frustrating when I am eagerly waiting and expecting the next release to address problems that I have so that I can move forward only to discover that effort has only been focused on releasing things line new hardware (sipy) and new IDE plugins rather than addressing the existing problems.

It's quite clear that the team are overloaded with too many things to do and the pressure or desire to release new hardware is resulting in users such as myself who just want to use a turbo-charged WIFI enabled IOT device having to wait for all the other new bells and whistles to be completed first, and the device to become production ready, which my gut feeling tells me is probably 6 months away.

Low battery power, reliable wifi and bullet proof firmware are absolutely fundamental to a IOT micro-controller, and I would really like to know if there is any user anywhere in the world that is actually using a WIPY2 device in a real live project ??

My patience is now wearing very thin and reluctantly I am now forced to consider switching to another platform, because as far as I can tell, fixes for the issues preventing me from using these devices in a real live environment are still many, many months away.

Update
I just noticed a new web page with several new hardware announcements, I guess this means that effort is going to be further diverted into developing the firmware for these new boards making the current situation even worse :-(

livius

@crankshaft

unable to run the onewire / ds1820b code any longer

you point me in direction to dig first why my working code in 1.5.0 not work in current firmware
i suppose that OSError: [Errno 104] ECONNRESET is because i got None from onewire instead of value - but i can check this in the evening

livius

I still have objection to pycom did not you inform on your website which is already implemented and what still needs to be improved before buying.
As I have once wrote - people will buy your product but do not expect that everything is fine and dandy and beautifully.
as you can see that another criticise post from new user @crankshaft.

Informing that the device consumes this much power in a deep sleep, or so and so enabled wifi it seems to me also pure guesswork until deep sleep mode will not be available. You can indicate that the promises esp so be it.

Additional - issue tracker - github is not very good for that - could you use something like jira -
e.g. Firebird use free version of it
http://tracker.firebirdsql.org/secure/Dashboard.jspa

but summarize this
i believe in pycom team i see many, many progress but without first point you loose more and more.

Pete

@crankshaft said in New firmware release, version 1.6.0.b1:

My patience is now wearing very thin and I am now forced to consider switching to another platform, because as far as I can tell, the issues preventing me from using these devices in a real live environment are still many, many months away.

I know what you mean. I bought WiPys because I need to quickly simulate a large piece of equipment we won't have our hands on for a couple of months. I had been waiting for some specs from its supplier before I got stuck into implementation (just received them this morning), but the few things I tried already and many of the posts I'm reading have me worried that the WiPy won't be stable during our tests in the field, when we'll be relying on it working all day while we debug our actual product and can't easily do resets and suchlike on the WiPy.

If it had been advertised as an alpha/beta product I'd have approached it differently, keeping an eye on it as an interesting future possibility but not building our test plan on it.

Fortunately I'm sufficiently cynical as to have also ordered a different brand of WiFi IoT module as a backup. This one runs a full Linux which is more complexity than I really wanted (serial -> Python -> UDP-over-Wifi and nothing more, as promised by the WiPy, was ideal) but at this point I'm rather glad I have them on the shelf as well.

I'm going to stick with WiPy for now in case our simple use-case happens to dodge round all the bugs, but posts like Livius's on this thread saying that WiFi station mode (which is what we need to use) is not stable, are not encouraging.

Pete

daniel

Hello,

Apologies for the bugs introduced on the latest release. There were 2:

pin.value() was not working correctly for OPEN_DRAIN pins, and this broke the one-wire library.
Bluetooth callbacks were not being called for characteristic read/write events.

Both are fixed now. We just published firmware 1.6.1.b1 with the following changes:

esp32/mods/modbt.c: Protect against writing chars value over 20 bytes long.
esp32/util: Increase callbacks stack size to 5K.
tools: Make the certification script compatible with US915 too.
esp32/mods/modbt.c: Mark the is_char field as true on characteristics.
This ensures that the read and write callbacks are actually called.
esp32/mods: Fix bug introduced when reading the value of an output pin.

Simply run the updater tool again and the new firmware will be flashed on your boards.

Cheers,
Daniel

livius

@daniel
i can confirm that problem was not with wifi
only pin.value() mentioned by you

i also fix my software to prevent such situation like None returned instead of value (maybe never happen again but...)
None value was send by wipy by wifi and i automatically drop connection on server side because of wrong packets sended and i then i see OSError: [Errno 104] ECONNRESET which was ok

now after your fixes may software work back ok
@Pete you can test now - it looks promising :)

after few minutes of testing i see only one thing not work
but all others are fixed - i see if it will be working tomorrow through night :)

@daniel can you test my sample from post below with getNTPTime
it still doesn't work

Neo

I am working with my colleagues on these products since 2 month almost... and I reinforce what @cranshaft is saying above.

Definitely, the products are not ready for real deployments... and we tried to slope between issues getting FRUSTRATION TO MAXIMUM.

The fact that from release to release, the basic issues are not resolved ... and more regressions are coming... going to an worse situation. I don't know what to say, might be pressure on workload/qualified manpower ... or we are dealing with a bad product development management that is focusing on new features

Here is a brief:

1.6.1b1 release, if yourscript is big (eats memory) the system is hanging on gc.collect()/gc.mem_free() .. huh this is CRITICAL
There is no way to compile modules as frozen byte - and this is solving the memory limitation
Release 1.30 - the OneWire code that was functional ... from 1.5.x is not working anymore.
I2C with some devices (like BMP180 or ADC1115) is not working. Please note that SAME devices are working with Arduino/PI/OrangePI/MCU
DHT22 can not be implemented due to sleep ms issue. You need to go at C level to make it work... not a chance on mpython.

... and more others that we got if you expose the product on low temp, or run it over days etc.

Guys, my advice is to take a breath from this race and focus on basic stuff .. then go and develop other new features. Take it slow and focus on QUALITY.... doing a lot of stress tests.

A bad reputation or negative web article, will not push you between champions. Doesn't matter if you have SiPy, LoPy or whatever Py ... if the reputation of the products are bad.

livius

@Neo
I do not want to defend PYCOM but you mentioned inoperative few things
which work for me in current firmware (1.6.1):

Onewire work for me - i read 3 x ds18b20 at the same time
BM180 work also for me (but i have some problems previously and do not test if i still must read first something from i2c e.g. light sensor)
I have many others sensors working but
I forgot about this sensor(DHT11) - because it was so inaccurate - but yes there are problems with timing - you remind me about it - I have to go back to it ;-) and when succed i will try dht22
I fight with color tft (some progress but..) but i wait until pycom merge github with official micropython

daniel

@Neo, thanks for your feedback. Let's go one by one:

Can you share a minimal example that reproduces this issue? Or is it only with big scripts?
Adding this option to MicroPython is easy. Then we have to write a guideline on how to do it for the ESP32 port. We have been working on this but is not finished yet.
One-wire is working on the latest release as @livius just confirmed. Are you sure it still doesn't work for you with 1.6.1.b1?
Are these devices I2C devices (BMP180 or ADC1115) requiring a really fast clock? Exactly what happens on the I2C bus? Are you getting exceptions somewhere?
If sleep_ms is not accurate enough, could it be done with sleep_us. If you really need this in C, share with us a C driver, and we can make it part of the C firmware and expose a Python API. Would that work?

Cheers,
Daniel

daniel

@livius I have reproduced your issue with NTP and I have fixed it just now. The problem was caused by the wlan.ifconfig() line. It turns out that Espressif fixed the issue with setting the DNS server that we have been working around for months. This is good but it made our work around break internet connectivity. I have tested the patch in 2 different ways:

Setting the static IP after connecting to the AP:

if not wlan.isconnected():
    wlan.connect('my-ap', auth=(None, '123456abc'), timeout=5000)
    while not wlan.isconnected():
        machine.idle() 
    print('WLAN connection to the router succeeded!')
    wlan.ifconfig(config=('172.20.10.8', '255.255.255.240', '172.20.10.1', '8.8.8.8'))

and Setting the static IP before connecting to the AP:

if not wlan.isconnected():
    wlan.ifconfig(config=('172.20.10.8', '255.255.255.240', '172.20.10.1', '8.8.8.8'))
    wlan.connect('my-ap', auth=(None, '123456abc'), timeout=5000)
    while not wlan.isconnected():
        machine.idle()
    print('WLAN connection to the router succeeded!')

In both cases setting the static IP works and there's internet connectivity. We'll release this along with other fixes tomorrow.

Cheers,
Daniel

crankshaft

@daniel - I have just upgraded again and onewire is now working, but I am still getting the same wlan error mesaages:

W (240989) wifi: post pm rx bcn failed
W (240990) wifi: post pm rx bcn failed

Questions:

How can I disable these debug messages as they fill up my screen so quickly and I am unable to read my own debug data as it gets pushed off the top of the screen by all of the wlan 'bcn failed' messages
[ Add parameter to enable/disable WLAN power save mode ] - How can I disable / enable wlan power saving, what is the parameter ??

Thanks

robert-hh

About frozen bytecode: I managed to enable it with the following steps, copied from the eps8266 build:

In mpconfigport.h of the ESP32 directory, goto the line

#define MICROPY_MODULE_FROZEN (0)

and change it to the four lines:

#define MICROPY_PERSISTENT_CODE_LOAD (1)
#define MICROPY_MODULE_FROZEN (1)
#define MICROPY_MODULE_FROZEN_MPY (1)
#define MICROPY_QSTR_EXTRA_POOL mp_qstr_frozen_const_pool

Then, put your python code like before into the directory modules and run:
make FROZEN_MPY_DIR=modules BOARD=LOPY
The first line added allows you to run crosscompiled code from the file system (.mpy files), which are generated by mpy-cross.
You have to build mpy-cross before in the mpy-cross directory.

That worked for my sample files, totalling to 750 loc, but @shaun had problems to replictate these steps. Maybe his code was too large and he would have had to adapt the linker map.

daniel

@crankshaft Power save is disabled by default now, and it's controlled by an extra parameter in the init/constructor:

https://docs.pycom.io/pycom_esp32/library/network.WLAN.html#network.wlan.init

How can I disable these debug messages as they fill up my screen so quickly and I am unable to read my own debug data as it gets pushed off the top of the screen by all of the wlan 'bcn failed' messages

This needs to be disabled internally in the IDF. Before there was not a way to do it, but now it's possible. We'll try to do it today.

Cheers,
Daniel

Explore Pybytes | Official Documentation | Report a Firmware Bug/Issue | GitHub

New firmware release, version 1.6.0.b1

Pycom on Twitter