New firmware release 1.4.0.b1

[mohpor]

@daniel

Hi,

just like @crankshaft, I have expressed my concerns on the state of things on firmware side before (hardware specs vs price is phenomenal, no doubt about it). A few weeks has passed and I regret expressing them the way I wrote them, in my defense, I was really frustrated by the fact that I couldn't do anything in the meantime, now, the firmware is getting closer to finish (very very far away at the moment, but big steps every week).
The reason I'm writing here again is that, as a veteran Software Developer, I can see the reason behind these frustrations: Issue Tracking, or the lack thereof. This forum is a great tool to discuss things, but when it comes to tracking or submitting an issue, it is not the right tool. If you provide a proper Issue Tracking service, be that your Github repo's Issues section or something as comprehensive as Jira, we would have a better understanding of what's missing, what is working, the progress, the plan ahead and many many more advantages that I'm sure you are aware of.
To see how important it is to provide a better Issue Tracking tool just read every Firmware update Announcement's comments! It's like a lot of issues being submitted to the post! This is not right @daniel.

Good news on Memory would make my week! I'm running into memory related issues left and right and I'm just hoping to optimize my code to get where I'm heading.

More power to you all,
Cheers,
M.

[sakis]

@mohpor

Hi,

We have an internal issue tracker but it's not exposed to the public. Public issues are available on GitHub, and they are updated (open/closed) as expected.

I agree that we should use GitHub more instead of the forum, especially for tagging and prioritising issues, but that's already happening (admittedly on a smaller scale than it should).

[mohpor]

@sakis
I'm well aware of the Github repo's existence but I guess the burden is on you guys to direct issues (and manage them to some extent) from here to there and make others engage the proper way. I will be submitting my issues there from now on.

Cheers,
M.

[livius]

@crankshaft said in New firmware release 1.4.0.b1:

You have right about overview but one point i must understand:

The amount of available ram is just too small to do anything that includes anything more than one or 2 libraries, I managed to get the fantastic bosch BMP280 enviro sensor working, and cobbled together a mqtt client, however they cannot work together due to insufficient ram, and so I have to now find an alternative sensor that is easier to decode.

i do not understand you corectly about this sentence
so I have to now find an alternative sensor that is easier to decode
do you mean that BMP280 is not usable to you because of RAM
I use BMP180 extensively (and difference between BMP280 is "only" accuracy) but i do not use "mqtt client,"
I connect to service(wirtten in Delphi) on computer and send there data throught socket. And back to Firebird database.
I suppose that your problem is about MQTT memory consumption.

I have connected to wipy many sensors at the same time e.g.
one wire temperature, i2c light, barometric, spi oled, pir, IR, few buttons, port expander
i have established connection between two wipy (i still wait for support AP+STA mode)
but now i simulate it by connecting one wipy to second and after packet of data arrive i turn my AP to sta and send data back to server throught tcp socket and back change to AP

[jmarcelino]

@crankshaft
I've been using umqtt.simple (https://github.com/micropython/micropython-lib/tree/master/umqtt.simple ) successfully with various sensors, umqtt only takes up about 7Kb. Admittedly I've not used it to do MQTT over SSL yet if that's what you're using, so it might have a larger footprint.

@mohpor
You touch on a good point and I'm also guilty of not following up quick informal reports here with proper Github issues. Everyone at Pycom is just so responsive it hasn't seemed necessary but you're right it would improve visibility and tracking.

That said following from the latest announcement I'd imagine in a couple of weeks most bug reports will really need to be filed with the main micropython repo and not on Pycom's.

[mohpor]

@jmarcelino
I'm sure it will get improved drastically if everyone follow the same repo. More power behind the tasks at hand.

Cheers,
M.

[jmarcelino]

@mohpor
Indeed it's excellent news having the the ESP32 - powered by Pycom - as an official member of the community, however the main repo is also a very noisy place. There are currently 300+ open reports there and from a quick glance most aren't labelled with the type of board despite being port specific, so if nothing there changes it could potentially be more confusing.

[mohpor]

@jmarcelino
I'm sure it'll worth the noise! I look at it this way: The more noise (activity) the more chance of getting stable! I am facing bugs everyday, imagine how long it would take for me to face all possible bugs. Now, when many others try what I'm trying, there is a good chance the time would be decimated massively! Not to mention use cases and code templates and known issues which will be documented for others to use!

The mess of not being labeled or assigned is a whole another issue though. I hope they'll get sorted out in time.

Cheers,
M.

[mohpor]

@crankshaft

The amount of available ram is just too small to do anything that includes anything more than one or 2 libraries, I managed to get the fantastic bosch BMP280 enviro sensor working, and cobbled together a mqtt client, however they cannot work together due to insufficient ram, and so I have to now find an alternative sensor that is easier to decode.

I was facing memory issues but I got to solve them. The key is gc.collect()! You can't over use it! I mean, I have gc.collect() all over the place! I'm not sure about it's implications on battery usage and performance, but that is the only way I can run my code now! Every loop iteration, every major memory altering event (especially around threads and interrupts). I have learnt it the hard way, it might help you with your application too.

A good pattern to use is try-finally in interrupt handlers and memory consuming functions:

try:
    # Do some memory consuming function
    for x in list:
       #do things
       gc.collect()
    wlan.deinit()
    gc.collect()
    ble.deinit()
    gc.collect()
finally:
    gc.collect() 

Hope it helps somebody.

Cheers,
M.

[robrush]

Hi folks,

• Still nothing about machine.sleep() or machine.deepsleep()?
• I also tried to measure the current i used by my LoPy with machine.idle() or not... I couldn't find any difference. Does someone have an idea of the current used by the CPU at 100% or close to 0%. The difference is probably too small to be seen.

[crankshaft]

@daniel - Thanks, and joining forces with Damien George sounds like a great move too :-)

[crankshaft]

@mohpor, @jmarcelino, @livius - Thanks for the suggestions regarding MQTT and BME280 and memory etc.

Compared to the Si7021 AND SHT21, the BME requires a lot more code in the driver library to extract the data, probably 3 times as much as for the S* devices, which are really simple and is what I am using now.

Other than getting data about the environment, I need to get data from other sensors, and really need to provide a simple HTTP server so that the device can initially be configured (an maintained) from a web browser and a few other things too.

My code needs to be very robust as once it is deployed (on the other side of the world) it needs to look after itself, and manage any and all errors, and with the limited code space right now I simply cannot do this.

I know that if I strip down to bare functionality, I could get the basics to work, but that's not what I am after, I need much more than I have now in order for this project to succeed.

If I were to use a 8266 device (nodeMCU or micropython) then I believe I can get it all in, and my expectations were that I would be able to do more on the ESP32 rather than less.

I don't really require any help with the programming side of things right now, just need more space to breathe :-)

[livius]

@crankshaft
you can compile py file it use then much less memory
but still this require custom firmware build with enabled option for compiled modules

look here:
https://forum.pycom.io/topic/486/v-1-8-7-micropython-released-support-importing-compiled-mpy-files-more-ram/2
and here - detile how to do this:
https://forum.pycom.io/topic/453/memoryerror-memory-allocation-failed-allocating-2048-bytes-lopy-1-2-0-b1/2

[tholden]

machine.idle is not working.

I am executing it as the first thing in the code, but it still continues execution

[robrush]

@tholden agree with you. See my comment below

[mohpor]

@sakis

Now I can see why everybody submit their issues here.

I have opened an issue on Github's repo, no one even acknowledges my issue there. People get way more attention from posting here.

[Colateral]

@livius Have you succeed to get data from BMP180 . I am always get "I2C bus error" when I am trying to instantiate bmp180.
Debuging, I2C bus scan displays 119 address available, but when I simply read the bmp180 chip id, with i2c.readfrom_mem(119, 0xD0, 2) I am getting "I2C bus error".

This is happening always on LoPy.

[livius]

@Colateral
yes i can read data from it but fix is not clear to me - i still can not diagnose what is crux of the problem.

I can not work with BMP180 alone i must have something more on bus like e.g. light sensor BH1750
if i reverse sensors initialization order i got always I2C bus error
it looks like i must write something one to i2c and then i can read from it or vice versa.
It is not connected with the resistance or the capacitance.

I do not know if i2c is now hardware or still software implementation

[jmarcelino]

@livius said in New firmware release 1.4.0.b1:

I do not know if i2c is now hardware or still software implementation

i2c is still done in software.

Is anyone using external pull-up resistors on their I2C? I see the internal ESP32 pull-ups are enabled in the code, but have not seen any reference to what the resistance value is like or attempted to measure it.

[livius]

@jmarcelino said in New firmware release 1.4.0.b1:

I tested with and without resistors but "nothing" change
only reverse order of initialization of sensors help

difference between sensors is that in light sensor first is memwrite
in bm180 first is memread

and problem appear in 1.4 version of firmware - previously all work without any problem
i do not dig what was changed on esp-idf in the mean time