Development status update
-
Dear all,
As promised yesterday, here are some further clarifications and comments around our past Friday firmware update and the next steps we are planning. Clearly some frustration, anger and reports among some of you including comments that only “30% of the features are currently available” on the firmware. I therefore wanted to give you a detailed overview as to exactly where we are.
At present we have the following functionality implemented in our firmware:
- Serial ports (UART class)
- GPIO Pins (Pin class)
- SPI
- I2C
- WLAN with AP and STA modes
- LoRaWAN and raw LoRa
- ADC
- DAC
- PWM
- FTP and Telnet servers
- Sockets over TCP/UDP and LoRa
- A bootloader that can select between the factory firmware and 2 OTA updates
- SSL and TLS up to version 1.2
- Multi-threading
- Bluetooth and WiFi co-existence. IMPORTANT POINT: We have in fact found an issue with the Updater Tool used for flashing. This will be rectified tomorrow AM and a new firmware version released first thing.
- Hashing (MD5, SHA1, SHA1, SHA224, SHA256, SHA256, SHA512)
- Encryption (AES256)
- A FAT16-FAT32 compatible file system
We are aware that there are still improvements to be made on some of the above features. We are also aware that the documentation needs improvements in several areas and we are already working to address that.
So that you may understand what goes on in the background, here’s a little information as to how we receive new ESP32 IDF features and updates. It may help make clear how the IDF development process affects our ability to develop and deliver new features and why it is difficult to accurately estimate times for what’s included in our firmware updates. The fact of the matter is that the ESP32 IDF is still work-in-progress with regularly running updates that can affect all or small parts of the Pycom firmware.
IDF changes are not always incremental when Espressif performs an update. Often they are incompatible with the previous IDF versions, which means that small or large chunks of the software that's based on it, break in many places. As you probably know, finding each affected area can be tricky and time-consuming.
Here’s an example from our latest firmware. Our goal for the latest release was to have an almost-complete BLE class, add the interrupts, the Timers and the RTC. Unfortunately, we had to spend the majority of the time set aside solving a number of memory issues introduced by the BLE/WiFi co-existence. This was in addition to getting the bootloader, the GPIO and UART interrupts (that we are using internally) working again. All were broken after updating to the latest state of the IDF.
Having done all that, we then found bugs on the WiFi stack which we solved and then had to update the IDF again and un-break things resulting from that process.
The result: the 2 weeks that we had set aside to bring new functionality were almost gone, which unfortunately left us with little time to work on Bluetooth. The good news though is that now that we have a solid IDF from Espressif to work on, all our future updates will be streamlined and no need to spend further development time fixing stuff that works.
So, the features that we still have to implement include:
- The rest of Bluetooth with connect, subscribe, publish, and advertisement transmission.
- RTC
- SDCard
- Interrupts
- Sleep modes
- Hardware timers
Please note that the SDcard driver is still not available in the IDF although Espressif are looking to get that to us by end of this coming week.
The next releases will focus primarily on adding more functionality to the BLE class, the Timers and the interrupts. The current planning is the following:
December 22:
API to receive the complete advertisement data from BLE devices, connect and get the services and characteristics.December 29:
BLE characteristics write, read and subscribe.January 5:
Hardware timers, interrupts.January 12:
Sleep modes and SD card.January 19:
RTC and WiFi/BLE disabling.For good reason, the above dates may be subject to change (we think we’ve already explained why that might be). We will however, try to make our progress reports more frequent to make sure you can track how we are getting on.
Please also note that from next week releases will be out on Thursdays instead of Fridays.
Finally, we know that some of you have tight deadlines with projects that are based on the LoPy or the WiPy 2.0. If you do and have any special requirements or features that need to be done earlier, please email to: daniel (at) pycom.io and we will try to address your specific requests as best as we can.
Thank you for your support.
Best wishes
Daniel
-
@daniel
please add to the list (may be optional list) but to not forget about:- Wifi AP+STA
- encryption of python scripts to protect code
https://forum.pycom.io/topic/309/two-questions-to-pycom-protect-code-and-delivery-of-fipy/3
-
@mohpor yes we are :-). Have you seen this post: https://forum.pycom.io/topic/420/firmware-release-1-1-0-b1?
-
@jose1711 yes, but we are still waiting for the Espressif IDF to support it. It should come around mid Januray.
-
This post is deleted!
-
hmmm, where's the plan for 1 km wifi range?
-
@livius said in Development status update:
@daniel said in Development status update:
It might not show on every firmware release, but we are working super hard to give you all the best results in the shortest amount of time.
Two mistakes here:
- Your biggest mistake was not showing above detailed information from first post in this topic as a starting info on the forum 2 month ago.
Especially this information:
IDF changes are not always incremental when Espressif performs an update. Often they are incompatible with the previous IDF versions, which means that small or large chunks of the software that's based on it, break in many places.i am desktop software developer and form that perspective this key information (essential).
I know how hard is sometimes is to write automatic tests in software and i suppose automatic tests for hardware are harder to implement. Once tested feature must be reviewed, and reviewed and reviewed...- In the shop before buy should be link to:
https://docs.pycom.io/pycom_esp32/pycom_esp32/underhood.html
and remove the link when supported all the main features.
People have this information, would continue to buy this unique board - believe me, but the frustration was close to 0.
Couldn't agree more!
- Your biggest mistake was not showing above detailed information from first post in this topic as a starting info on the forum 2 month ago.
-
@daniel said in Development status update:
It might not show on every firmware release, but we are working super hard to give you all the best results in the shortest amount of time.
Two mistakes here:
- Your biggest mistake was not showing above detailed information from first post in this topic as a starting info on the forum 2 month ago.
Especially this information:
IDF changes are not always incremental when Espressif performs an update. Often they are incompatible with the previous IDF versions, which means that small or large chunks of the software that's based on it, break in many places.i am desktop software developer and form that perspective this key information (essential).
I know how hard sometimes is to write automatic tests in software and i suppose automatic tests for hardware are harder to implement. Once tested feature must be reviewed, and reviewed and reviewed...- In the shop before buy should be link to:
https://docs.pycom.io/pycom_esp32/pycom_esp32/underhood.html
and remove the link when supported all the main features.
People have this information, would continue to buy this unique board - believe me, but the frustration was close to 0.
- Your biggest mistake was not showing above detailed information from first post in this topic as a starting info on the forum 2 month ago.
-
@mohpor thanks for your feedback. I do understand your point of view. Thanks for putting your trust in us. It might not show on every firmware release, but we are working super hard to give you all the best results in the shortest amount of time.
Cheers,
Daniel
-
Hi @livius,
Regarding the connection between 2 WiPy's, please have a look at my reply to you in: https://forum.pycom.io/topic/376/interim-update/6
The problem is just that you are using the wrong password.
Cheers,
Daniel
-
Now this is much more informative. Especially
IDF changes are not always incremental when Espressif performs an update.
Sadly that RTC is the last point. But i know that someone priority is different then other person.Please also note that from next week releases will be out on Thursdays instead of Fridays.
this is also good step - you know that if you post new firmware in Friday and something go wrong - you can not rest in weekend ;-)
-
@daniel
I personally thank you a lot for this detailed report of status, it really helped me in a way. I mean when we don't have features implemented yet, the next best thing is to know When we're going to get them and your report helped me get a perspective on the plan and the time (approximately) I can get my hands on the features I need. All being said, I can't express enough how important it is to keep the documentation in sync with the development.P.S.: I can see that you are still referring to my 30% estimation of job done. Although it might be a rough and harsh estimate, as my experience goes (I have been a project manager for years now) it is pretty much correct. Because DONE has a definition and what you have listed is not DONE until they have proper test cases, documentation, sample code , etc... I do have the best of hopes for your products and company and I am very much rooting for you to succeed and help us succeed too, so don't take me the wrong way. What I've said doesn't change the fact that you are doing an awesome job.
Best of lucks
M.
