Q&A on MicroPython Multi-threading and Garbage Collector

[Bettina]

As we completed the MicroPython Multi-Threading and Garbage collector features with Damien George we sat down with our CTO Daniel to take a look at what that means for the use of the scripting language. Here’s the resulting Q&A.

Q: Why did you decide to fund Multi-Threading and a re-entrant Garbage Collector?
A: MicroPython is already feature packed and 99% equivalent to the CPython implementation of Python 3 in most of its aspects. Apart from giving you the expected built-in functionality features like tuples, list, dictionaries, strings, byte objects, floating point and complex math, a file system, BSD sockets and an interactive prompt, it also provides a simple, yet powerful hardware API that let’s you access all the peripherals of a microcontroller in a very elegant way. Handling analog to digital converters, serial interfaces like SPI and I2C, or even GPIO pins are not common things that you’ll do on a PC, but those are one of the main tasks performed by an embedded system, and there’s therefore a need to provide a flexible API for those features in MicroPython.
The peripherals in a microcontroller involve mostly input/output operations, which take time to execute, and more often than not, block until they are complete. When operations block, then the need for multiple “threads” of execution arises to allow for other tasks to run while others are blocked and waiting for an I/O transaction to finish. There are other ways to work around the blocking I/O problem, like using an event loop with cooperative tasks, but such mechanisms are not very natural for most software developers, and doesn’t really fit well into the Python programming model.

Q: What are those features useful for?
A: A typical IoT application goes like this:

1. A device connects to the Internet via WiFi,
2. it collects data from sensors and sends it to a server in the cloud.
   Most likely,
3. it’ll also receive data and control commands from the server to adjust it’s behavior based on certain parameters. In some cases, it could also have a user interface (switches, LEDs, a display, etc.)

Things will seem to work until something happens, for instance the WiFi connection is lost and the device needs to re-connect. During the seconds the WiFi connection is being re-established, nothing else happens, no more data comes from the sensors, and the user interface doesn’t respond, and that’s no good. Maybe some of the sensors are slow, which means that when reading data, the communication with the server may stall. Remember those times when your PC application crashes and your mouse stops moving for several seconds? That’s what happens here too.

The solution is to have for instance 3 threads. One thread that handles the WiFi and the Internet connection. Another thread takes data from the sensors, and the third one controls the application logic and the user interface. The threads communicate via queues and semaphores which let them share data and stay in sync.
That way no thread waits for the other and the application can work more seamlessly without interruption.

Q: What can you do on the WiPy/LoPy that you couldn’t do before?
A: You can use blocking sockets, for instance. You can divide the tasks of your application into several threads of execution with different priorities. To do this you use the _thread module which over the most low-level features, or the threading one. Here’s some example code:

import urllib2
from threading import Thread
def url_test():
for i in range (10):
page = urllib2.urlopen(“www.pycom.io”)
for i in range(10):
t = Thread(target = url_test)
t.start()

Q: Will Pycom fund other features of MicroPython?
A: Yes, that’s very likely. We have great ambitions for our products and MicroPython is one of the enablers for fast IoT applications which is one of our core promises. Plus we like giving stuff back to our growing community of developers and funding future features is a way of achieving that.

Q: In your opinion what should be the next thing MicroPython should have developed as features?
A: The next feature should be the ability to run compiled byte code directly from the flash memory of the microcontroller. This will have the advantage of being able to run literally infinite Python code while leaving the entire RAM space for the data of the application thus speeding things up.

[tttadam]

I am just started to dig into to threading, but my understanding is that pycom only support _thread, which has limited functionality compared to threading. Like lack of .start or .stop. Is my understanding false? Cause the sample code is not working on my wipy.

[daniel]

Does that mean standard MicroPython will also have multithreading?

It already does. The feature is part of the core. The unix and the cc3200 ports have it implemented as well.

[betabrain]

Interesting. Looks like PyCom managed to influence MicroPython more than some discussions on their Forum and Github.

• Does that mean standard MicroPython will also have multithreading?

• Is there a list of features you intend to fund?

• Can you add tail-call optimisation to that list? ;-)