LoPy/WiPy 2/ESP32 specs and performance

[rcolistete]

Some WiPy 2/LoPy specs and performance that it would be useful to know, theoretically and measured :

• free RAM (depends on firmware version);
• flash memory available for MicroPython user;
• LoRa speed and range in real user cases;
• WiFi speed and range in real user cases;
• Bluetooth speed and range in real user cases.

ESP32 (used by LoPy and WiPy 2) specs, theoretically and measured :

• Xtensa LX6 performance, using some MicroPyton benchmark code;
• ADC (12 bits, 16 ports, 8 channels) speed in ksamples/s;
• DAC (10 bits, 2 ports, 2 channels) speed in ksamples/s.

Topics about the same subject, in MicroPython forum :
LoPy specs and performance
WiPy 2/ESP32 specs and performance
Free RAM on MicroPython boards
Benchmark comparison of MicroPython boards
ADC performance of MicroPython boards

[rcolistete]

Measured specs with firmware v0.9.2.b2 (28/10/2016) :

• WiPy 2.0 : free RAM 71152 bytes; 498 KB of free internal "/flash".
• LoPy : free RAM 71152 bytes; 498 KB of free internal "/flash".

Free RAM was measured with :

import gc
gc.collect()
gc.mem_free()

Is there any command to measure the free internal flash space ? I've measured by uploading files and summing up the sizes.

[rcolistete]

With firmware v0.9.3.b2 (05/11/2016), the measured specs for :

• WiPy 2.0 : free RAM 79152 bytes; 498 KB of free internal "/flash".
• LoPy : free RAM 79152 bytes; 498 KB of free internal "/flash".

So the free RAM increased 8KB.

[rcolistete]

With firmware v0.9.4.b1 (14/11/2016), the measured specs for :

• WiPy 2.0 : free RAM 79152 bytes; 498 KB of free internal "/flash".
• LoPy : free RAM 79152 bytes; 496 KB of free internal "/flash".

So the free RAM wasn't changed from v0.9.3.b2.

[rcolistete]

Some benchmark comparison of MicroPython boards :

1. pystone_lomem (with '%g' changed to '%u' in lines 69 and 71 to be compatible with WiPy 1.0, download here), in pystones/second :

• Pyboard v1.1 with MicroPython v1.8.6 : 1,754;
• Pyboard Lite v1.0 with MicroPython v1.8.6 : 1,048;
• WiPy 1.0 with MicroPython v1.8.2-103 : 659;
• ESP8266 with MicroPython v1.8.6-7 : 223 (80 MHz) / 415 (160 MHz);
• BBC Micro:bit with MicroPython v1.7.9 : not enough RAM to run;
• LoPy with MicroPython 0.9.6.b1 : 829;
• WiPy2 with MicroPython 0.9.6.b1 : 836;
• Teensy 3.2 with MicroPython v1.8.6 : 846;
• Teensy 3.5 with MicroPython v1.8.6 : 1086;
• Teensy 3.6 with MicroPython v1.8.6 : 2212;

2. PerformanceTest using 'time.ticks_ms' (download here versions with 'pyb.millis', 'time.ticks_ms' and 'microbit.running_time()'), in number of times in 10s :

• Pyboard v1.1 with MicroPython v1.8.6, using 'pyb.millis' : 2,815,970 / 4,610,390 (native) / 8,771,699 (viper);
• Pyboard v1.1 with MicroPython v1.8.6 : 2,783,122;
• Pyboard Lite v1.0 with MicroPython v1.8.6, using 'pyb.millis' : 1,738,503 / 2,686,262 (native) / 5,392,659 (viper);
• Pyboard Lite v1.0 with MicroPython v1.8.6 : 1,732,370;
• WiPy 1.0 with MicroPython v1.8.2-103 : 918,031;
• ESP8266 with MicroPython v1.8.6-7 : 171,145 (80 MHz) / 341,224 (160 MHz);
• BBC Micro:bit with MicroPython v1.7.9 : 168,826;
• LoPy with MicroPython 0.9.6.b1 : 651,394;
• WiPy2 with MicroPython 0.9.6.b1 : 680,488;
• Teensy 3.2 with MicroPython v1.8.6 : 1,041,445;
• Teensy 3.5 with MicroPython v1.8.6 : 1,204,193
• Teensy 3.2 with MicroPython v1.8.6 : 3,451,931;

3. hsquare.py (type Ia supernova calculation with float point for 50,000 points), mean time for each calculation in us :

• Pyboard v1.1 with MicroPython v1.8.6 : 83;
• Pyboard Lite v1.0 with MicroPython v1.8.6 : 139;
• WiPy 1.0 with MicroPython v1.8.2-103 : - (no float point support);
• ESP8266 with MicroPython v1.8.6-7 : 460 (80 MHz) / 239 (160 MHz);
• BBC Micro:bit with MicroPython v1.7.9 : 1,589;
• LoPy with MicroPython 0.9.6.b1 : 107;
• WiPy2 with MicroPython 0.9.6.b1 : 143;
• Teensy 3.2 with MicroPython v1.8.6 : 278;
• Teensy 3.5 with MicroPython v1.8.6 : 138
• Teensy 3.2 with MicroPython v1.8.6 : 73.

MicroPython release 0.9.6.b1 (for LoPy and WiPy 2) has default ESP32 frequency set as 160 MHz, instead of 80 MHz as before.

With the last releases, LoPy and WiPy 2 are a lot faster than before :

1. pystone_lomem, in pystones/second :

• LoPy with MicroPython 0.9.4.b1 : 321;
• WiPy 2 with MicroPython 0.9.4.b1 : 404;
• LoPy with MicroPython 0.9.5.b1 : 354;
• WiPy 2 with MicroPython 0.9.5.b1 : 388;
• LoPy with MicroPython 0.9.6.b1 : 829;
• WiPy 2 with MicroPython 0.9.6.b1 : 836;

2. PerformanceTest, in number of times in 10s :

• LoPy with MicroPython 0.9.4.b1 : 91,810;
• WiPy 2 with MicroPython 0.9.4.b1 : 281,083;
• LoPy with MicroPython 0.9.5.b1 : 293,032;
• WiPy 2 with MicroPython 0.9.5.b1 : 260,478;
• LoPy with MicroPython 0.9.6.b1 : 651,394;
• WiPy 2 with MicroPython 0.9.6.b1 : 680,488;

3. hsquare, mean time for each calculation in us :

• LoPy with MicroPython 0.9.4.b1 : 542;
• WiPy 2 with MicroPython 0.9.4.b1 : 184;
• LoPy with MicroPython 0.9.5.b1 : 167;
• WiPy 2 with MicroPython 0.9.5.b1 : 191;
• LoPy with MicroPython 0.9.6.b1 : 107.
• WiPy 2 with MicroPython 0.9.6.b1 : 142.

Performance improvements for LoPy and WiPy 2 are expected with new firmware releases.

(20/11/2016 : added BBC Micro:bit)
(21/11/2016 : added Teensy 3.x)
(27/11/2016 : updated LoPy and WiPy 2 with firmware 0.9.6.b1)

[betabrain]

When will MicroPython get native/viper support for ESP32? Obviously, this would make the benchmarks a lot more interesting.

[rcolistete]

@betabrain Good question. But in LoPy Kickstarter project, I don't see any mention about native or viper decorator.
The MicroPython for ESP8266 project lists the implementation of @micropython.native, @micropython.viper and @micropython.asm_xtensa decorators, but not yet implemented.

[rcolistete]

ADC performance tests of MicroPython boards :

1. 'ADC_timedtimer_Pyboard.py' as only Pyboard has "ADC.read.timed()" method to ADC read burst into buffer in high speed. Test with 20 thousand readings, mean speed in ksample/s :

• Pyboard v1.1 (12 bits ADC) with MicroPython v1.8.6 : 1,604;
• Pyboard Lite v1.0 (12 bits) with MicroPython v1.8.6 : 899.6;

2. 'ADC_loop' (download here for different boards) with 20 thousand readings, mean speed in ksample/s :

• Pyboard v1.1 (12 bits ADC) with MicroPython v1.8.6 : 34.9;
• Pyboard Lite v1.0 (12 bits) with MicroPython v1.8.6 : 20.1;
• WiPy 1.0 (12 bits) with MicroPython v1.8.2-103 : 62.5;
• ESP8266 (10 bits) with MicroPython v1.8.6-7 : 5.44 (80 MHz) / 7.34 (160 MHz);
• BBC Micro:bit (10 bits) with MicroPython v1.7.9 : 6.51;
• LoPy (12* bits) with MicroPython 0.9.6.b1 : 21.2.
• WiPy2 (12* bits) with MicroPython 0.9.6.b1 : 21.3.
• Teensy 3.2 (16 bits) with MicroPython v1.8.6 : 59.6;
• Teensy 3.5 (16 bits) with MicroPython v1.8.6 : 62.3;
• Teensy 3.6 (16 bits) with MicroPython v1.8.6 : 107.

(*) : this firmware is limiting the ADC to only 10 bits, 0-1V.
For example, "ADC_loop_LoPyWiPy2.py" :

# Reads the LoPy/WiPy2 ADC and measures the time and speed to read 20000 samples.
import machine
import time
def ADCloopBenchmark():
    adc = machine.ADC(0)
    adcread = adc.channel(pin='P13')
    t1 = time.ticks_ms()
    for i in range(20000):
        val = adcread()
    t2 = time.ticks_ms()
    print("20000 ADC readings done after %u ms." %(t2-t1))
    print("Mean time for each ADC reading = %15.13f us" % ((t2-t1)/20.0))
    print("ADC reading = %15.13f ksamples/s" % ((20*10**3)/(t2-t1)))
ADCloopBenchmark()

Pyboard (v1.1) is the champion MicroPython board, with 1.6 Msamples/s for ADC burst reading ! Followed by Pyboard Lite with 0.9 Msamples/s in this mode.

For continuously ADC reading inside a loop, Teensy board and WiPy 1.0 are the fastest, followed by the Pyboard's. BBC Micro:bit surprised me, as its MCU is nRF51822 with ARM Cortex-M0 @ 16MHz.

With the last firmware releases, LoPy and WiPy 2 ADC is a lot faster than before.
'ADC_loop', mean speed in ksample/s :

• LoPy with MicroPython 0.9.4.b1 : 3.65;
• WiPy 2 with MicroPython 0.9.4.b1 : 13.8;
• LoPy with MicroPython 0.9.5.b1 : 13.0;
• WiPy 2 with MicroPython 0.9.5.b1 : 12.3;
• LoPy with MicroPython 0.9.6.b1 : 21.2;
• WiPy 2 with MicroPython 0.9.6.b1 : 21.3.

Performance improvements for LoPy and WiPy 2 are expected with new firmware releases, including in ADC reading speed.

(27/11/2016) : ADC benchmarks for releases 0.9.5.b1 and 0.9.6.b1.

[rcolistete]

With firmware v0.9.5.b1 (19/11/2016), the measured specs for :

• WiPy 2.0 : free RAM 79152 bytes; 499 KB of free internal "/flash";
• LoPy : free RAM 79152 bytes; 497 KB of free internal "/flash".

With firmware v0.9.6.b1 (25/11/2016), the measured specs for :

• WiPy 2.0 : free RAM 79152 bytes; 498 KB of free internal "/flash"..
• LoPy : free RAM 79152 bytes; 497 KB of free internal "/flash"..

So the free RAM wasn't changed since v0.9.3.b2.

Updated the benchmarks below with v0.9.5.b1 and v0.9.6.b1 releases.