Efficiency of Flash vs NVRAM (and some NVS questions)

[dbrgn]

I did some efficiency tests regarding write/read of values in the flash memory vs the NVS.

The NVS is more efficient by about three orders of magnitude.

For both memory types, I did 10 write and read cycles. Flash takes ~370ms and ~20uWh per cycle, while NVS (that tiny peak up there) takes ~1ms and ~30nWh per cycle.

Here's the test script:

def test_flash():
    for _ in range(10):
        with open('file.dat', 'wb') as f:
            f.write(b'7\x13\x00\x00')
        with open('file.dat', 'rb') as f:
            assert f.read() == b'7\x13\x00\x00'


def test_nvs():
    for _ in range(10):
        pycom.nvs_set('test', 0x1337)
        assert pycom.nvs_get('test') == 0x1337

(The flash variant writes 4 bytes, since pycom.nvs_set only supports 32 bit integers.)

Some questions arise from this:

• There's probably nothing we can do to make the flash access more efficient, right?
• Will there be functions to read/write arbitrary bytes? Currently only int32 are supported.
• How large is the NVRAM? (see also)
• What happens when trying to write a value while the NVRAM is already full?
• Is there a way to delete values from NVRAM again?

[jcaron]

@dbrgn The library may also have a local cache, which would be used for reading, and could compare values against the cache before actually writing to flash.

There could also be some way to write only a few bytes rather than full sectors like a filesystem would do, not sure if that's actually possible with the flash interface used.

[dbrgn]

@this-wiederkehr said in Efficiency of Flash vs NVRAM (and some NVS questions):

You are wrong. See here:
https://esp-idf.readthedocs.io/en/latest/api-reference/storage/nvs_flash.html#

NVS is just a key-value store stored on the internal flash which takes care to not wear out the flash to fast ...

Oh, interesting. I thought that it was a different type of memory.

So what would explain the big difference in power consumption? Just the overhead of the file system?

[this.wiederkehr]

@dbrgn said in Efficiency of Flash vs NVRAM (and some NVS questions):

No, the nvs functions write to NVRAM.

You are wrong. See here:
https://esp-idf.readthedocs.io/en/latest/api-reference/storage/nvs_flash.html#

NVS is just a key-value store stored on the internal flash which takes care to not wear out the flash to fast ...

[dbrgn]

No, the nvs functions write to NVRAM. I think that's backed by different hardware than the regular flash memory.

Regarding the error handling of the nvs_set function, I looked it up in the firmware:

STATIC mp_obj_t mod_pycom_nvs_set (mp_obj_t _key, mp_obj_t _value) {
    const char *key = mp_obj_str_get_str(_key);
    uint32_t value = mp_obj_get_int_truncated(_value);

    if (ESP_OK == nvs_set_u32(pycom_nvs_handle, key, value)) {
        nvs_commit(pycom_nvs_handle);
    }
    return mp_const_none;                                                           
}
STATIC MP_DEFINE_CONST_FUN_OBJ_2(mod_pycom_nvs_set_obj, mod_pycom_nvs_set);

Looks like None is returned if there is any error. That can happen if the flash is full, or if one of many other errors happen, according to the ESP32 docs.

What's also interesting is the nvs_get function:

STATIC mp_obj_t mod_pycom_nvs_get (mp_obj_t _key) {
    const char *key = mp_obj_str_get_str(_key);
    uint32_t value;

    if (ESP_ERR_NVS_NOT_FOUND == nvs_get_u32(pycom_nvs_handle, key, &value)) {
        // initialize the value to 0
        value = 0;
        if (ESP_OK == nvs_set_u32(pycom_nvs_handle, key, value)) {
            nvs_commit(pycom_nvs_handle);
        }
    }
    return mp_obj_new_int(value);
}
STATIC MP_DEFINE_CONST_FUN_OBJ_1(mod_pycom_nvs_get_obj, mod_pycom_nvs_get);

If a value is not found, then it's initialized to 0, and 0 is returned. Judging by this, when reading a value there is no way to determine whether 0 was stored previously, or whether it was simply not initialized.

Furthermore, every read results in a write if the value did not exist. So it seems possible to fill up NVRAM by reading many non-existing values.

And since the nvs_erase_key function is not exposed through MicroPython, it doesn't seem possible to delete any values. I wonder whether LoRaWAN still works in that case, since it also reads/writes to/from NVRAM. I don't dare to try it... :/

[jcaron]

@dbrgn I’m surprised, as I was convinced that the nvs_* methods were actually using flash underneath.

Have you tested writing a different value each time to see if that makes any difference?