Compression lib

[this.wiederkehr]

Is there a compression library for micropython which can do stream compression?

This would be especially useful when transferring large amount of data over wlan. As ram is limited it would have to be a stream compression library.

does any body know of something like this?

I see that upstream has a lib uzlib https://docs.micropython.org/en/latest/pyboard/library/uzlib.html which supports decompression. However compression seems not supported.

I think miniz is included in esp-idf (https://github.com/espressif/esp-idf/blob/master/components/esp32/include/rom/miniz.h )so at best there would be an interface to this mirroring the cpython api (ttps://docs.python.org/3/library/zlib.html).

[Iplooky]

@this-wiederkehr I am also interested in this. I have short test messages that are sent and the next line that contains useful (large) data is completly ignored. Did you find something useful ?

[this.wiederkehr]

No sorry. Nothing useful. However here is the upstream thread on the implementation of the uzlib:

https://github.com/micropython/micropython/issues/741

Not sure if there is a blocker with respect to compression ...

[campbellsinclair]

This post is deleted!

[Shriya dhar]

I am facing a challenge when attempting to decompress data compressed using the miniz library on the ESP32 platform. My objective is to compress data using miniz library in flasher_stub compoment of esp-idf and then decompress the data on the cloud side using some python script that can be compatible with miniz decompression style like zlib.

As per the documentation and information available, the miniz library is described as a "lossless, high-performance data compression library" that implements the zlib (RFC 1950) and Deflate (RFC 1951) compressed data format specification standards. However, I am encountering difficulties when attempting to decompress the compressed data using Python.

Specifically, when using the Python zlib module, which should be compatible with the zlib and Deflate formats, I am receiving an "unknown compression method" error. I have verified that the file and data are correct, and they were compressed using the miniz library on the ESP32. Despite this, I am unable to successfully decompress the data.

I would greatly appreciate any help or any insights that you can provide regarding how to properly decompress data that has been compressed with the miniz library using Python.