Details on different Rust implementations of deflate?

Is there somewhere an analysis of the different Rust implementations of DEFLATE? I'm particularly interested in the decoding part, for miniz_oxide, libflate and inflate. For example, which algorithms they chose for building the Huffman tree/table(s) and decoding symbols (corresponding research paper if exist). I'm new to this and so diving into the code is quite difficult (just spent two days reading specs and trying understand implementation details of these libs without much success).

Background story

For some computer vision tasks, I'm reading data from PNG files, using the png crate. When porting to wasm the code, I realized that the PNG reading was taking way too much time (0.1s for one 640x480 image) to have anything realtime possible. Turns out the issue seems to be related to inflating the png data. So out of curiosity, I was wondering if I could try writing a PNG decoder based on the new 5.0 version of the nom parser. I've managed to parse PNG uncompressed chunks but the struggle started when I tried decoding the deflated chunks, such as zTXt and IDAT. I've had issues reading bits in the correct order to decode symbols.

For the time being, I think I'll just use miniz_oxide that seems to have great performances, but I'm still interested in reading an analysis of those algorithms if that exists since I came this long ^^.

3 Likes

TL;DR: use miniz_oxide; libflate is not 100% safe code and inflate is impractically slow.

I've spent a while doing a security audit of all three, optimized inflate and libflate a bit, purged a lot of unsafe code from libflate, and even found potential security issues in all three. I've also fuzzed all three in addition to a manual audit to find the panics. So they're all in reasonably good shape security-wise.

miniz_oxide is by far the fastest one. miniz_oxide is 2.5x faster than libflate and 4x or so faster than inflate. It's even faster than the original C implementation.

Compatibility is hard to quantify. If you could scrape the Internet for compressed data like gzip files or PNGs and report bugs against libraries that decode them differently than zlib, that'd be great.

12 Likes

Thank you for the performance and security insight! Glad I made the miniz_oxide choice.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.