Porting mozjpeg to Rust ($10k gig)

help
1

Currently, there's no way to decode jpegs in Rust while also handling errors gracefully - libjpeg-turbo and mozjpeg both use a setjmp/longjmp-based error system that crashes the process.

I really need a safe, fast, async-friendly (incremental) jpeg decoder for Rust.

Full parity with mozjpeg would make it incredibly useful https://github.com/mozilla/mozjpeg

I can offer $6,000 USD for the decoder and another $4,000 for the encoder. Performance would need to be within 20% of mozjpeg, and it would need to successfully decode the same range of files. SIMD acceleration would be wanted for x86_64.

Result to be Apache 2/MIT licensed, or with libjpeg's license if code is ported as-is.

I realize $10k USD is not a lot for such a large project - hopefully some other users can pledge as well, as I think this is a common pain point.

28 Likes
2

For porting the existing library, I'd look at Federico's Rust articles for all the work done on librsvg and now starting on bzip2.

5 Likes
3

The JPEG encoder/decoder from image-rs does not fulfill your requirements? See image::jpeg - Rust and https://github.com/image-rs/image/tree/master/src/jpeg

1 Like
4

This is going to sound like a stupid question:

Why is the jpeg encoder/decoder so complicated? I thought there were very well understood algorithms.

Are the encoding/decoding algorithms themselves complicated, or is it crazy levels of bit twiddling optimizations to make the algorithms blazingly fast? (Since decoding jpeg is a common web browser operation.)

2 Likes
5

Many evolved standards (JPEG - Wikipedia), multiple compressions possible, different color schemes, huffman and quantization tables, extensions (EXIF), ... JPEG is a pita :wink: Let's stick to bmp!

2 Likes
6

Actually, image-rs uses GitHub - image-rs/jpeg-decoder: JPEG decoder written in Rust for jpeg decoding,
but it looks like that it lacks of simd support.

2 Likes
7

I would be happy to collaborate with https://github.com/kaksmet/jpeg-decoder, but I'm pretty sure it doesn't have feature or speed parity with mozjpeg. Nor do any current Rust codecs support async parsing (as most files are fetched over http, this is relevant).

3 Likes
8

So it should probably be a pull parser.

1 Like
9

Relevant question: can you recomend an extensive collection of JPEG/PNG pairs which can be used for testing decoders? I think we will need it for verifying correctness of implementation, measuring feature parity, and comparative benchmarking.

4 Likes
10

Check out the ImageSharp jpeg test suite:
https://github.com/SixLabors/ImageSharp/tree/master/tests/Images/Input/Jpg

Collected by the community posting issues, it contains many kinds of degenerate cases. We are validating our output against libjpeg-turbo.

10 Likes
11

I have started working on a parser based on nom for another project I'm working on. As far as I can tell, for my use case the parser is complete. I'm busy working on the decoder but because of other projects, development is currently on hold.

There is no separate library for the parser and decoder yet. But I'd be very happy to refactor the code to make a separate library and open-source everything. I'm a beginner in Rust and do the project for educational purposes. I wanted to open-source the code in the end anyway.

I'm on mobile now, so cannot give further information. If you're interested I'd be very happy to work on this together! So please ping me in that case. :slight_smile:

1 Like
12

I feel some responsibility for your plight, because I think Rust should be able to interoperate with existing libraries that use standard non-local control flow constructs, and I've taken on a small part of the burden of trying to make that happen (more on that below) but sadly haven't been as diligent in making progress on that front as I hoped to be.

Are you aware of the existing discussions about Rust wrappers around mozjpeg? If not, the relevant part for your purposes is that this is undefined behavior, but in theory it should be possible to make it work reliably and in a well-defined way, and in many cases it should already work (and has been used successfully).

Here's an existing library with mozjpeg bindings: GitHub - kornelski/mozjpeg-sys: Rust bindings for mozjpeg

If you're interested, here is the discussion about making its behavior well-defined: Unwinding through FFI after Rust 1.33 - language design - Rust Internals

And here is my effort to make an RFC based on that discussion: https://github.com/rust-lang/rfcs/pull/2699

3 Likes
13

Thank you. I'm currently using mozjpeg-sys as well as a C shim to the libjpeg API, but even with the error handler in C the setjmp/longjmp behavior leads to horrible-to-debug problems.

I personally think it's time for a better (Rust) codec for jpeg decoding and encoding. I'd love to see it be fast enough for Firefox.

@niklaas I'd love to see what you have! A nom-based parser might be better for a pull design.

3 Likes
14

If I understand correctly, that's actually not the scenario @kornel said was working; Rust can't be expected to behave correctly with setjmp/longjmp except on Windows with MSVC (which is itself kind of a fluke). The scenario that should work is when the error-handling callback is in Rust and performs a true panic!, and that panic is subsequently caught in Rust code (after unwinding some C stack frames in mozjpeg itself, which I believe must be compiled using LLVM with -fexceptions for non-Windows platforms). I do not yet know, however, under what conditions rustc inserts noexcept into the LLVM IR, which could easily make that scenario ill-behaved.

(Note that my interest in getting the mozjpeg bindings to work should not be taken as evidence that I would be anything other than thrilled to see more crucial pieces of software rewritten in Rust!)

1 Like
15

In any case, this all seems kind of off-topic. We were talking about JPEG codecs, not unwinding.

:moneybag: I'll add another $4,000 for the decoder and $4,000 for the encoder. I know it's not as much, but I don't have as much money to throw around. Anybody else got anything to throw into the pot?

13 Likes
16

I don't have money to add to the pot, and I'm not interested in taking the contract.

As a neutral third party:

decoder:

It would be nice if you (as the client) could provide a test suite of jpegs/timings the decoder needs to handle. Otherwise, "Everything mozjpeg can handle" is going to imply a function by function conversion.

encoder:

Do you require bit equivalency? Again, if so, this will likely require a function by function conversion. If not, given JPEG is lossy, how do you determine is a compressed image is "good enough" ?

3 Likes
17

Does anyone know of a full specification of the jpeg format? There's something on wikipedia, but those documents are quite expensive.

18

@lilith I'm a Rust consultant who works in high-performance data storage systems. I'd be happy to try to take this on. If you want to chat directly about it, please send me a message or email.

However, for the sake of completeness, I'd like to point out that rewriting mozjpeg in Rust might not be that useful.

  1. I think that mozjpeg (and the libraries it's based on) have become a de facto specification. This is an example of Hyrum's Law, which says:

With a sufficient number of users of an API,
it does not matter what you promise in the contract:
all observable behaviors of your system
will be depended on by somebody.

(Hyrum's Law)

If mozjpeg's code -- not just its intentions -- are the specification to which we need to adhere, then our porting efforts will be error-prone. Matching bug-for-bug compatibility is hard.

The mozjpeg codebase goes back to 1994, and has many different authors and styles involved.

Furthermore, the mozjpeg family of software is robust. For example, Chromium uses libjpeg-turbo, which is an ancestor of mozjpeg.

Finally, Mozilla created mozjpeg expressly to be backwards-compatible with deployed systems. It would be a big effort to replicate that. (Although, the Mozilla Research team may be interested in porting it to pure Rust, for the sake of Firefox.)

So, if possible, we should try to use mozjpeg as-is.

  1. Which leads me to how to use mozjpeg.

If you're doing JPEG processing, then it is probably fair to say that your code will be CPU-intensive. As a result, even with Rust's async support, you will end up using it from within a thread pool. This is because a dedicated thread pool will allow you to take advantage of CPU cache locality (especially L1d and L1i, as well as L2/L3) to increase throughput, and also to reduce latency.

If you agree that you'll be using a threadpool, then perhaps we can look at the problem from a different angle.

I recommend creating a standalone mozjpeg program that accepts commands over IPC.

To pause execution after a timeslice has elapsed, you can have a timer fire in your async loop that sends SIGSTOP to the mozjpeg process. Later, you can send SIGCONT to resume JPEG processing. Or, if the child mozjpeg process is acting erratically, possibly due to bad input, you can end the process with SIGTERM or SIGKILL. In which case you can safely abort processing, then start a new mozjpeg process to handle the next request.

In conclusion, I recommend creating a standalone mozjpeg program that you can use from a threadpool via IPC, and thereby treat the situation as a noncooperative multitasking problem.

Best,
Robert
https://rwinslow.com

4 Likes
19

A separate program would be rather complex to do cross-platform, and not very useful for embedding in libraries like Imageflow (my personal use-case).

The libjpeg interface is certainly a standard, but I don't see why it wouldn't be possible to mimic it with a C shim or once Rust's unwinding situation is better.

Supporting the same input files for decoding as mozjpeg doesn't box us into having the same code. Decoding and encoding behavior differences aren't ideal but also aren't breaking changes.

1 Like
20

cloc shows the project as being massive:

---------------------------------------------------------------------------------------
Language                             files          blank        comment           code
---------------------------------------------------------------------------------------
C                                      125           8241          11606          42751
Assembly                                94           6560           6816          32491
HTML                                    58            175           1255          12515
C/C++ Header                            37           1325           3230           9061

Are you expecting a function by function conversion for $10k, and if not, how are you defining "decoder/encoder is good enough" ? (I'm not applying, just curious.)

1 Like