Hello, I have created a series of PNG images in a time series. I am interested in creating a tool, in Rust, that then converts these PNG files into an MP4 movie. I can already turn them into an animated GIF, using the wonderful Image crate, but there are limitations to this approach. Specifically, animated GIFs are limited in their colour depth and they are so much larger than the equivalent in an MP4. I've seen the MP4 crate, but it seems to be very, very low-level and there are no examples for how you would create a movie using a series of input images, or frames. I know that this type of thing can be done using online converters, such as EasyGIF, but I am interested in doing this programmatically as part of a tool that I am writing. Any advice would be greatly appreciated.
Hi,
... that then converts these PNG files into an MP4 movie
Unfortunately mp4 is a container and not a video codec format and the mentioned https://crates.io/crates/mp4 crate is not really low level. It is just designed to interact with "MPEG-4 Part 14" container files (also known as .mp4 files) which is is orthogonal to actual image to video encoding.
Specifically, animated GIFs are limited in their colour depth and they are so much larger than the equivalent in an MP4
This comes at the expense of modern video codecs being orders of magnitude more complex than the humble gif which is actually an image format.
While there exist pure rust video codec implementations like rav1e I'm guessing that encoding with AV1 is not what you are after. On the other hand the more common video codecs like H.264 or H.265 will only have rust bindings to C based implementations.
It is hard to give any suggestions as the best approach will highly depend on your usecase, constraints and design.
If I were not constrained in regard to the design, my go to solution would be nearly always to shellout to well defined version of https://www.ffmpeg.org/ binary bundled with my software for simplicity sake. Even big players are not embarrassed to do exactly just that:
Otherwise you will likely need to use one of semi-high level rust wrappers over C libraries like https://crates.io/crates/gstreamer
@budziq That is extremely insightful. Thank you for your response. I guess it's not quite as easy as feeding frames to an encoder to create a movie in the same way that it is for an animation. I suppose I'll look into alternatives then.
I guess it's not quite as easy as feeding frames to an encoder to create a movie in the same way that it is for an animation. I suppose I'll look into alternatives then.
I would not give up just yet!
While technically it is more complex, there are free, opensource and cross platform tools which handle all the nitty gritty details. Using it would be as simple as using std::process::Command.
But this all depends on your design.
I would prefer FFI over spawning ffmpeg, but both are good options.
It usually boils down to what you want to achieve.
I work with gstreamer and ffmpeg fairly often. In case of playback or streaming my go-to approach is linking to system libraries (that might be specialized for given platform). If I work with small tools that need some manner of reproducibility like conformance testing, transcoding, etc. I usually tend to write thin wrapper scripts over bundled tool like ffmpeg, mp4box or whatever because this is just less work than using FFI with negligible to none performance impact.
But as these things go, it often just comes down to personal preference 
MP4 is usually used with the H.264 codec, which is commercial and patented. I don't think there's a Rust implementation of it.
You could try your luck with the ffmpeg crate. It's not simple, but ffmpeg already has ability to use a couple of H.264 codec implementations, but you need to be mindful of licensing. Depending on your country and licenses you bought (typically with your OS or hardware), it may or may not be legal for you to create MP4/H.264 files.
Alternatively, you can use the free VP9 codec. There are crates wrapping the official vpx library. I have example code here:
@kornel I wasn't aware of the patent issues with MP4 before. Thank you for letting me know. I'll look into VP9 as an alternative. Hopefully it is as widely supported in browsers as H.264 is.
This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.