Create mp4 video from pictures in rust

help
3

I think I found a solution (at least sort of). Using the aforementioned libraries that wrap C libraries.
WebAssembly seems out of reach for now, but at least it works on all major desktop OSs.

Using these dependencies:

image = "0.23.6"
imageproc = "0.22.0"
minimp4 = "0.1.0"
openh264 = "0.2.11"

I was able to get this very basic sample working (it is a bit convoluted, because eventually I want to send the video out from a webservice, if you just want to write to a file it can be done more concisely):

use std::io::{Cursor, Read, Seek, SeekFrom};

use image::{EncodableLayout, Rgb, RgbImage};
use imageproc::drawing::draw_filled_circle_mut;
use minimp4::Mp4Muxer;

use openh264::encoder::{Encoder, EncoderConfig};

fn main() {
    let config = EncoderConfig::new(512, 512);
    let mut encoder = Encoder::with_config(config).unwrap();

    let mut buf = Vec::new();

    for i in 0..512 {
        let frame = get_next_frame(i);
        // Convert RGB into YUV.
        let mut yuv = openh264::formats::RBGYUVConverter::new(512, 512);
        yuv.convert(&frame[..]);

        // Encode YUV into H.264.
        let bitstream = encoder.encode(&yuv).unwrap();
        bitstream.write_vec(&mut buf);
    }

    let mut video_buffer = Cursor::new(Vec::new());
    let mut mp4muxer = Mp4Muxer::new(&mut video_buffer);
    mp4muxer.init_video(512, 512, false, "Moving circle.");
    mp4muxer.write_video(&buf);
    mp4muxer.close();

    // Some shenanigans to get the raw bytes for the video.
    video_buffer.seek(SeekFrom::Start(0)).unwrap();
    let mut video_bytes = Vec::new();
    video_buffer.read_to_end(&mut video_bytes).unwrap();

    std::fs::write("circle.mp4", &video_bytes).unwrap();
}

fn get_next_frame(index: u32) -> Vec<u8> {
    let red = Rgb([255u8, 0u8, 0u8]);

    let mut image = RgbImage::new(512, 512);
    draw_filled_circle_mut(&mut image, (index as i32, index as i32), 40, red);

    image.as_bytes().to_vec()
}

Performance is rather bad though and a pure Rust solution to this would definetly be great, however as I myself do not have the required knowledge to create a H.264 encoder, I will have to work with this for now.

I also found rav1e which seems to be great for the future (when for example Apple also hopefully starts supporting the AV1 format) however I was unable to get this to work with mp4-rs.