I'm looking for a Rust library that provides video -> iterator<image> and iterator<image> -> video abstractions.
In particular, I need to be able to do ops like (1) take a video, create a shorter video consisting of ever 10th frame, (2) take 9 input videos, create a single video of 3x3 grid.
Similar in spirit to ffmpeg / imagemagick, but I need to be able to do this in Rust/wasm32.
I'm not familiar with video crates, but you should take a look at the list of video crates, maybe there is one that works. Needing it to work in wasm sounds like it may be a challenge, because many of the video crates depend on non-Rust libraries, and while the C/C++ code can be compiled to wasm (sometimes), it's not guaranteed that the build.rs file is written to support this.
If you can use browsers' native decoding (i.e. only H.264 or VP9), then don't touch WASM, and do it by rendering <video> to <canvas>.
I have found sample code for drawing video to canvas frame by frame, by registering a callback on each frame. This solves the decoding problem.
However, I also need to do encoding. Is there a nice way to do encoding in wasm32? It's okay if I can only encode to one format as long as it can do both audio and video.
Did you look at using GitHub - Kagami/ffmpeg.js: Port of FFmpeg with Emscripten it is compiled to asm.js, you could put it to worker too for encoding.
Browsers can natively encode for WebRTC, and it may be possible to create a stream from canvas or something like that. Due to WebRTC involved this will be a lot of hassle tho.
In Rust land there is GitHub - xiph/rav1e: The fastest and safest AV1 encoder. which supports a pretty cool AV1 codec. I haven't checked if it's WASM-compatible.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.