Creating an AV library! :)

[quadrupleslap]

framing

Documentation | Code

What does it do?

It'll have basic constructs for handling images and audio samples, and particularly de/muxing them so that I can make that terrible Skype clone one day. Right now all it does is represent and manipulate images, though.

Why are you telling me this?

I want some feedback on:

• how to improve the video module.
• what an audio module might look like.
• how multiplexing might work.
• A better name would also be nice. :P

Also, what libraries should I wrap? Right now, I've written wrappers for:

• tomaka/vulkano - hardware acceleration.
• pornel/lodepng - PNG encoding and decoding.
• quadrupleslap/scrap - screen capture (and shameless plugging).
• FFmpeg in progress.

Any input is appreciated!

[kornel]

Fantastic!

For video you'll need to handle non-contiguous planes, so PlanarFrame needs to accept up to 4 planes (YUV+A), at varying resolutions (due to chroma subsampling). And with varying stride/pitch (since sometimes planes are exact width, sometimes even-only width, sometimes rounded to block sizes).

It would be nice if the library was fully aware of BT. 601 and Rec. 709 color spaces, including gamma-accurate conversions between them. It's such a pain to get colorspaces accurate across formats and HD/non-HD videos.

[quadrupleslap]

Thanks!

1. Regarding making the PlanarFrame more general, I’ll definitely try making a more general implementation of a planar frame at some point!
2. Gamma correction and BT.601 would both be really useful, but I’m not an expert on either of them, sadly. If you know how all of that works, mind sketching up an interface, or writing a pull-request?
3. Any ideas about multiplexing? Currently all I have is a Rustic clone of ffmpeg's frame type, which I’m reluctant to push because there has to be a better way.