Creating an AV library! :)

#1

framing

Documentation | Code

What does it do?

It’ll have basic constructs for handling images and audio samples, and particularly de/muxing them so that I can make that terrible Skype clone one day. Right now all it does is represent and manipulate images, though.

Why are you telling me this?

I want some feedback on:

  • how to improve the video module.
  • what an audio module might look like.
  • how multiplexing might work.
  • A better name would also be nice. :stuck_out_tongue:

Also, what libraries should I wrap? Right now, I’ve written wrappers for:

  • tomaka/vulkano - hardware acceleration.
  • pornel/lodepng - PNG encoding and decoding.
  • quadrupleslap/scrap - screen capture (and shameless plugging).
  • FFmpeg in progress.

Any input is appreciated!

#2

Fantastic!

For video you’ll need to handle non-contiguous planes, so PlanarFrame needs to accept up to 4 planes (YUV+A), at varying resolutions (due to chroma subsampling). And with varying stride/pitch (since sometimes planes are exact width, sometimes even-only width, sometimes rounded to block sizes).

It would be nice if the library was fully aware of BT. 601 and Rec. 709 color spaces, including gamma-accurate conversions between them. It’s such a pain to get colorspaces accurate across formats and HD/non-HD videos.

#3

Thanks!

  1. Regarding making the PlanarFrame more general, I’ll definitely try making a more general implementation of a planar frame at some point! :slight_smile:
  2. Gamma correction and BT.601 would both be really useful, but I’m not an expert on either of them, sadly. :frowning: If you know how all of that works, mind sketching up an interface, or writing a pull-request?
  3. Any ideas about multiplexing? Currently all I have is a Rustic clone of ffmpeg's frame type, which I’m reluctant to push because there has to be a better way.