[HELP - BIG PICTURE] Big picture about: video editing, painting, vector graphic, animation, gui, 2d image processing... rust ecosystem, libraries and other things to know?

help
1

Hi Rustaceans and Graphic Connoisseurs :smile:!

I'm here to ask to the rust community about some topics that fascinate me!

But! I need to put in the table that I'm a completely noob about this stuff and I can make bad and unfocused questions!

first of all I need to cite some of my favorite software:

  • Gimp - image editor
  • Krita - painting software
  • Inkscape - vector image editor
  • Sygfig Studio - vector and animation editor
  • Obs Studio - streaming video software
  • Davinci Resolve - video editor

I've always used these software wondering how they worked on a technical level :thinking:.

But when I try to understand these things better, I hit a wall of hard technical details, my noobness and unfriendly C++ stuff... :thinking:

Today I feel like diving down this rabbit hole again with Rust :sweat_smile:, but I think I need to have a better overview and a clearer roadmap to follow.

if I want to understand how these software work and try to make some sketches in Rust, I think i need to separate in topic and subjects what these software do and what have in common each other.

I would like to try to make/experiment a GUI application with these features:

  • create a vector-layer-like canvas and modify vector-objects with the Bezier Curve
  • create an animation of a plain blue square that follow a path and export it in mp4
  • using alpha channel, make a selection and remove the background of a photo
  • paint 5 frames of a stupid stick man with the mouse and create an mp4/gif with them in succession
  • play and stop a video, crop it smaller, make appear a text with fade-in fade-out effect and save it

is it too hard? :sweat_smile:

I've been googling a lot and now I'm posting here a summary of what was found and I hoped I understood:

Windowing, Canvas, GPU and Cross-platform issues
When I want to create a desktop application, the first task is to take care of creating the GUI. In the web I found many different toolkits to do this: gtk, qt, wxWidgets. But I don't like dealing with cpp stuff :sweat_smile:. In Rust I found some interesting ones: egui, slint, druid and others: areweguiyet :thinking:
The second problem is to create and manipulate some kind of infinite canvas that allows me to create animations and image manipulation, I found egui very interesting in this topic with it's canvas widget.
But I don't know how low level I'll need to go on this kind of canvas. In this regard I found the lower level libraries winit and wgpu. But here I also discover the rabbit hole of os-specific-api, gpu and graphics drivers (webgpu, webgl, opengl, vulkan, proprietary drivers :dizzy_face:). I found these things and the cross platform issues to be quite confusing.

Math, Vector graphics, and Animations
in my google research in order to handle vector images/objects i found the beizer curve topic and regarding it's applications i found this two interesting projects: pathfinder, lyon and the fantastic egui canvas.
Furthermore, to make animations :thinking: i only found game engines and this interesting plugins for bevy engine: bevy+lyon and bevy+egui, but i don't know if this is the right way to go... :thinking:

Image/Video Processing \ Reproduction
When I tackled how to reproduce/manipulate/apply effects/export/import/get frame streaming/texture in a video in rust i don't found a lot: only a not-so-mature rust-media, but instead lot of threads cited these software: gstreamer, ffmpeg, sdl2. I see there is a rust binding for everything! but I honestly don't understand much which is better to use, if i need some of them or all of them :thinking:

now! some questions:

  • What is your vision about the rust ecosystem about this topic?
  • Do you have advises and material to share?
  • How difficult is this field, what are the challenges and difficulties i will face?
  • What are your recommendations?
  • Where should i start from? what are the right tools? what i should use?
  • What I'm missing?

plus some curiosity/technical questions:

  • What are the architectural/low level canvas difference in video playback, painting, apply effects to images, vector image handling, animate 2d objects? are all the same thing?
  • What technical aspects game engines and video editors have in common or do they differ?? :thinking:

thanks for your time! :grin:

2

Game engines and video editors can also differ in terms of their underlying technologies and design goals.
My advice would be to start with a small project and explore the various libraries and tools available to you. As you work on your project, you'll gain a better understanding of the technical details and challenges you might face. In terms of resources, the Rust documentation is always a great place to start, and there are also plenty of tutorials and examples available online.
Good luck and have fun exploring the world of image and video processing in Rust! And if you ever need to add text in video online, there are plenty of tools you can use.