Computer Vision in Rust?

So "this weekend" turned out to be 17 days later XD.

I think I can split my experiments into 3 categories.

  1. Tried and successively used Rust to do it:
    • Most of the linear algebra I needed. For this I used nalgebra. It is very well designed and covers all the needs I had, like matrix manipulations, decompositions (cholesky, svd, ...), rigid body motions (rotations, translations, ...).
    • Image reading / writing with PistonDevelopers/image. For 16 bits png images though I had to do it directly with PistonDevelopers/image-png and I mentionned how I did it in this issue.
    • Meta data files reading / writing with serde.
  2. Tried and would consider it a failed attempt:
    • Graphics visualizations. The GUI lanscape in Rust is actually quite complicated. It's memory model makes it complex to have efficient and simple GUI apis. I've experimented with PistonDevelopers/conrod a bit. It felt too complicated for my needs. I've written my thoughts in this issue in which mitchmindtree redirected me to nannou that would potentially be easier. I've not digged into it. In any case, I strongly recommand that you read this excellent post about interfaces and Rust.
    • Rapid prototyping with plots. I haven't found a way to produces plots "quick and dirty" to visualize if I'm doing things properly. The library milliams/plotlib could be nice but in the end, I chose to "just" export to csv files and visualize what I want with some vega-lite dataviz code. Not optimal but kinda work.
    • Video reading. It just felt that the overhead to make it work over just making images out of the video and read/write images was not worth it.
  3. Did not try yet:
    • graph manipulation
    • appearance algorithms for image matching

That's basically it. My overall impression is that with nalgebra I can re-implement most of the algorithms I need but the tooling for exploring approaches and evaluate them quickly is not there yet.

Rust has also a steep learning curve in my opinion. I very much like it's type system but I'm often fighting with the compiler for memory management, lifetime annotations and trying to understand macros / traits issues. The good thing is that most of the time, when it compiles, it works.

6 Likes