Graph IR in Rust for ML


#1

Hi,

So I recently started the project here https://github.com/Metadiff/graph-ir. I will briefly explain the idea - essnetially of of ML frameworks these days, like Theano, Tf, MXNET, they work on this abstract graph first, optimize it and the evaluate it. Note that the graph is essentially an SSA form of an AST, but where operands are large arrays of different shapes. This this begs the questions of why we don’t have something like an LLVM-IR instead, which can be common for all. This would allow us to write optimizations, which similar to LLVM , benefit everyone. One of the main goals of the project is to totally separate the IR from the actual backend which evaluates the graph. Note that the IR allows for very efficient static memory allocation, which could benefit embedded devices (this idea is to some extend use in MXNet).

Now to the real bit - since I’m a big fan of rust, writing this meta-compiler with autodiff (hence Metadiff), I was wondering if someone would be interested in writing it. I have a working version in C++, but I recently started seeing much better build times with Rust and maybe while I haven’t started working on the actual optimizations now is a good time to try to migrate it to Rust. Another benefit is that Rust will have much easier interface to other languages/front ends. The main burden is that this is a very large project as well as I’m not “a really good” software engineer. I did contacted some of the guys from Leaf, an old Rust ML framework, but did not get a response. Since I always liked the Rust community I just wanted to throws this out here.

On a side note, a 1000 more complicated thing would be to add this meta-diff compiler as an plugin to rustc and allow the whole process to work on native rust code. However, for this we first must have it working on its own.