I just finished my first AI project fully written in Rust a multi-class MLP trained on the Iris dataset. It was a realy cool experience and i wanna improve in it but i have some question :
Why isn’t Rust more used in AI today?
With its performance, memory safety... , Rust seems like a realy interesting language for AI. And yet the ai world is dominated by Python.
Is it cause we haven't mature libraries? The probleme when prototyping? It's harder to code in Rust? Or just its not the moment for it?
Id love to hear from others who have tried Rust for machine learning.
I don't know if i wasting my time to work on it...
So is it actually worth continuing to work on AI in Rust, or is it just a waste of time since Rust will never really evolve in that direction?
i saw for immature integration with GPU/CUDA that make me a bit mad. For the moment i work on my CPU but i know this have limits
The issue is path dependency. The entire AI ecosystem is already built on top of Python (and C), and pivoting to Rust is not a jump most people are prepared or willing to make.
There's ongoing effort on making the current situation better. But it's still quite clear that using Rust (at the time of writing ofc) would introduce unnecessary roadblocks.
Unless you value "Rust should be more common in AI" more important than "I want to achieve something with AI", and contribute back to Rust-GPU or Rust-CUDA related project, I won't recommend working on AI in Rust.
With all that said, since you are still at a very early learning stage, trying out tch or burn won't hurt. It will not be a waste of time, since the knowledge is transferable. It's just that you have to deal with python or C++ when you delve deep.
Even with a desire to use Rust for training or inference, there is just a lot of work involved in building out sufficiently complete and performant infrastructure. There are some trends in industry that make this easier (all AI models evolving into transformer variants) and some that make it harder (a proliferation of AI accelerators which each have their own SDKs, usually written in C++).
What I take from your answr is that: the industry and tool popularity make things locked in. It seems unlikely that we’ll see AI models running natively in Rust at scale any time soon
that said, continuing down this path may not be a waste of time
but this time could be invest more directly or productively elsewhere
I have to admit "I really enjoy working with Rust". I m already using it for decentralized/distributed networks and Tangle like models.
That’s what got me thinking about its potential for AI
For now, it s clearly not the most straightforward path but I ll probably keep contributing to some open source projects in that area, in my own small way
And in the meantime, I’ll explore other ideas and directions ^^
Rust is beneficial when building reliable code that is meant to last.
AI at the frontier is full of experiments and prototypes. Even AI tools that end up being shipped have a short shelf-life, and get obsoleted by a new thing relatively quickly.
Python (not counting the package management hell) has an ecosystem of very powerful libraries that allow easy prototyping and have low barrier to entry for GPU acceleration. Even if Rust could have that, it's not there yet, and the popularity of the Python ecosystem is self-reinforcing.
On the other end, there's CUDA dominating lowest-level implementations. It's tied to C++, but GPU-specific code is usually small enough to make C++ manageable. It's not application-wide memory management, just a bunch of performance hacks on mostly fixed-size buffers.
So Rust is too low-level for the high-level glue code used for research and experimental tooling, while also too high-level to make a difference for development of the assembly-like lowest-level GPU code.
AI is a very generic term, and as such it covers a wide range of applications/use cases. Rust may find some niche space where it excels. I’m thinking, for example, inference on the edge. The training for the most part may still be done in Python, where the ecosystem is very mature. But with ML standards like ONNX, you can deploy your trained model basically on any language with an ONNX library that supports the operations within your model. If low latency inference is a requirement, Rust may be a good choice.
Modular is building MAX - their AI runtime. It is a huge task - I think it will succeed because there has to be an alternative to CUDA. Mojo is the language they use at the top of their stack. Eventually they could support Rust and other languages if they wanted - I'm not sure they want to though; Might not be in their interest.
This is similar to what Tracel AI is doing with Burn. They support different back ends (from GPUs with e.g. Vulkan, down to Embedded CPUs with ndarray) so that the Rust code (and model) is independent of the hardware. They recently released some benchmarks that show they are on a good track in terms of performance.
At the moment, AI is cutting-edge research, so development is driven by researchers. Most researchers love Python. Once the field matures, focus will shift from research to engineering.
I wonder about that after watching this guy Your GPU is a JavaScript runtime* (TypeGPU deep-dive) creating GPU shaders in JavaScript. Except of course he does not use Javascript he uses TypeScript to get all the type checking. If one is prepared to do that what not Rust?
I would not say Rust is too high level. No more so than C. One can write C level code easily in Rust. Below that it would have to be assembler either way.
At the end of the day I think Rust would be quite suitable for all that. It's just that somebody has to make it happen...
No, this is very very far cry from what is used for AI, and is missing all of the reasons why people use Python even though plenty of programming languages can access GPU in some way.
Writing actual shaders for AI is too low-level, tedious, laborious and likely even result in slower code. Writing a shader this way is like writing assembly code by hand instead of using a compiler.
People writing "AI" these days don't write shaders. They use frameworks like pytorch or tensorflow (and even higher-level domain-specific machine learning libraries built on top of them). They already have plenty of high-level functions for math and statistics that WebGPU doesn't give you.
But also very importantly, such frameworks are optimizing compilers for ML-specific workflows. They can take your computation pipeline that consists of hundreds of naive step-by-step operations, and fuse multiple operations into more optimized shaders, with the shader code JIT-compiled and fine-tuned for a particular GPU.
This is sort-of what you get with Iterators in Rust, but for way more complex multi-dimensional data than a simple Item. You get nothing like that with TypeScript and WebGPU.
And WebGPU is too high-level. It's very nice for games and UIs, but for the obscene amount of computation that AI uses the lowest-level CUDA code is written with awareness of the hardware architecture that WebGPU abstracts away, and has many low-level primitives that WebGPU doesn't support.
though I've never tried using them with existing models.
But as others have pointed out - it's a very volatile field of CS and frequent changes and redesigns make it really difficult to pin down the requirements for a library. Even frameworks are constantly getting updated with new operators. So... these crates are useful if you have a specific model you want to bake into your application, but keep in mind you'll maybe have to run bindgen manually one day and fix the bindings on your own. Most models are likely to drastically change as well within a span of a year so doing this introduces a large maintenance burden unless you're professionally working on something and can pass that burden down to an employee.
There's large differences between different AI models as well and the ecosystem is very fragmented. Even looking just at something like layout parsing, there's a lot of different models you'd need to support to get a generic-ish library to achieve a certain task.
Alternative route would be writing a local server in python and interacting with an AI model through that (if you don't mind serialization), or even running via python bindings. This will add bloat and latency but be much more maintainable in long run even though you're essentially doing Rust -> Python -> C.
As for actually training models instead of running existing ones, it will always be easier to do from a dynamic language and there's little to no benefit from switching that part to Rust. I guess it would be interesting to maybe see a new markup language being developed that makes that part simpler than using Python.
Somebody wrote a platform-independent CUDA compiler... in Rust, no less... but because it violated the license terms of the Cuda language, he was forced to delete it and take it off the internet. It's a shame too because it ran faster on AMD than nVidia and showed that Intel's discrete GPUs didn't actually suck at all!
ZLUDA is still alive. The author was forced to remove AMD code because AMD lawyers approved it at first and then changed their mind. I can't 100% recollect in what form this temporary support from AMD came, but the new code is written from ground up without it.
When I was selecting a programming language for my AI project, I also consider Rust. But after careful weighting all benefits given by Rust, I decided to skip. But, I think you should continue using Rust in your ML solution and create a serious competitor to Python.