I get your point. When I brought up "Your GPU is a JavaScript runtime", WebGPU and such I really meant that as an example of the kind of thing that is going on, not any particular solution.
I know very little of what goes on underneath the hood of AI, all that CUDA, PyTorch, Tensorflow stuff. But as far as I can tell the heavy lifting in all of it is done by C and C++ and the likes of CUDA. Python is the "glue" people use to drive all that from high level. Where people can do all that rapid prototyping and experimenting etc. (Mind you I have just been tasked with modifying a project that uses all that stuff from C++)
What I was getting at is that I don't feel that Rust is too high level to do what C and C++ are used for in this picture. Meanwhile it is not too low level to what people are using Python for. The fact that the example I gave is using TypeScript indicates to me that people want strict type checking, hey, welcome to Rust.
Something else to add here - many (most?) people ‘doing AI’ are building models by running interactive-ish scripts (Jupyter notebooks are big), and they don’t have a heavy software engineering background. It’s easy to learn enough Python to make this task manageable. Like others have said, Python drives a bunch of GPU code which is tied to C++ via (usually) CUDA. The ‘interactive-ish’ nature of AI workflows shouldn’t be understated and that’s where a scripting language like Python shines and Rust isn’t a viable option.
Of course, I personally prefer doing that sort of thing in Julia instead but I digress…
I just built and trained a neural network using Burn. For me, the motivation was that I preferred to use rust to explore the API and write my own code, and rely on cargo for building the whole system, than switch to python tools I am less familiar with. It was a good experience. It supports GPU backends.
Why do you think it would not be possible/beneficial to go deep with it?
If you want to chase new progress in the field, then it's most likely implemented in C++/Python. So you have to either port them or create complicated FFI to call them.
If you want to write low level kernel code, then you need CUDA or Triton, which neither can be called from Rust with ease.
Thanks so much to all of you for the thoughtful & super replies
honestly I was nervous posting this figured its probably a question that’s been asked like a 100 times already But your answers really helped and gave me some motivation to keep messin’ with it
I think I’ll just keep playin’ with diff datasets in Rust for now, and still use Python when it makes more sense tbh If I can help out or contribute to some open source stuff around AI in Rust, I’d def be up for it feel free to share links if you’ve got any projects in mind (yours or not)
I do think Rust’ got a bright future in a bunch of areas and I hope AI might be one of them someday But yeah, for that to happen, someone’s gotta start pushing a bit, so I guess I’ll just try to do my part, even if it’s small Gonna keep this as a side thing for now, but def with curiosity and the hope to maybe contribute to something cool eventually
And here there is a blogpost by the same author (2023), that talks about OP’s topic as well, and a condensed version in this part of the readme (you have to click once).
I’ve found it after starting with another article they wrote (2025), though it is less related. This was in the Newsletter 609, I think.
A brief summary of the first link above (human-made):
Python
Large Ecosystem (of available software)
Low barrier of entry (easy to use)
Messy dependency interaction
Rust
Cargo makes build, test, deploy easy (in comparison)
No runtime requirement (for binaries)
Easier to write safe concurrent software.
High-level abstractions and Performant
High entry-barrier, small ecosystem (in comparison)
Isn’t the issue that ZLUDA runs into the infamous CUDA EULA?
1.2 Limitations
8.You may not reverse engineer, decompile or disassemble any portion of the output generated using SDK elements for the purpose of translating such output artifacts to target a non-NVIDIA platform.
This is more related to running code on GPU than AI. The TL;DR extracted from the link further down is:
This demo is the first time all major GPU backends run from a single Rust codebase without a ton of hacks. It's the culmination of hard work from many contributors and shows that cross-platform GPU compute in Rust is now possible.
Will companies worry about providing ways to compile WGSL to their format, that runs fast?
Because WGSL would make a single codebase run on every GPU so it will become popular even if it is slow at first. WGSL will be the default on the web (i.e highly used.) by replacing WebGL.
wgpu(translates API) internally using Nagawhich translates WGSL shaders to the ones needed. But there are also ways to directly compile Rust code to other representations, see "Rust running on every GPU" test.
I think this is still fragmented, unless one writes 1. Rust, 2. To SPIR-V or NVVM-IR, 3. SPIR-V to any other shading language (Via Naga). See link above for examples.
That would cover most hardware.
Shaders may not be too hard to get (the GPU code may exist in other languages already.) but may indeed not be optimal, since manufacturers have optimisations for linear algebra operations that may be inaccessible.
There is Burn as well which was mentioned earlier, directly optimised for AI.
But your answers really helped and gave me some motivation to keep messin’ with it