I'd like do some work in this field, because it can get better performance than c++. Dose anyone give me some advice?
Could you explain more about why you think rewriting stuff in Rust can get you better performance than C++? I personally feel what are more important are a clean-slate design of system architecture and some technical innovations (e.g., smarter ways of doing kernel fusions). Those are important factors for better performance (and perhaps usability and more).
But having said that, I'm also very interested in the current state of the ecosystem in Rust for developing ML frameworks (e.g., linear algebra libraries, Rust bindings for CUDA/ROCm, and the ability to write CUDA kernel in Rust).
2 Likes
Thx~
I have been developing neural network operators for 3 years. There are a lot of bugs when our operators interactive with our own neural network framwork. These problems are difficult to solve, espeically the overflow bug. So I think if we can use Rust rewrite the framwork, we do not need to waste out time in the overflow bug.
Yes, it is.
About performance:
Rust is a little faster one than C++, but Iād not recommend to expect 100000% boost. The most important thing about performance is algorithm.
1 Like
No, not really. There are indeed some optimizations that are easier to perform in Rust than in C++ (e.g. due to the no mutable aliasing guarantee), but those probably aren't going to make or break the overall performance of a framework. Rust's optimizer is LLVM, the same behind Clang, which is on par with GCC. So if you write some C or C++ and the exact equivalent Rust, and then compile the former with a widely-used compiler, they will almost always have the same performance. I'd also guess that Rust is not as well supported as C++ on a variety of GPUs and especially specialized hardware such as TPUs (mostly of primary interest when you are doing deep learning).
The big advantages of Rust compared to C++ are:
- guaranteed and enforced memory safety (always a good thing)
- a strong type system (perhaps a little less valuable when all you have is tensors, but still very useful for developing high-level abstractions in other parts of the framework, e.g. error handling)
- a substantially smarter metaprogramming and macro system (great for automating away repetitive code that's not amenable to functional or generic abstraction)
These are all much more pronounced differences compared to C++ than peak potential raw compute. Yet, I'd argue that Rust is an excellent language to build an AI/ML/data analytics framework in! The aforementioned properties enable designs and abstractions that prevent a large class of bugs.
For example, not so recently I re-implemented the inference part of CatBoost in Rust because I needed it to run in the browser via WebAssembly. I generated strongly-typed, generic evaluators from the exported JSON model at compile time which statically ensured that the supplied input features matched in type and name those expected by the model. This was important because I generated a dozen or so different models with different input features, as ā unsurprisingly ā different sets of features proved to be useful for predicting different kinds of outcomes.
Had I had to deploy the same model in its native Python, I'd have been stuck with discovering feature space dimensionality (and semantics!) mismatches at runtime, perhaps even once it would have already been running in production. So while the implementation required some work on my part, it was totally worth it ā there have been zero crashes reported since the service started last year (granted, the user base is not enormous, but still).
1 Like
This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.