Hi All, I'm new to Rust, coming from a systems engineering and Python background, with the idea of learning Rust for AI tooling and safe agent runtimes. To help get more fluent, I want to start converting some of my Python projects to Rust.
I figure this would be a good, ambitious, and robust starting point:
I'm hoping someone can point me in the right direction on how to proceed, which libraries to use, etc. In this project I'm using the python transformers library to pull in the DistilBERT model off Huggingface for news sentiment analysis. I'm using yfinance to get a data pipeline for ticker data. I use Matplotlib for graphs, and Pyside6 for the UI. The next step for this project is to move from json files to a database to use ML for predictions.
If there are packages built to setup a RAG harness for local AI, or learn to build/fine tune models with Rust's equivalent of PyTorch, those are more recent Python projects I thought might be better in Rust and what brought me to start learning it very recently.
Glad to meet you all, and thanks in advance for any suggestions.
How much Rust do you know already? Are you starting from scratch?
Assuming you already read the book but have little experience, I would recommend you start replicating the ticker data from yfinance: there you can get your hands dirty with data structures, parsing, networking, async (of you want to go that way) and error handling (among other things). That’s already challenging for a beginner.
Regarding ML in Rust, there’s nothing compared to PyTorch in ecosystem maturity. Burn may similar in capabilities, but it is not as easy.
learn to build/fine tune models with Rust's equivalent of PyTorch, those are more recent Python projects I thought might be better in Rust and what brought me to start learning it very recently.
What makes you think that Rust might be better for this in particular? honestly, in general, the strengths of Rust as a language offer no significant advantage for training. For inference, the advantage may be that you can integrate it easily with other Rust code without sacrificing performance (assuming Rust uses your GPU/CPU optimally).
I use better as a relative term. With PyTorch, I know I'm able to specify the device to use for the model as Cuda or CPU. I had heard that Rust may be able to offer more compatibility, especially in the use of different GPUs, and potentially TPUs or NPUs. It's possible I may just not be aware of the correct PyTorch commands, but being able to use what I have or something cheaper than a current generation Nvidia GPU, and also not renting time on someone else's computer, is better for me.
By book, I assume you mean Rust for Rustaceans? It's next on my list once I find a copy. I'm juggling a few responsibilities(parenthood, day job, health, etc.) so my reading list is a slower grind and I'm not able to pay attention as I'd like to audiobooks on the commute.
I'm barely 3 weeks into practicing Rust. I just started on the arrays unit of the guide I've been using and decided to have fun practicing building neurons from scratch. In the middle of that, I had the thought I should start planning out converting these projects now and decided to strike while the iron was hot.
That's where I started originally as well. I'm guessing yfinance-rs is the crate to use?
Usually, when we say "the book", we are referring to the official introductory book, "The Rust Programming Language". It is available online here:
...and is the standard recommendation for pretty much anyone who wants to learn Rust. It is possible they were referring to something else, but I suspect this is the book they mean.
I've noticed a good handful of Python libraries have actually been written in Rust, so my first step would probably be checking if I can just use the Rust version of whatever libraries I was using in Python (if it exists). Failing that, I'd probably look around on crates.io, lib.rs, and I'd check for any popular projects on GitHub that might already have some nice dependencies picked out.
At a quick glance, it does seem like this is probably the crate you want to use. I would generally advise you to be cautious, though; dependencies dealing with money (especially cryptocurrencies) seem to be particularly popular choices for supply-chain attacks these days. You might be surprised to learn that opening a project in your editor or running cargo commands may be enough to allow malicious dependencies to attack, even if you never cargo run. You should review your dependencies carefully, check in your lockfile (Cargo.lock) with version control, and use the --locked flag when running cargo commands.
The current state of Cybersecurity sometimes has me reminiscing of the days where I would disinfect systems using an antivirus that ran from a 5.25 floppy doing extracirriculars at my grade school. Now I have to have the conversation with the current net admin that no, this "new" program install is going outside standard install actions, requires a currently blacklisted installer, likely needs to be run with elevated privileges, not just an exception made, and should be reviewed in a sandbox before I proceed.
Thank you very much for the resources, I'm looking over them now.
If you are converting project from Python to Rust you will almost definitely want to have a transition step, doing only partial rewrites and calling new Rust modules from Python. In that case I recommend you read about PyO3.
If you do decide to go the route of calling rust from python then I wrote up my first foray in rust as a "how to for pythonistas": Combining rust & python - FizzBuzz
I'd probably do a lot differently now ... but it's still a reasonable insight into the differences between the languages, and practical PyO3 tips