After a week with Rust, I find that I keep referring back to chapter 4(Understanding Ownership) in the book(The Rust Programming Language). I thought the concepts were simple and odd but I neglected to understand the implications for data structures that ventured beyond the simple.
I think this small chapter holds the first keys to understanding simple Rust programs and I rushed right past it to write programs.
Is this a normal trend for a Rust rookie? Rushing right past the keys to the Rust language?
Note: I have a rudimentary background in OCaml so expression based languages and pattern matching are not sticking points and I have some experience(many years ago) in C and C++(more C than C++) so pointers are no problem.
I'd say this is fairly common. Usually most languages are very similar to each other and most concepts transfer to the point were you can often jump right in. However in Rust ownership, borrowing, aliasing & mutation and co. are so fundamental that you can't do even basic things without understanding them. This is why I'm heavily recommending reading the book before getting into Rust to everyone I meet that wants to learn it.
I feel your pain.... am in a similar position and would even say that I know C/C++ (not a super experienced developer, but I feel I know a bit), but Rust is giving me a hard time here. Yesterday I was close to decide to stick with C/C++. But will continue trying :).
You'll often hear this referred to as "fighting the borrow-checker". Rust forces you to be a bit more disciplined in the way memory is owned and managed so it can prove at compile time that your program is correct (in regards to memory usage and ownership anyway).
If I may, here are a couple tips:
Read the compiler's error messages. Like, actually read them. When I started out I'd see a message saying "Error at line 42" and immediately jump to that line of code to figure it out. Often, if you read the rest of the error message it'll tell you exactly what's wrong, and more often than not the compiler will give you a suggestion that fixes the problem. rustc's error messages are really good!
Try to avoid explicit lifetime annotations when you are starting out, especially when it comes to structs. 90% of the time you'll try to store a "pointer" to some object so you'll reach for a reference, when you actually meant for that type to own the object and could store a copy (check out the Clone trait). Keep it really simple then once you gain confidence you can start pushing for more advanced things. A good exercise when rustc complains about lifetimes is to look at your program and figure out how things are owned. If you have one object/variable owned by multiple others, that's usually a good indicator of why rustc is complaining.
Don't be afraid to ask questions! People on the user forums are very nice and will try to help work through your problem. A lot of the time if you can present a snippet of your code and explain your thoughts someone will be able to help you work through the problem and identify where your mental model of Rust may not align with how it's implemented.
References aren't special and don't add any extra runtime code. You can think of them as pointers which must always point to valid memory, where the variable they point to must outlive the reference. It's not uncommon to see people try to return a reference to something that'll be destroyed when the function returns (i.e. a dangling pointer). Unlike languages with a garbage collector, returning a reference won't magically make sure the object being pointed to lives for longer.
Complex webs of objects tend to be painful to work with. GC'd languages love having multiple objects that all have references to each other, but for languages with manual memory management (like Rust and C) these webs of references make it hard to figure out who is responsible for freeing a particular object. Doubly-linked lists are a textbook example of something that's easy to implement in other languages but painful to do in Rust (Learn Rust With Entirely Too Many Linked Lists is the canonical resource on this).
Complex webs of objects tend to be painful to work with
One common way to get around that is by using indices instead of references.
For example you can implement a graph on top of a Vec. Each Vec element is a node, and graph edges are implemented not as pointers/references (as you probably would do in C++) but as Vec indices. In terms of Rust safety features, that replaces lifetimes and borrowing (static checking) with bounds checking (dynamic checking, some of which the compiler may be able to elide).
Yes. Really getting ownership is the key to be productive in Rust without fighting the borrow checker.
C has ownership the same way Python has types. The concept exists, but you can make a mess of it and the language will let you get away with it.
Rust is not just statically-typed, but also "statically-owned". Just as usage of types in Python is not going to be much help in using C, experience with ownership in C is not going to be much help in Rust.
Forget about pointers, and treat structuring programs in ownership-compatible way as a new thing to learn.
I'm not sure if this is what you're doing, but note that starting by writing complex data structures is a terrible idea in Rust. Even a linked list -- widely considered easy elsewhere (either because GC or because YOLO) -- is hard enough that there's an entire book about it, because the ownership implications are complex.
It's definitely important to get used to using Vec, HashSet, BTreeMap, and friends before making complicated things like that of your own. (Basic structs are fine.)
It has been my habit over decades and many languages to start tinkering and experimenting with a new language way before I have read and understood the whole book. As it were. New, complex and advanced features can wait, there are things I want to know up front. How easy is it to get anything done in the language and with the tool chain? How is it going to perform? Is this even worth bothering with? I want confidence that I can get even the simplest things done before investing a lot of time.
Perhaps you have a point. Rust has this life time business to worry about. One might be tempted to skip over that as an "advanced" feature that one will attend to later.
But no. All that borrow checking is not just an advanced feature. It's a whole new feature that, correct me if I'm wrong, does not exist in any other language. Not only that it is, as you say, key to everything.
Some not so good analogy might be of a Javascript programmer trying to get their first C program to compile and being frustrated when the compiler complains they have not put types on any of their declarations. Then being frustrated that they can't just magically assign an integer into a string.
Yep, I've seen this happen a lot I often tell folks that if they're only going to read one chapter of the book, make it Chapter 4
@shepmaster and I made the Rust in Motion video series specifically targeted at helping folks who are used to picking up new languages by playing around with them and looking up syntax differences. Rather than trying to explore the majority of Rust like the book does, we concentrate on the concepts that are the most different in Rust from other programming languages so that you're not surprised when you run into them while playing around
I've been through everyone's pain, having struggled with Rust myself. I have been writing code for almost 60 years in just about every language you can think of (and some you can't) and Rust was the most difficult. I've written about 10,000 lines of Rust code and the result performs extremely well and is rock-solid. But it was a bloody struggle.
I have no desire to cast aspersions on anyone, but I am not a big fan of the "affectionately named"(!!) Book. The information is there, but I don't think it is well-organized or well-written. I highly recommend Blandy and Orendorff "Programming Rust: Fast, Safe Systems Development". It is among the best books I've ever read about a programming language, up there with K&R, Harbison and Steele and the Scheme Reports.
I've said this before here -- for ordinary applications that would not suffer from the use of a garbage-collected language, I would use one of them instead, e.g., Go, Nim, or Haskell (if you like Scheme, as I do, Chez Scheme is an excellent implementation, very fast, very mature). Rust imposes a significant cost on the programmer in its (successful) attempt to provide memory safety without a garbage collector. I paid that price because I was intrigued by the language, I'm retired and writing code for my own use, and I could afford the Rust experiment. If the work I did was a project I was managing in the real world, with actual costs and deadlines, I would not have chosen Rust.
But for work that really justifies use of C or C++, I highly recommend Rust instead. The learning curve will be justified by the correctness of the outcome and the absence of very hard-to-find bugs.
Lastly, I would recommend that if you pursue working in Rust and you get confused, use this forum. There are a number of very smart, well-informed people here who generously provide help to the needy. If I had had to rely solely on the documentation, I never would have gotten my project done. The help I received here was absolutely crucial.
I would also say that it there's any particular error that makes no sense, you're highly encouraged to file tickets in the main repo. Confusing or misleading rustc output is a bug and we're extremely receptive of proposals to fix them. I semi regularly trawl the different forums in search of as-of-yet unreported issues to file tickets, and you will see that in any period of time between a couple of days to a year or so they will get fixed (or at the very least improved).
One problem the people in the team have is that we are no longer newcomers, we don't have the fresh eyes to realize that some things don't make sense to someone who doesn't already know the language, so filing bugs for these is crucial for us to even know we have a problem.
As an aside, my personal 2020 goal is to make going back to The Book or other documentation for clarification on how to do something redundant: the diagnostics should give you enough context to teach you terminology and syntax as you go along.
Incidentally, I don't get how people who (claim to) have a lot of experience in writing C++ can struggle so much with Rust? The same concepts (e.g. ownership and RAII) exist in the language and even if one marginally tries to follow any sort of good practice regarding memory management and avoiding undefined behavior (i.e. not applying shared mutability, using smart pointers, wrapping low-level memory management into high-level data structures), surely the basics should be familiar…
Just what is it that people don't get? I see that "fighting the borrow checker" is a popular thing to say but rarely anyone provides specific feedback as to what that in particular means.
As a long time user of C/C++ and other similar languages I have to agree. As I like to say: You cannot escape the the borrow checker, it is always with you. In Rust it is there to help you at compile time. In other languages it is there in all the weird results and segfaults you have to debug later.
As for "actual costs and deadlines" it is often stated that the cost of fixing a bug goes up very non-linearly the later it is found in a project's life. What could be more cost effective than having as many bugs detected and fixed before you even have code that compiles?!
For this reason our new little company is developing everything in it's first project in Rust from the ground up. We don't have the man power or the will to ever be up all night desperately trying to track down a fatal memory error discovered after deployment again.
As for "what is it that people don't get".... my difficulty with Rust is not the borrow checker. It's the functional programming style and the whole language of FP used to talk about Rust. That is a whole new world for me. Perhaps not even so much the Rust language itself but the language people talk about it with. But that just comes down to my own ignorance and I'm sure I can get used to it.
The permitted concepts should be familiar, sure. But consider this exercise: Take a C++ code base you're familiar with, transliterate it into Rust without raw pointers, and see what Rust doesn't like. It's a trainwreck. And it's a trainwreck made of everyday practices that the C++ programmers have to find an alternative to if they want to work in Rust.
In the C++ code I've worked on, back pointers are ubiquitous, lifetimes are justified based on high-level application invariants, and there's just so much that you have to re-architect to do it in (safe) Rust.
Are you suggesting that the problem is that one cannot take a pile badly written, unmaintainable and no doubt buggy code in C++ and easily rewrite it as a pile badly written, unmaintainable and no doubt buggy Rust?
In the C++ code I've worked on lifetimes were justified based on some ill thought out hand waving argument that it's OK, that bad situation you imagine can never happen!
I would suggest something similar. But take some C++ code you are not familiar with, but you assume that it is well designed and which needs no GUI and is multi-threaded. I have chosen to implement the Rust counterpart to this book's C++ code, but I admit it might be too large to start with. But a raytracer or a renderer in general is a good starting point. A lot of people seem to have read and re-written Peter Shirley's Ray Tracing in One Weekend, which is smaller and probably easier to get started with. Anyway, I think something with multi-threading is good because a lot of concepts in Rust exist single threaded (e.g. Rc) and multi-threaded (e.g. Arc) and it's eye opening to see that you can start without multi-threading, then read a bit and try to implement something with multi-threading using more or less the same concepts you have learned before. Comparing against something you can compile and debug/single step through is good because you will learn about the heap and the stack, you can watch how things (especially pointers to something, like Vec or String) get stored in memory etc. I guess it's hard to find a small enough project and something well designed, but it's should be possible in the open source world to find something you can learn from ... and try to install and use tools like heaptrack and flamegraph ... my 2 cents
I've been writing code for 13 years, all of which I've only used GC languages, and most embarassingly, PHP. I've had some minor experience with C/C++, but nothing fancy. Nowadays, I'm learning Rust for about a month now, and I never had to "fight the borrow checker". However, I think the reason for that is that I didn't have to change my "conception" of programming - I simply never had any! RAII, ownership, thread- and memory-safety - all of those for me were just something you should know for a successful job interview.
And since I've started with Rust, I never asked myself "why doesn't it let me do this?", but regularly thought: "well, if I can't do it this way, how should I do it then?". Because I knew that rustc (and Rust core team) surely know better than I do.
So in my opinion, those who "fight the borrow checker" are just reluctant to change their habit of writing programs. Which is not necessarily bad, but this approach just doesn't work with Rust.
It's not bad, it's human nature. But that can make or break adoption of a new language. That is why it's critical to have a nice on-ramp not only for people for whom the concepts are new, but also as critically for people who have to relearn them. The documentation and diagnostics should explain what is happening and why, but the compiler also needs to understand concepts and patterns from other languages that do not work in Rust to guide devs towards equivalent, valid patterns.
Even when your ownership tree is fine, Rust doesn't like shared mutable references. e.g. you can't make a tree with parent pointers, or a doubly linked list easily. In C++ this sort of thing is a problem for multi threaded code. (It can also cause problems in single threaded code, but people usually get away with it, so it is a common pattern.)
In C++ people use shared_ptrs. In rust people seem to shy away from Rc and Arc. I'm not entirely sure why this is.
Iterator invalidation is the most common single threaded concurrency problem. It's basically what can happens if you tries to mutate the hashmap while iterating it. Java throws exception, C++ UB, and Rust fails to compile.
AFAICT it's mostly you can't normally mutate the data behind Rc/Arc. I don't think they themselves are bad in any way, but Rc<RefCell<T>> or Arc<Mutex<T>> can be costly in both performance and code reasonability.
I often tell folks that if they're only going to read one chapter of the book, make it Chapter 4