Isn't rust too difficult to be widely adopted?

Can we maybe have a book or booklet exclusively covering lifetimes? I don't think the first level of instruction material on lifetimes which is found in the rust book and others which talks about the syntax and the aliasing rules and elision is enough. It leads to an incomplete model which only frustrates when you discover its incompleteness. Rust nominoc does go further. For example it shows with detailed examples how lifetimes start with a let binding and how they interact with other lifetimes in the same scope. This is fundamental stuff and absolutely essential to understanding. But there are still aspects not covered there. For instance I realized that lifetimes can be shrunk as needed by the compiler only from this forum (thanks @vitalyd) which invalidated my previous model. And I'm sure there are other aspects I don't know about. I think we just need one place where one can learn everything about lifetimes and be done with it.

3 Likes

The nomicon covers this (to some degree) under the variance/subtyping section but I agree that there’s no single place to get “all lifetime related things” in one place.

1 Like

Any examples? Or more elaboration on specific pain points?

Just the general "viral" nature of lifetime annotations when attempting to store a reference in a struct. Any method that accesses those fields also need the same annotations. (I read recently that this may have changed? Lifetime elision is really nice when it can be used!)

For a specific example, the sdl2 crate has a Texture struct which requires an explicit lifetime. These structs are owned by a TextureCreator, which was supposed to alleviate this issue, but in practice does little. Storing a Texture in a struct requires the same lifetime annotation, as does every method that uses the texture. This specific example has come up in their issue tracker, which is an additional resource for material covering user experience problems with explicit lifetimes. edit: Here's another one.

If I'm not mistaken, this "explicit lifetime propagation" also begins breaking semver compatibility. Which could be a show stopper, depending on the context.

Both sides are possible, depending how you interpret the question.

  • Is Rust too difficult to become as broadly-used as something like Python? Probably. The majority of the code in the world doesn't need that last 10% of control and rigor that's the pay-off for the extra complexity.

  • Is Rust too difficult to supplant C++ as the default choice for systems programming? I've actually argued that Rust is easier than C++, so no. Having the compiler check more things is incredibly freeing in what you can just do without needing to stress over it.

8 Likes

Vast majority of cases will not require repeating the lifetime parameter of a field in a method of a struct holding that reference.

The sdl2 stuff I’ve seen on this forum as well - they all boil down to self referential structs, which is a known pain point. I suspect some domains (like graphics) tend to rub up against this much more so than others.

Lifetimes are contagious, no different than generic type parameters though. The notable difference is generic types can be erased in some cases (ie trait objects) whereas you can’t erase lifetimes. So that part is true. I suppose the closest analog to erasing lifetimes is to use owned objects instead, either passing things by value or putting them behind a pointer.

3 Likes

Exactly. This is what I have been alluding to above. Using this "subset" of Rust is a breeze. I also commented on explicit lifetimes eventually leading to self-referential structs in the other thread.

1 Like

But how often do you actually hit self referential cases (sincere question)? Just like trait objects should be relatively rare, I suspect most lifetime situations are relatively straightforward (think immutable iterators over a borrowed container). There are definitely libs that make it hard to use lifetimed types in certain cases (think tokio and higher layers on top), and so you learn the Rc<RefCell<...>> approach. So I think, perhaps, part of the learning challenge is knowing which approach to take when, rather than being afraid of one necessarily. This boils down to experience with rust and these libs - I don’t think there’s an easy way out.

1 Like

Serious answer: Never, since I stopped storing references. I also put my Tokio-based project on hold due to the same class of issues (with borrows across yield points -- I know progress has been made on this in nightly). As I mentioned before, I think there are ways to lessen the learning curve and provide better educational opportunities. I also don't think Rc<RefCell<...>> is at all ergonomic (and I don't believe you implied that it is). For that you trade propagating explicit lifetimes for propagating .borrow() and .barrow_mut() everywhere. But at least it doesn't contaminate the semantic versioning of public APIs.

1 Like

There is an entire second section as well, that covers more advanced topics.

I think a mini-book on lifetimes would be great, but it's hard to know what it should actually cover. And it should probably wait until after NLL lands.

7 Likes

Thanks @steveklabnik. I saw the second edition book, but I feel the lifetimes coverage can still use some expansion.

About the mini-book, I think it should cover anything and everything needed to be proficient in the area. (because you do kind need to know everything to be that way as I've found). Importantly, I feel the book should lean a lot on illustrations and examples. We can scour through this forum and see all the different roadblocks people have hit. They could form the different "patterns" which the book should treat individually. For instance: this is scenario A where you'd typically stumble and this is how you get out of scenario A and so on. It'll be great resource for new people and may help flatten the learning curve a lot (since most of it consists of the topic of lifetimes anyway).

1 Like

See, this is too broad: I need specific things to cover. Everyone agrees that we should do what you've said, but nobody is sure what exactly that is, you know? :slight_smile:

1 Like

Some ideas off the top of my head:

  1. Talk about what exactly 'a means in &'a T. For instance, it’s not necessarily the scope of where T itself is live but potentially a sub region that represents this particular borrow.
  2. Relatedly, draw a distinction between lifetimes and scopes/regions of values.
  3. Explain why methods of the form fn m(&'a mut self), where 'a is a lifetime parameter of the struct, is almost always not what’s needed.
  4. Talk a bit more and provide examples of how compiler is able to coerce longer lifetimes to shorter ones when possible (eg variance of immutable refs).
  5. Explain how lifetime parameter bounds (eg `b:'a) can’t actually enforce that some value outlives another. This has come up a few times on this forum.
  6. Explain and give examples of what it means for &'a mut T to be variant over 'a but invariant over T
  7. Show some examples of variance in play (eg rerurning a longer lived reference, such as 'static, from an otherwise generic lifetime using fn.
  8. Show examples where a struct should have multiple independent lifetime parameters vs being able to reuse one across fields. This is back to invariance.
  9. Show some examples of traits having a lifetime parameter, and explain the purpose.
  10. HRTB examples, such as expressing generic bounds for references
  11. Explain difference between mut ref moving and reborrowing, and ways to select one of them manually when need be
  12. Explain what the T: 'a bound means in struct Foo<'a, T: 'a> and when/why it's needed.
15 Likes

You're right that it's not sufficient. But by "anything and everything" I kinda meant all that a person proficient with lifetimes has needed to know to become so. I was hoping someone like @vitalyd would show up and spell it out (and what d'ya know, he already did that :grinning:). My own knowledge being incomplete, I wouldn't have been able to provide reliable answers. What I can say is that the material should not shy away from CS theory or compiler internals if that is what it takes to complete the understanding. Also I feel the use of illustrations to show where lifetimes start, how they are longer or shorter than other lifetimes etc. will work very well.

Yeah it's all good! I didn't mean that it had to be you, just the hard part is coming up with the details.

@vitalyd this is excellent, thank you! If anyone else has stuff like this, it would be excellent.

1 Like

Just added #12 there that I forgot to jot down yesterday.

1 Like

Bonus point if you can explain lifetimes without using words covariance/invariant. IMHO they're PLT-jargon, and wikipedia article for Covariance and contravariance is quite bad (long-winded introduction, and explains it in terms of type constructors, whichi is another PLT-jargon term).

4 Likes

Well, see this is the tradeoff. If you want an in-depth treatment of lifetimes, you have to talk about variance.

What do you think of Subtyping and Variance - The Rustonomicon ?

1 Like

I think you have to mention and explain (to some degree, it doesn't have to be a full on treatise) variance in a (mini) book on lifetimes. There's only so much handwaving before the rubber hits the ground. Subtyping might be an easier way to relate it, rather than the different variances. But, most importantly - show plenty of different examples so people can build at least an intuitive understanding/feel for it even if they want to gloss over the formal terms/definitions.

The intuition about lifetimes, or rather what they're used for, is something that a lot of people will understand. For example, a dangling reference due to a mismatch between the lifetime of a referent and reference is fairly intuitive, even for people coming from managed languages. The difficult part is learning how the syntax and the rules are used to express these different relationships, and why some cases appear different from others (i.e. immutable vs mutable refs). One example that springs to mind is iterators returning immutable vs mutable references. Explaining why a streaming iterator cannot be built out of the current Iterator trait is a useful exercise, which has taken place many times on this (and likely others) forum.

A particularly helpful thing would be to demonstrate different lifetime errors that a compiler spits out, explain what it's trying to prevent in that particular case, and then show how to express the desired contract to the compiler (or if it's not expressible in the current type system, provide a suitable way around it). I think there's quite a bit that can be done without constantly mentioning the different variance terms.

3 Likes

That's a pretty gentle introduction to these concepts, +1

Better yet, explain with and without those words and provide clear, concise definitions (as used by Rust) for those terms with links/references for further understanding of those terms.