Lifetimes, seemingly making owned variable borrowed?!

[cos]

On the bottom of a rabbit hole, I encountered this error message which might be the most unexpected message I have ever seen from rustc since we first for acquainted years ago. Would value if someone could help me understand it.

Undoubtedly my code is incorrect. However I can not understand exactly why, and the error message leaves me puzzled. While I do believe to grasp to concept of lifetimes in theory, I must admit it is one of the areas where I can get stuck for quite a while in practise.

Please consider the following example: (switching the signature of bar() for its comment makes it compile.)

struct Foo<'borrow_lt> {
    value: Option<&'borrow_lt bool>,
}

impl<'silly_lt: 'borrow_lt, 'borrow_lt> Foo<'borrow_lt> {
    fn new() -> Self {
        Self {
            value: None,
        }
    }

    fn new_with_value(value: &'borrow_lt bool) -> Self {
        let mut foo = Foo::new();
        foo.bar(value);
        foo
    }

    fn bar(&'silly_lt mut self, _value: &bool) {
    // fn bar(&mut self, _value: &bool) {
        unimplemented!();
    }
}

fn main() {
    let owned = true;
    let _foo = Foo::new_with_value(&owned);
}

My expectation would be that new_with_value() would return a Foo with a lifetime which would allow main() to claim ownership. To quote, or slightly paraphrase, rust-by-example:

<'silly_lt: 'borrow_lt, 'borrow_lt> reads as lifetime 'silly_lt is at least as long as 'borrow_lt.

Surely that constraint should be possible to meet, as everything gets dropped when exiting main() in this simple example? In other words, it surprises me that the compiler picks different lengths for the two named lifetimes. However instead of successful compilation, the following is returned:

   Compiling lifetimes v0.1.0 (/tmp/lifetimes)
error[E0515]: cannot return value referencing local variable `foo`
  --> src/main.rs:15:9
   |
14 |         foo.bar(value);
   |         -------------- `foo` is borrowed here
15 |         foo
   |         ^^^ returns a value referencing data owned by the current function

error[E0505]: cannot move out of `foo` because it is borrowed
  --> src/main.rs:15:9
   |
5  | impl<'silly_lt: 'borrow_lt, 'borrow_lt> Foo<'borrow_lt> {
   |                             ---------- lifetime `'borrow_lt` defined here
...
14 |         foo.bar(value);
   |         -------------- borrow of `foo` occurs here
15 |         foo
   |         ^^^
   |         |
   |         move out of `foo` occurs here
   |         returning this value requires that `foo` is borrowed for `'borrow_lt`

Some errors have detailed explanations: E0505, E0515.
For more information about an error, try `rustc --explain E0505`.
error: could not compile `lifetimes` due to 2 previous errors

Obviously I'm missing something. What's with the talk about borrowing? All immediately related data is owned, or isn't it? How can I understand which lifetime actually is picked, and why is it too limited? My best guess is that maybe lifetimes are cut short for other (implicit) reasons whenever constraints are coerced? Possibly whenever a variable is moved, returned from a function, or what? (Or does 'a: 'b really only mean make 'a equal to 'b? Is it possible to get the compiler to elaborate on how it comes up with lifetimes?)

Maybe this has been asked a thousand times already? I did search the forum, yet found nothing. Pointers to either direct answers, guides on how to think like our compiler or any other helpful insights are most welcome!

[H2CO3]

@cos

Obviously I'm missing something. What's with the talk about borrowing? All immediately related data is owned, or isn't it?

Foo::bar() takes &mut self, hence the receiver is borrowed when you call it.

[2e71828]

You have included 3 conflicting lifetime requirements in your code:

1. Because Foo is annotated with 'borrow_lt, it must be destroyed before the end of 'borrow_lt to avoid a use-after-free error.
2. 'silly_lt: 'borrow_lt says that 'silly_lt cannot end until 'borrow_lt has ended
3. &'silly_lt mut self requires self to live at least until the end of 'silly_lt so that the reference remains valid.

Thus, self needs to be both destroyed and alive within the same region, between the end points of 'borrow_lt and 'silly_lt. As this is impossible, the compiler infers that this conflicting region must be empty, and therefore both lifetimes must be the same.

Because new_with_value receives an &'borrow_lt bool as an argument, this lifetime must necessarily extend beyond the end of new_with_value's body. It is then an error to borrow one of new_with_value's local variables for that long, as all local variables need to be dropped or moved when the function returns. Both of these actions are incompatible with an outstanding reference.

[alice]

Basically, a lifetime is the duration in which something is borrowed. To call bar, you must borrow the Foo object for at least the duration 'silly_lt, however that lifetime corresponds to a lifetime that lasts until after new_with_value returns, so foo is still borrowed when you try to perform a move of foo by returning it, but moving something while it is borrowed is not allowed.

[quinedot]

First a reminder: 'x: 'y means that 'x must be valid for at least everywhere that 'y is valid, if not more. But definitely not less.

Note that here:

impl<'silly_lt: 'borrow_lt, 'borrow_lt> Foo<'borrow_lt> {
    fn bar(&'silly_lt mut self, _value: &bool) {
// ...

bar is taking a &'silly_lt mut Foo<'borrow_lt> with 'silly_lt: 'borrow_lt due to your bound at the top. But a reference cannot have a lifetime strictly greater than the underlying type! There is an implicit 'borrow_lt: 'silly_lt bound here in order for the &mut to be well-formed (valid). Thus, there must be equality here -- the only valid choice is

impl<'borrow_lt> Foo<'borrow_lt> {
    fn bar(&'borrow_lt mut self, _value: &bool) {
// ...

But this too is problematic -- &'x mut Thing<'x> is an anti-pattern as the Thing is uniquely borrowed for the entirety of its lifetime. It's unusable by anything except the borrow at that point, which includes moving the underlying Thing.

So in other words, the problem is exactly that

the compiler picks different lengths for the two named lifetimes

is untrue. The problem is that it picks the same lifetime; this code will only work if it is allowed to pick a shorter lifetime. But it wasn't allowed to do so because of the bound you put at the top. And this is why it compiles with this signature, as you noted:

    fn bar(&mut self, _value: &bool) {

And also with this signature, where the explicit bound between lifetimes at the top has been lifted:

impl<'silly_lt, 'borrow_lt> Foo<'borrow_lt> {
    fn bar(&'silly_lt mut self, _value: &bool) {

There is still an implicit bound that 'borrow_lt: 'silly_lt (because you can't borrow for longer than the underlying type's validity), but this is fine -- the compiler is still allowed to pick a shorter lifetime than 'borrow_lt for the &mut.

[cos]

Thank you, thank you, thank you and thank you! I'm a bit overwhealmed by all your great answers, which helped me recognize where my thinking was flawed. I've hopefully reached a better understanding now. At least it feels so when rereading the book yet again today. Even the Rustonomicon makes more sense. While still not yet at the level where I can clearly describe the lifetimes resolvation process using my own words, I am confident now about getting there.

Grasping concepts is one thing, but ideally one could also wish to be able to verify what happens is what is expected. So far no more fine grained feedback than success/failure to validate lifetime validity has been mentioned. For completeness, it might be worth adding that Guide to Rustc Development mentions some ways to retrieve representations of intermediate compiler stages. That's quite a dragon filled dungeon though, obviously mainly intended for compiler development.

Indeed &'x mut Thing<'x> was what started leading me astray, but that's a different story relating more to finding the right data model. Which is comparingly simple to stepping out of being perplexed.

Thanks again!