Lifetime issue for a struct X (optionally) owning another struct Y that has a reference to X

[kunbin]

I have a problem about the lifetimes of the following code. I was wondering if someone could give me a hint.

struct Listener<'a> {
    stream: &'a mut Stream<'a>
}

impl<'a> Listener<'a> {
    fn new(stream: &'a mut Stream<'a>) -> Self {
        Listener {
            stream
        }
    }
}

struct Stream<'b> {
    listener: Option<Listener<'b>>
}

impl<'b> Stream<'b> {
    fn new() -> Self {
        Stream {
            listener: None
        }
    }
}

impl<'b> Drop for Stream<'b> {
    fn drop(&mut self) {}
}

fn main() {
    let mut stm = Stream::new();
    let lis = Listener::new(&mut stm);
}

(Playground)

The code cannot be compiled since there will be an error:

error[E0597]: `stm` does not live long enough
  --> src/main.rs:31:34
   |
31 |     let lis = Listener::new(&mut stm);
   |                                  ^^^ borrowed value does not live long enough
32 | }
   | - `stm` dropped here while still borrowed
   |
   = note: values in a scope are dropped in the opposite order they are created

My understanding is that the stm will be dropped after lis is dropped. That means stm will live longer than lis. If it's right, then I am confused about the error I got.

On the other hand, if I comment the implementation for Drop trait, then the code can be compiled successfully.

struct Listener<'a> {
    stream: &'a mut Stream<'a>
}

impl<'a> Listener<'a> {
    fn new(stream: &'a mut Stream<'a>) -> Self {
        Listener {
            stream
        }
    }
}

struct Stream<'b> {
    listener: Option<Listener<'b>>
}

impl<'b> Stream<'b> {
    fn new() -> Self {
        Stream {
            listener: None
        }
    }
}

// impl<'b> Drop for Stream<'b> {
//     fn drop(&mut self) {}
// }

fn main() {
    let mut stm = Stream::new();
    let lis = Listener::new(&mut stm);
}

I don't why the drop can affect the lifetime checking here. If someone know how to fix the above problem properly, please let me know. Thanks!

[parasyte]

This is a self-referential struct, which does not work in Rust. You can find some more context on the issue in this thread written by the author of rental, which might help you.

I don't think that was right. On second inspection, it looks a lot more like this explanation.

[vitalyd]

Presence of Drop causes the compiler to require a strict outlives relationship between the struct (Listener in your case) and the references it holds. You'd need to introduce a 2nd lifetime parameter to "disconnect" the mutable borrow of Stream and the lifetime inside the Stream itself, like so.

If you're ok with nightly compiler and using unstable features, you can promise to the compiler that your Drop doesn't access any of the references, which drops the strict-outlives requirement: example

[kunbin]

vitalyd, thanks for your hints! I deeply appreciate that. However, I am not sure if my understanding is right, could you check the following assumptions that I thought ?

1. References are never null
2. To make sure the references with the lifetime 'a in a struct T are never null(rule 1), the lifetime 'a >= the lifetime of the struct T, where the relationship of a and T is defined by struct T<'a>. Therefore we can make sure the references with lifetimes 'a in T must be alive when T is alive.
3. The presence of Drop for T will force the lifetime 'a for the reference to T < the lifetime for T. If the code is something like: ... r: &'a mut T, then it requires lifetime of T > 'a.

Thus, the code r: &'a mut T<'b> with a Drop implementation for T requires lifetime 'b > 'a since

1. By rule 2: 'b >= lifetime of T
2. By rule 3: lifetime of T > 'a
3. By above two steps, 'b >= lifetime of T > 'a, so 'b > 'a

Back to the original code, ... : &'a mut Stream<'a> with impl<'b> Drop for Stream<'b> will requires a lifetime 'a > 'a, according to rule for ... r: &'a mut T<'b> above. It doesn't exist a lifetime that is larger than itself, so the compiler gives an error. Is that right?

[kunbin]

Well, I guess I misunderstood the "presence of Drop" (rule 3) above. By the comment here(thanks parasyte!)

The essence of the Drop trait is that the drop method gets called when the value becomes unreachable, but while its members are still valid. This means that the owner of a value that implements Drop must have a strictly shorter lifetime than any references the value contains — those references had better still be valid when the owner goes away.

It should mean that: for a struct T<'a> that implement impl<'a> Drop for T<'a>, it requires the lifetime 'a > the lifetime of T. Without implementing Drop, T<'a> requires a lifetime 'a >= the lifetime of T (rule 2 above).

Thus, the following code requires a lifetime 'x > lifetime of Stream. (Without implementing Drop, lifetime 'x >= lifetime of Stream)

struct Stream<'x> {
    listener: Option<Listener<'x>>
}

...

impl<'x> Drop for Stream<'x> {
    ...
}

and the following code requires a lifetime 'a where 'a >= lifetime of Listener and 'a > the lifetime of Stream

struct Listener<'a> {
    stream: &'a mut Stream<'a>
}

When the following line is executed,

let lis = Listener::new(&mut stm);

it needs to find a lifetime 'a where 'a >= lifetime of Listener and 'a > the lifetime of Stream. The 'a we use here is the lifetime &mut stm. However, this 'a is not greater than(>) the lifetime of Stream, since it's a lifetime of a reference to a Stream.

Without implementing Drop for Stream, lifetime 'a is greater than or equal to (>=) the lifetime of Stream instead of greater than (>) lifetime of Stream. So it can be compiled successfully without implementing Drop, since the lifetime 'a can meet bothe 'a >= lifetime of Listener and 'a >= the lifetime of Stream.

Is that right?

[vitalyd]

I think that’s about right, but let me try a different explanation to see if it makes sense. I may mention things you already know so bear with me.

There are two aspects in play here:

• &'a mut T makes T invariant (ie mutable references make the value invariant) but is still variant over 'a itself.
• Drop adding a strict outlives requirement

The first aspect is only interesting when T has a lifetime parameter, such as Stream<'a> - a &mut i32, on the other hand, isn’t interesting - its variance is no different than &i32.

So what does this mean for the code in main()? When the compiler selects the borrow region for stm, what’s it going to pick? Well, since the 'a becomes invariant, it has to pick the entire lifetime of stm but moreover, it has to cover the 'a inside Stream itself because the same lifetime parameter is used in Listener for the borrow and the Stream lifetime.

The above is “fine” - if you omit the Drop impl, the code compiles. But there’s still an issue that the code in main() didn’t showcase - if you had tried to borrow stm again the compiler would complain that it’s still borrowed, even if the other listener were dead. Essentially, stm was “locked” for its entire lifetime due to the invariance. But ok, let’s ignore this bit and come back to the single borrower case and consider why Drop has the effect it does.

Drop requires that any references you hold must strictly outlive yourself. Why is that? Well, consider that some values can have the same lifetime scope - for example:

let (x, y) = (create(), create());

It doesn’t matter what the exact types are but the important bit is they’re created “at the same time” - lexically, there’s no order. As such, the order in which they drop is unspecified.

So in the general case, it’s possible for different values to be in such a predicament. When your Drop::drop is running, and you have references, the compiler wants to ensure that your possible access of the contained reference doesn’t touch an already dropped value - the only way to ensure that is with a strict outlives requirement: the value behind that reference must definitely outlive the reference you have to it. You can read more about this in the nomicon drop check section.

So putting all of that together, what happens here is the compiler tries to borrow stm for the entire function because Stream itself has a lifetime parameter. And then it’s also doing drop checking but to the compiler it looks like the borrow lasts beyond the lifetime of stm itself, and hence the complaint.

Alright, I’m on mobile and this took way too long to type . Hopefully it helps (and is correct!).

Oh, and using two lifetime parameters allows the compiler to pick the proper borrow region - forgot to state that! That’s safe because within Listener you now can’t erroneously stick the wrong lifetime reference inside the mutable borrow - the outlives requirement ('b: 'a) ensures that.