Annoying lifetime problem

I'm trying to do some fancy stuff with storing references in a collection but it doesn't work like it seems like it should. Can anyone explain what is going wrong? I'm tempted to call it a bug but I'm not sure. Here is a minimal case:

// Here is it working with immutable borrows.
fn foo<'a>(vec: Vec<&'a (dyn Fn())>) {
    // Not sure why this is needed. Maybe so that the lifetimes can be coerced?
    let mut vec = vec;
    // Create some inter-referential data on the stack.
    let x = 0;
    let func = || {
        dbg!(&x);
    };
    // Add a reference to the vec.
    vec.push(&func);
    // Drop the vec, the references are still valid.
}

// Being mut should not change the borrow lifetimes, right?
// But the function suddenly does not compile.
fn bar<'a>(vec: Vec<&'a mut (dyn Fn())>) {
    // -- lifetime `'a` defined here
    let mut vec = vec;
    let x = 0;
    let mut func = || {
        //         -- value captured here
        dbg!(&x); // `x` does not live long enough
    };
    vec.push(&mut func);
    //       --------- cast requires that `x` is borrowed for `'a`
}
// - `x` dropped here while still borrowed

use std::marker::PhantomData;

struct Baz<'a>(&'a dyn Fn(), PhantomData<dyn Fn() + 'a>);

// This breaks it even more without mut at all.
fn baz<'a>(vec: Vec<Baz<'a>>) {
    let mut vec = vec;
    let x = 0;
    let func = || {
        dbg!(&x);
        //    ^ borrowed value does not live long enough
    };
    vec.push(Baz(&func, PhantomData));
    //           ^^^^^ borrowed value does not live long enough
}

Here it is in the playground.

1 Like

Mutable and immutable references certainly act differently wrt. lifetimes. One thing is that mutable references must be unique, and the other is that unlike immutable ones, mutable references are not covariant. The latter is what matters here.

The issue is that since 'a is a parameter, it is chosen by the caller, and the caller may choose e.g. 'a = 'static. Now obviously a reference to a variable on the stack does not live that long, so you can't put it in the original vector. The reason it works in foo is that when you take the vector of type Vec<&'a (dyn Fn())> and assign it to the variable, the vector is actually coerced to a supertype, namely Vec<&'b (dyn Fn())> where 'b is some lifetime short enough that you are allowed to put a reference to func inside.

Notice I said super type. Shorter lifetimes are supertypes of longer lifetimes, and types such as Vec are what's called covariant, which means they inherit this subclassing behaviour.

The thing is that mutable references are invariant. They do not allow casting to a shorter lifetime, because even though the lifetime is a super type, the mutable reference generic over that lifetime is not a supertype.

You can read the link in the beginning of my post for some examples of why mutable references are invariant.

2 Likes

Thanks for the fantastic explanation! I hadn't thought at all about variance here. Looks like I am going to have to go back to the drawing board!

I don't think that variance plays a role here. &'a mut T is covariant in 'a and invariant in T, compared to &'a T, which is covariant in 'a and T.
OP seems to be using references as long lived pointers, which isn't going to work out. I.e. @samsartor is basicslly trying to create a dangling reference in bar

1 Like

Wait that's a very good point, and I got the explanation slightly wrong.

It's still variance's fault though: &'a mut (dyn Fn()) is implicitly &'a mut (dyn Fn() + 'a), and the reason bar fails, is that referencing x in the closure means that the closure is (dyn Fn() + 'b) for a very short lifetime 'b. In fact, this compiles perfectly fine:

fn bar<'a>(vec: Vec<&'a mut (dyn Fn())>) {
    let mut vec = vec;
    let mut func = || {
        println!("bar");
    };
    vec.push(&mut func);
}

playground

2 Likes

Seems to me that a better option here would be, since x is copyable, just pass by value to the closure. If it were an expensive allocation though, I'd probably think having an external var binding of Rc<T> or Box<T> would be better practice here.

Just to follow up, the issue does seem to be solely with variance here. The case I showed was simplified and I actually do have good reasons for building collections of references that Box and Rc don't solve. I tried for a bit to get all the types involved to be covariant but I'd have to throw away a lot more code. Right now I'm just going to live with invariance and avoid lifetime coercion entirely by introducing the collection later in the function. It isn't as nice but it seems to work.