Lifetime problems with an immutable borrow lasting too long

[nicholasbishop]

I've run into a problem with lifetimes I can't seem to get past. So far I've just put in lifetimes when the compiler yelled at me, but now I've run into a compiler error that doesn't give me a clear idea of how to proceed. (Sorry for the somewhat long example here; it's simplified a fair amount from the original source, but I didn't want to risk simplifying it too much and being unable to translate a solution back to the original code.)

trait MyTrait<T> {
    fn get(&self) -> &T;
}

struct Slice<'a, T> {
    slice: &'a mut [T],
}

impl<'a, T> MyTrait<T> for Slice<'a, T> {
    fn get(&self) -> &T {
        &self.slice[0]
    }
}

struct Component<'a> {
    name: &'a str,
}

struct Metadata<'a, Storage: MyTrait<Component<'a>>> {
    components: Storage,
    _phantom: std::marker::PhantomData<Component<'a>>,
}

impl<'a, Storage> Metadata<'a, Storage>
where
    Storage: MyTrait<Component<'a>>,
{
    fn do_mut_stuff(&mut self) {
        todo!();
    }
}

enum ValidationResult<'a> {
    Revoked(&'a Component<'a>),
}

fn validate<'a, MetadataStorage>(
    metadata: &'a Metadata<'a, MetadataStorage>,
) -> ValidationResult<'a>
where
    MetadataStorage: MyTrait<Component<'a>>,
{
    ValidationResult::Revoked(&metadata.components.get())
}

pub fn does_not_compile() {
    let mut metadata_storage = Vec::new();
    let metadata_storage = Slice {
        slice: &mut metadata_storage,
    };

    let mut metadata = Metadata {
        components: metadata_storage,
        _phantom: Default::default(),
    };

    validate(&metadata);

    metadata.do_mut_stuff();
}

(Playground)

Errors:

error[E0502]: cannot borrow `metadata` as mutable because it is also borrowed as immutable
  --> src/lib.rs:59:5
   |
57 |     validate(&metadata);
   |              --------- immutable borrow occurs here
58 | 
59 |     metadata.do_mut_stuff();
   |     ^^^^^^^^^------------^^
   |     |        |
   |     |        immutable borrow later used by call
   |     mutable borrow occurs here

In my mental model, the call to validate borrows metadata, and then it returns a struct that references the data in metadata. Since the return value isn't used, that struct would be immediately dropped and then metadata would no longer be borrowed. Then the do_mut_stuff call should succeed since metadata isn't borrowed anymore. Clearly the compiler sees things differently

Would appreciate any help understanding the error better, and how to solve it. I'd also be interested to know what resources I should be looking at to figure this out myself. Or, coming at it from a different angle, is there any way to get more useful info from the compiler here about why the immutable borrow keeps living?

[kpreid]

You're running into a problem with invariance and reusing lifetimes. Whenever you work with a mutable reference, it is a good idea to avoid using the same lifetime parameter for the mutable reference as other elements. In particular, the lifetime of the mutable reference should never appear inside the type of the referent, because a mutable reference is invariant over the referent type.

struct Slice<'a, T> {
    slice: &'a mut [T],
}

This contains a mutable reference, so we have to treat its parameters in the same way as we would a mutable reference.

struct Metadata<'a, Storage: MyTrait<Component<'a>>> {
    components: Storage,
    _phantom: std::marker::PhantomData<Component<'a>>,
}

We're going to use this with the Slice as MyTrait, so we need to treat 'a as we would a mutable reference's lifetime.

fn validate<'a, MetadataStorage>(
    metadata: &'a Metadata<'a, MetadataStorage>,
) -> ValidationResult<'a>

Oops! We've used the lifetime 'a for the name lifetime inside the mutable-referenced components of Metadata (therefore it is an invariant lifetime), but we've also used it for the reference to the Metadata. When the borrow checker sees this, it concludes that the two lifetimes must be equal, which has the result that calling validate on a Metadata results in the lifetime of the borrow passed to validate is necessarily extended to be equal to the lifetime of the mutable borrow the Metadata contains. This produces the error you are seeing: the lifetime of the & in validate(&metadata); has been extended to "until metadata_storage is dropped”, which makes it impossible to use metadata again.

You can fix the error by introducing a second lifetime for the borrows of Metadata, separate from the lifetime it contains:

...
enum ValidationResult<'r, 'a> {      // two lifetime parameters here
    Revoked(&'r Component<'a>),
}

fn validate<'r, 'a, MetadataStorage>(      // two lifetime parameters here
    metadata: &'r Metadata<'a, MetadataStorage>,
) -> ValidationResult<'r, 'a>
where
    MetadataStorage: MyTrait<Component<'a>>,
{

However, I should also mention that it is might be a mistake to use &mut as the basis for your structure's storage at all. The alternative path to “put in lifetimes when the compiler yelled at me” is “use owned values instead of references”. I suspect you might be doing this more thoughtfully than the usual beginner, since you're abstracting Metadata over its storage type using MyTrait, so it might be that you're already aware of this.

[nicholasbishop]

Neat, thanks for the explanation and solution. I'll be rereading that a few times for sure to make sure I've understood it properly, I appreciate all the detail!

Re. using owned values, I'm intentionally trying to abstract over the storage to handle different situations like having alloc available vs not.

[kpreid]

Not that you should stop trying, but the simplest no-allocator solution would be owning an array:

impl<T, const N: usize> MyTrait<T> for [T; N] {
    fn get(&self) -> &T {
        &self[0]
    }
}

which contains no lifetimes in the type. If the storage needs to occupy different memory than the Metadata, then you could also use &'static mut [T] as the storage type — of course, this implies a single statically allocated buffer.