Adding an explicit lifetime leads to a `cannot borrow `*self` as mutable because it is also borrowed as immutable`

Hi,

i had to add a lifetime to a generic parameter in a trait so that it lives long enough when used in an iterator. Now, whenever an immutable borrow of self happens within a &mut self it no longer compiles.

Playground

I think i understand why but do not know a workaround. Any ideas?

I don't recall ever seeing any useful/correct use of a lifetime on self. It may exist, but every time I've seen it used it was a straight-up error.

There are two problems here:

  1. When you put 'a on a reference, it sets scope of this reference to the scope denoted by 'a. When this 'a is defined on the trait, it is maximally long, outliving any use of this trait. It means 'a self has already been borrowed before this trait was used with it, and may remain borrowed after.

  2. &mut loans are maximally inflexible (in technical jargon: invariant). It's also borrowed for exactly the scope of &mut, no more, no less. When something has been borrowed as &mut it is an exclusive loan, meaning no other loan anywhere can allow use of the same data.

Usually when you mix &mut and & loans, the compiler can use reborrowing to create temporary shorter-lived loans for you implicitly. It will take exclusive &mut, "pause" it, make a temporary shorter shared & out of it, let you use the shared loan in a short scope, and then forget the shared loan, and "resume" use of exclusive &mut. This is what happens when you call self.search() in the first case.

However, because you've said that &'a self takes maximally large scope (forever from perspective of this trait), it means it had to already exist even before search_and_remove has been called, so it's impossible to create it inside search_and_remove. It's too late for that, the scope of search_and_remove is too small (smaller than 'a).

And because &mut is always exclusive, you can't have &'a mut and &'a for the same scope. If they have different scopes, the compiler can ensure their uses don't overlap. But when they are sharing 'a it means their uses are always overlapping.

Oh, and &mut is an object that cannot be copied. Usually the compiler can hide this fact by "reborrowing", but the fixed invariant scope on it ruins that.

This is terrible:

trait VeryAwkward<'a, T: 'a> {
    fn search(&'a mut self) -> &'a mut Self;
    fn remove(&'a mut self);
    
    fn search_and_remove(&'a mut self) {
        self.search().remove();
    }
}

Just don't put lifetimes on self.

Hi, thank you. That's what i've feared.
The situation i am in is caused by the desire to make a concrete type generic.

In the context of iterators rustc gave me the advise to add a lifetime: the parameter type N may not live long enough and help: consider adding an explicit lifetime bound.
So maybe that road is not the right one to take.

It depends what you need to do. Sometimes it's possible to design an interface differently to avoid lifetimes on traits, sometimes you hit limitations of the type system (Rust is working on expanding generic associated types).

For example, there's an obscure for<'a> syntax that is typically used for Fn generic callbacks that take arguments:

fn foo(callback: impl for<'temp> Fn(&'temp i32))

Also check out interior mutability (article). You can have fn remove(&self) without mut, which makes lifetimes much more flexible. In Rust mut is not for mutation, but for exclusive access. You can mutate via shared loans too.

2 Likes

If we walk through what it would take to make this work...

trait DoesNotWork<'a, T: 'a> {
    fn search(&'a self);  // This borrow...
    fn remove(&mut self);
    
    fn search_and_remove(&'a mut self) {
        self.search(); // ...lasts for the entirety of the &'a mut borrow
        self.remove(); // ...and so the &'a mut can no longer be used
    }
}

We would need the search borrow to be arbitrarily short (just the one line in search_and_remove), which is what elision gives you:

// Does actually work now
trait DoesNotWork<'a, T: 'a> {
    fn search(&self);  // <-- changed
    fn remove(&mut self);
    
    fn search_and_remove(&'a mut self) {
        self.search();
        self.remove();
    }
}

And at this point all you're annotating is that Self: 'a, but that's already implied by having a reference with lifetime 'a to Self. So you can remove the parameter, and you're back at your Works trait.


Can you supply an example of the code with an iterator that produces the error you referred to?

1 Like

Thank you. The explicit lifetime came from an iterator. I've left out quite a bit to not confuse too much.

I had somewhere in the trait the return type something like dyn Iterator<Foo<Uuid>> and wanted to generalize it to dyn Iterator<Foo<N>>. This is where rustc complained about "N might not live long enough".

This generalization was quite a complex refactoring. My workaround now is to replace the iterators with vectors. :frowning_with_open_mouth:

I think I had it easy before, because Uuid is Copy. Nevermind.

Have you tried using an N: 'static bound on your iterators? The compiler is complaining that an arbitrary type N might have short-lived borrows, which would need to be handled specially. An N: 'static bound denotes that values of type N live for 'static, i.e., N has no short-lived borrows.