Question from a beginner

I'm having some issues understanding lifetimes. Consider the following code:

struct Small{
    doto : i32, 
}

struct Big<'a>{
    unit : &'a mut Small,
    data : i32,
}

fn func1<'a>(arg : &'a mut Big<'a>) -> i32 {
    return 3;
}

fn func2<'a>(arg : &'a mut Big<'a>) -> i32 {
        let res = func1(arg);
        println!("small doto {}", &mut arg.unit.doto);
        return 4
}

fn main() {
    println!("hi");
}

When compiling, I get the following error:

error[E0499]: cannot borrow `arg.unit.doto` as mutable more than once at a time
  --> src/main.rs:16:35
   |
14 | fn func2<'a>(arg : &'a mut Big<'a>) -> i32 {
   |          -- lifetime `'a` defined here
15 |         let res = func1(arg);
   |                   ----------
   |                   |     |
   |                   |     first mutable borrow occurs here
   |                   argument requires that `*arg` is borrowed for `'a`
16 |         println!("small doto {}", &mut arg.unit.doto);
   |                                   ^^^^^^^^^^^^^^^^^^ second mutable borrow occurs here

For more information about this error, try `rustc --explain E0499`.

I don't understand why the compiler thinks that I'm borrowing mutably more than once. It issues seem to disappear if I remove the lifetime annotations of the references. Would anyone be able to enlighten a beginner here? Thanks!

If I understand correctly: To be able to call func1 from func2, the argument types must be compatible. This means that Big<'a> in func1 must refer to the same type as the Big<'a> in func2. This means that the two 'as must refer to the same lifetime. This means that because the 'a in func2 must obviously last at least the duration of the function call, the reborrow of arg by func1 must also last as long (because they're literally the same lifetime)! This means that even after func1 returns, the compiler cannot let you borrow arg again because for all it knows, func1's borrow of arg is still ongoing (it might have stowed the reference somewhere! In this case it did not, but its signature allows it do to so, and Rust by design does not peek inside functions to see what they really do with borrows, it must all be in the signature.)

This works for me:

-fn func1<'a>(arg : &'a mut Big<'a>) -> i32 {
+fn func1<'a: 'b, 'b>(arg : &'b mut Big<'a>) -> i32 {

-fn func2<'a>(arg : &'a mut Big<'a>)
+fn func2<'a: 'b, 'b>(arg : &'b mut Big<'a>) -> i32 {

(Playground)

Output:

hi

Note that writing : 'b is optional, as it's implied by the function's signature.

If you ever end up with a &'a mut SomeStruct<'a>, then SomeStruct is exclusively borrowed for the entirety of its lifetime -- only that reference can make use of it from there on out. This is pretty much never what you want.

A signature like this:

fn func1<'a>(arg : &'a mut Big<'a>) -> i32 {

means, "I can handle any lifetime 'a -- but the lifetime of Big<'_> and the lifetime of the &'_ mut reference to the Big<'_> must be the same!"

Where as a signature like this:

// Please don't write it this way, it hides the fact that `Big<'_>`
// has a lifetime, which is usually relevant information
fn func1(arg : &mut Big) -> i32 {

Is short for

// This is the prefered way to elide the lifetime of `Big<'_>`
fn func1(arg : &mut Big<'_>) -> i32 {

Is short for

fn func1<'a, 'b>(arg: &'a mut Big<'b>) -> i32 {

wherein every elided lifetime has been given its own fresh, independent lifetime. In this particular example there is still an implied relationship between the two -- 'b: 'a ('b is at least as long as 'a) -- because you can't have a reference that lasts longer than the thing it points to. But they are no longer required to be equal -- the borrow 'a can be arbitrarily shorter than the Big<'b>.

I.e. this signature means, "I can handle any well-formed* &mut Big<'_>, no matter what the lifetimes are."
*: Well-formed meaning, the reference can't outlast the referent.

So that's the difference between the signatures.


Making the error more sensible to you will probably require explaining a few things... I'll try to circle back when I have some more time if no one else covers it.

6 Likes

Ugh, I often omit it :see_no_evil:

Puzzle piece number one: When you have a generic parameter, the caller of the function determines what "value" (concrete lifetime, concrete type...) that parameter takes. They have to respect any bounds that you set ('a: 'b, 'T: Clone, lifetimes being equal), but if they can meet the bounds, they can call your function.

For lifetimes, this means that when you have something like:

fn foo<'a>(_: &'a str) { /* ... */ }

The only thing you can assume about 'a are the bounds (none in this example), and that 'a lasts at least as long as the body of foo. In particular, 'a here can last for any arbitrary amount of time longer than the body of foo. It could be 'static for all you know.

You can't redefine 'a in the middle of your function. (Once a lifetime is determined, you can't redefine it at all, in fact. But things like reborrows and variance sometimes make it seem like you can.)


Puzzle piece number two: &mut and reborrows. Unlike &, &mut references do not implement Copy. If they did, they wouldn't be exclusive. However, you might wonder why you can do things like this then:

fn increment(i: &mut i32) {
    *i += 1;
}

fn not_an_owner(i: &mut i32) {
    increment(i);
    // Wait, didn't `increment` consume `i`?  It's not `Copy`...
    increment(i);
    increment(i);
    increment(i);
    // ...but I can keep passing it to `increment`?
}

Well, you can reborrow a mutable reference, and the reborrow can have a shorter lifetime than the original, which goes something like this:

fn not_an_owner(i: &mut i32) {
    // Another exclusive borrow of the underlying `i32`.
    // Rust knows that this borrow is dependent on `i`,
    // so you won't be able to use `i` while `tmp` is
    // around / you can't use `tmp` after you use `i` again.
    let tmp = &mut *i;
    // Pretend you "lost" `tmp` when you did this because it is not `Copy`
    increment(tmp);
    // Now for the next one... our original `tmp` has "expired".
    // So we can use `i` again.
    let tmp = &mut *i;
    increment(tmp);
    // Or just inline these reborrows...
    increment(&mut *i);
    // etc
}

Rust does this reborrowing for you normally. It would be much less ergonomic if it did not. That's why you can pass (reborrows of) a mutable reference around.


Puzzle piece number three: &mut and invariance. When you're dealing with lifetimes of borrows, you're probably used to dealing with them being covariant -- that means, the compiler can make the lifetime shorter if that will make things compile. So let's say I have something like this:

fn same_lifetime<'a>(_: &'a str, _: &'a str) {}
fn main() {
    let static_ref: &'static str = "";
    let local = "".to_string();
    let local_ref: &str = &local; // Definitely not 'static

    // This is ok -- `&'static str` will just automatically get shrunk
    // to match `local_ref`'s lifetime.  (Actually, both will probably
    // be shrunk to be only as long as this function call!)
    same_lifetime(local_ref, static_ref);
}

However, as it turns out, you break memory safety if you allow lifetimes to change behind &mut references. So if you have a &'a mut T, the lifetime of 'a can still be shortened -- but any lifetimes that are part of T cannot be changed. We say &mut T is invariant over T.

For example:

fn same_lifetime<'a>(_: &mut &'a str, _: &mut &'a str) {}
fn diff_lifetimes<'a, 'b>(a: &mut &'a str, b: &mut &'b str) {
    // Error: We can't shrink `'a` and `'b` down to be some
    // smaller lifetime where they are equal, because they
    // are invariant inside the `&mut`s.  Replace `&mut` with
    // `&` to see that it works in that case.
    same_lifetime(a, b);
}

Variance can be hard to wrap your head around in the general case. "Things behind &mut are more restricted than things behind &" is a good place to start though.


Alright, now let's try to tackle your error.

error[E0499]: cannot borrow `arg.unit.doto` as mutable more than once at a time
  --> src/main.rs:16:35
   |
14 | fn func2<'a>(arg : &'a mut Big<'a>) -> i32 {
   |          -- lifetime `'a` defined here
15 |         let res = func1(arg);
   |                   ----------
   |                   |     |
   |                   |     first mutable borrow occurs here
   |                   argument requires that `*arg` is borrowed for `'a`
16 |         println!("small doto {}", &mut arg.unit.doto);
   |                                   ^^^^^^^^^^^^^^^^^^ second mutable borrow occurs here

func1 takes a &mut Big<'_>, same as func2. It's saying "first mutable borrow occurs here" because you're not giving up arg -- you try to use it later on. So there's an implicit reborrow going on when you call func1.

Big<'a> is behind a mutable reference, so it is invariant -- which means the 'a can't be shortened or otherwise changed when you call func1. The 'a in func2 and in func1 must be the same.

Your declaration of func1 said that the borrow of Big<'a> must last for 'a. Since 'a can't change, the exclusive borrow of Big<'a> you pass to func1 must also be 'a. That is:

  • You're creating a reborrow with type &'a mut Big<'a>, i.e.
    • "argument requires that *arg is borrowed for 'a"

How long is 'a? It's a type parameter input to this function, func2. It could be arbitrarily large. The one thing you know is that it definitely lasts throughout the body of func2, though.

So when you try to use arg again in the println!, it's still exclusively (mutably) borrowed by the reborrow you passed to func2. You just can't use it any more.


You could have fixed the OP by changing only func1 to not require the &mut to be as long as 'a. But you would have ran into the exact same sort of problem later when trying to use func2 from somewhere else.

&'a mut Thing<'a> is a red flag that your lifetimes are too restrictive / that you're requiring a lifetime equality that you shouldn't.

5 Likes

BTW, if you want to store structs by reference, use:

struct Big {
    unit : Box<Small>,
    data : i32,
}

or in practice, don't add any layer of indirection if you can avoid it:

struct Big {
    unit : Small,
    data : i32,
}

You can still temporarily borrow Small from there, and get &mut Small when needed.

&/&mut is never storing any data. It's not equivalent to references in other languages. It's a feature that intentionally forces structs to be confined to a small scope, and unusable elsewhere. &mut also intentionally adds extra restrictions on exclusive access and additionally limits flexibility of this usage (called "invariant" lifetime).

Thanks everyone for the answers, I understand this much better now!

Do you have any reference explaining this as well and concisely as you did in the rest of your post?

This example should give an idea as to why:

fn break_memory_safety(ref1: &mut Option<&'long T>, ref2: &'short T) {
    // If `&mut Option<&'long T>` could be shortened to
    // `&mut Option<&'short T>`, then this assignment would compile:
    *ref1 = Some(ref2);
}

fn make_lifetime_longer(my_ref: &'short T) -> &'long T {
    let my_opt: Option<&'long T> = None;
    break_memory_safety(&mut my_opt, my_ref);
    my_opt.unwrap()
}
2 Likes

Here's another couple examples, one where a lifetime behind a &mut is lengthened and another where it is shortened. Run with Miri under Tools to see the undefined behavior. (When compiling normally, sometimes I got garbage output, sometimes I got no output. That's UB for you.)

1 Like

Ok. Let me see if I can put my understanding into words. Is any of the following wrong?

&mut T is invariant in T. Allowing it to be covariant in T would effectively allow you to mutate the lifetime of T. But you don't own T, and the owner could be depending on the existing lifetime. As I see it, allowing this would require one of two "solutions". Either lifetimes would have to become a runtime concept, or you would have to do whole-program analysis, where the function signature is not enough information to know whether calling it is valid or not. But neither of these is acceptable. Is that correct?

That Box<T> and friends are covariant in T makes sense to me. However, there's one wrinkle that I don't understand. As an owning type, shouldn't UnsafeCell<T> be covariant in T as well? I think it could be, but only in the owning case. As soon as you introduce &UnsafeCell<T>, it must be invariant in T. Is that correct?

I guess I'm asking why this couldn't, in principal, compile:

fn test<'long, 'short>(inv: UnsafeCell<&'long str>) -> UnsafeCell<&'short str>
where
    'long: 'short,
{
    inv
}

Since the function takes ownership of the UnsafeCell, there can't be any outstanding references, so it seems to me like it would be sound to shorten the lifetime. But maybe it's too complicated to have different variances for T in UnsafeCell<T> depending on the context?

Your test function is perfectly fine. In fact, you can write the function in safe code like this:

use std::cell::UnsafeCell;

fn test<'long, 'short>(inv: UnsafeCell<&'long str>) -> UnsafeCell<&'short str>
where
    'long: 'short,
{
    UnsafeCell::new(inv.into_inner())
}

Hmm. But didn't you "cheat" with inv.into_inner()? You pulled the reference out of the UnsafeCell, then had the compiler apply the subtyping relationship, and then put it back in. The compiler didn't apply subtyping to the UnsafeCell directly.

With this example, the compiler does the subtyping directly, doesn't it?

fn covariant<'a, 'long, 'short>(cov: &'a &'long str) -> &'a &'short str
where
    'long: 'short,
    'short: 'a,
{
    cov
}

Oh wait, I see what you were saying. Yes, it's sound. And you can do it with safe code.

My point is that it would be safe for the compiler to auto-coerce an UnsafeCell<&'long str> to an UnsafeCell<&'short str> because the function I wrote does not use unsafe code and is able to perform that conversion.

The compiler doesn't actually do the conversion, which is why I need to "cheat".

Yes, got it. Ok. So why can't the compiler do it then? Is it just a practical limitation of keeping the rules simple?

Yes.