Why does the mutability of the field impact the borrow check result?

help
1

Consider this example:

struct Foo<'a>(&'a i32);
impl<'a> Foo<'a>{
	fn try_ret_i32<'b>(&'b mut self)->Option<&'a i32>{
		Some(self.0)
	}
}

struct Bar<'a>(&'a mut i32);
impl<'a> Bar<'a>{
	fn try_ret_i32<'b>(&'b mut self)->Option<&'a i32>{
		Some(self.0)
	}
}

The confusion is that the borrow checker pass try_ret_i32 for Foo but rejects that of Bar. It's so confusing.

First, both receiver expressions' lifetime 'b should satisfy 'a:'b, respectively. It does make sense that the borrow checker rejects Bar, the return reference has the lifetime 'a while self.0 has the lifetime at most the same as lifetime 'b. So, the compiler emits the error

9  | impl<'a> Bar<'a>{
   |      -- lifetime `'a` defined here
10 |     fn try_ret_i32<'b>(&'b mut self)->Option<&'a i32>{
   |                    -- lifetime `'b` defined here
11 |         Some(self.0)
   |         ^^^^^^^^^^^^ associated function was supposed to return data with lifetime `'a` but it is returning data with lifetime `'b`

However, Foo should have the same treatment as that of Bar, the difference between them only is that Foo has a field of immutable reference type while Bar has its field with mutable reference. What's the exact reason that Foo is well-formed but Bar is not?

Another issue is, the type of self in either case is &'b (mut) T<'a>, so what is the lifetime of self.0? Is it &'a i32 or &'b i32?

1 Like
2

&'a i32 is Copy and &'a mut i32 is not Copy.

An anonymous lifetime which can not be longer than 'b.

1 Like
3

&'a i32 is Copy and &'a mut i32 is not Copy.

Whether it can be Copy or not, it does not impact the lifetime checking, I think. As said in the question, the expected return type has the lifetime 'a, which is longer than 'b, if we return a value with lifetime 'b, it will be an error. This analysis should be consistent in both cases, but the first is ok, I don't think it is the contribution of Copy.

4

In the first case, you return reference with lifetime 'a, since you can copy it from inside the structure and thus detach from &'b mut self. In the second case, you have to reborrow it from &'b mut self, and reborrow indeed can't last longer then 'b.

8 Likes
5

Do you mean

For self with type &'b mut Foo<'a>, since the field 0 has type &'a i32, which is a immutable reference, hence, (&'b mut Foo<'a>).0 produces the result value with type &'a i32?

For self with type &'b mut Bar<'a>, since its field 0 has type &'a mut i32, which is a mutable reference, hence, the result value should reborrow from &'b mut Bar<'a>, whose lifetime at most is 'b?

But, why does the reborrow need to happen for the second case but not happen for the first case?

6

As explained above, the immutable reference does not need to be reborrowed because it is Copy.

7

Does it mean, the compiler says we cannot move the field out through a mutable reference(i.e. the borrowing does not have the ownership), in order to make the code work, the compiler decides to reborrow a & mut i32 value from that mut self, Right?

8

I don't understand what you mean by this.

By default, no reference allows being moved out of. You can't move out of eiher a &T or a &mut T. Moving out of the pointed place would invalidate that, which is not allowed by the language definition (references must point to a valid value at all times). Given r: &String or r: &mut String, you can't say let s = *r;, it won't compile.

Of course, when moving does not invalidate the moved-from place, is when the type is Copy. Thus, for example, given r: &u32 or r: &mut u32, you can write let x = *r; and it will compile and work correctly.

Now, a shared reference is Copy, because it's just a pointer which is allowed to be aliased harmlessly. Thus, when you say self.0 in Foo::try_ret_i32, you are trying to move out from behind *self, which would not normally be allowed, but it is in this case because self.0 happens to be an &i32 which is Copy.

In contrast, when you are doing the same thing in Bar::try_ret_i32, self.0 would actually be a (destructive) move, because self.0 has type &mut i32, which is not Copy. In the absence of automatic reborrowing, the story would end here with a compiler error about not allowing a move out of a reference.

However, the compiler tries to be helpful, and in the case of references, it attempts an automatic reborrow. That sometimes helps, although it doesn't help in this case, since the reborrow is not allowed to outlive the original borrow. So you get an error anyway.

4 Likes
9

I don't understand what you mean by this.

I just meant we cannot move a field value out of the value pointed to by a borrowing(regardless of whether it is immutable or mutable). For instance:

struct B<'a>(&'a mut i32);
fn main(){
    let mut i = 0;
    let b1 = B(& mut i);
    let mrf1 = b1.0; // it can, because `b1` has the ownership 
   
   let mut i2 = 0;
   let b2 = & mut B(& mut i2);
   let mrf2 = b2.0;// it cannot occur because `b2` does not have the ownership
 
}

IIUC, the key point that is used to interpret the original question seems that:

Since the field with type &T is copyable, when we use a borrowing of the containing value to refer to the field, the result can have the original lifetime that &T has.

Since the field with type & mut T is not copyable, when we use a borrowing of the containing value to refer to the field, the result at most can have the lifetime of the borrowing because the result is reborrowed from the borrowing.

Is my understanding right?

More generally speaking, if a field is accessed through a borrowing and the field is of Copyable type, the result can have the original lifetime (if any) of the field. Otherwise, the result should be reborrowed(if necessary) from the borrowing and hence will have a lifetime that is at most the same long as the lifetime of the borrowing.

10

It's basically right. But an important detail here:

&i32 is not a type. &'a i32 is a type. When we say &i32 is Copy, we actually means for all 'a, &'a i32 is Copy. Then it's obvious we can get the same lifetime since Copying does not change the type.

1 Like
11

After thinking a lot, seems this rule is only applied to the field with a reference type because we are only permitted to reborrow a reference to a field if the field has a non-reference type.

12

In my opinion, you're overthinking it.

Foo compiles because it is sound - there is no way to cause UB using it. Bar fails to compile because it is unsound.

How the compiler comes to the conclusion that Bar should not compile is an interesting question and you'll develop a feel for how it works over time, like learning how your friends think because you've spent a lot of time with them. But it's less important than the fact that Bar is, in fact, unsound.

IMO you need to first develop an intuition for why things might be sound or unsound, then you can kind of follow the clues and hints dropped by the compiler to understand why a particular thing doesn't compile. The quickest way to see what the borrow checker is rescuing you from is to try to exploit it to create unsoundness.

For instance, Bar doesn't compile because if it did, you could write this:

fn main() {
    let mut n = 5;
    let mut bar = Bar(&mut n);
    let ref_to_n = bar.try_ret_i32();
    *bar.0 = 10;
    println!("{ref_to_n:?}");  // what should this print?
}

ref_to_n has to be a reference to 5, because it is an immutable reference, but the 5 it originally referred to now has a value of 10. If this worked, it would be UB. But if we imagine changing the body of try_ret_i32 so it returns &1, or any reference that doesn't borrow from *bar.0, there's no problem - main is sound, assuming try_ret_i32 is implementable. So the problem has to be in try_ret_i32 itself.

10 Likes
13

In the first example you essentially have a &'short &'long i32. You know the reference to the i32 will be valid for 'long and there's no way an exclusive reference can be created in that lifetime because it is already borrowed for 'long, so it's safe to get another &'long i32.

In the second example you essentially have a &'short mut &'long mut i32. You know that the reference to the i32 will be valid for 'long, but you don't have the guarantee that another reference won't be created in that lifetime. In fact when 'short ends you can get access to the &'long mut i32 again, and thus you can only get an exclusive reference for 'short, thus a &'short mut i32.

This is the intuition of why the first example works but the second doesn't. See also this playground

1 Like
14

Frankly, it appears that this way is not so intuitive for me. We need to consider whether there can exist a reference that would have lifetime 'long simultaneously. That's a bit indirect. I would prefer to understand this question in the "reborrow way".

15

Another part I may have some confusion is which stuff is what we are reborrowing. For instance

struct A<'a>{
    a: i32,
    b:&'a u8
}
fn main(){
  let mut r = A{a:0,b:&1};
  let mut mrf = & mut r;
  let rf_a = & mut mrf.a; // this is called, borrow from mrf.a
  let rf_b = & *mrf.b; // #1
}

Look at #1, is it called reborrowing from mrf.b or reborrowing from mrf? The syntax *mrf.b means we acquire the value pointed to by mrf.b by dereferencing mrf.b, then we borrow a reference from the resulting value. From this perspective, it should say we reborrow from mrf.b. However, as said in the above comment, the lifetime of the (re)borrowed reference will be associated with mrf, seems to we should say we reborrow from mrf to get the reference, from this perspective. How do you determine which stuff from which we are reborrowing？

16
  • Look at the definition of prefixes here

    We say that the prefixes of an lvalue are all the lvalues you get by stripping away fields and derefs. The prefixes of *a.b would be *a.b, a.b, and a.

  • And then the reborrowing section, which defines supporting prefixes

    The supporting prefixes for an lvalue are formed by stripping away fields and derefs, except that we stop when we reach the deref of a shared reference. Inituitively, shared references are different because they are Copy -- and hence one could always copy the shared reference into a temporary and get an equivalent path.

  • And continue on to the Reborrow Constraints section

    In that case, we compute the supporting prefixes of lv_b, and find every deref lvalue *lv in the set where lv is a reference with lifetime 'a. We then add a constraint ('a: 'b) @ P, where P is the point following the borrow (that's the point where the borrow takes effect).

  • And read the following examples too

Your #1 is reborrowing from mrf.b as per example 2; it is a borrowing of *mrf.b with a lifetime constrained to that of mrf.b.

More generally, working from the inside out (i.e. from longest to shortest lifetime), the lifetime of a reborrow will be capped by the first shared reference if there is one, and otherwise by the shortest (outermost) lifetime.

3 Likes
17

Thanks for your link. I have read that document. I have certain issues with the document and the relevant conclusion in your answers.

  1. Seems that "Supporting prefixes" and your conclusion:

the lifetime of a reborrow will be capped by the first shared reference if there is one, and otherwise by the shortest (outermost) lifetime.

does not apply to the simple case, consider the first example in the document

let x: &'x i32 = ...;
let y: &'y i32 = &*x;

If we were to use "Supporting prefixes" to think about this example, the path would be *x because x is a shared reference, so we just stop at *x, which would mean 'y == 'x, your conclusion would also have the same result. However, the example says 'x:'y

  1. Seems there is something wrong with the second example of the document, the comment says

Therefore, we would add one reborrow constraint: that 'a: 'c. This constraint ensures that as long as r_c is in use, the borrow of foo remains in force, but the borrow of r_a (which has the lifetime 'b) can expire.

Recall the example

let mut foo: i32     = 22;
let mut r_a: &'a i32 = &'a foo;
let r_b: &'b &'a i32 = &'b r_a;
let r_c: &'c i32     = &'c **r_b;
// What is considered borrowed here?
use(r_c);

According to the "Supporting prefixes" criteria, the path should be **r_b because *r_b is a shared reference, so what is the relevant lifetime that will be used in the constraint to 'c? This will be the third issue, however, according to the context of the whole document, I suppose it is the lifetime of value represented by *r_b(not sure), but *r_b will acquire the reference r_a whose lifetime is 'a that is used in the comment

Therefore, we would add one reborrow constraint: that 'a: 'c.

The wrong is exact in the last sentence

This constraint ensures that as long as r_c is in use, the borrow of foo remains in force, but the borrow of r_a (which has the lifetime 'b) can expire.

r_a anyway has the lifetime 'a, why does it say 'b? Seems it should say r_b can expire.

  1. According to the "Supporting prefixes" criterial, the document didn't clearly specify which lifetime will be used in the reborrow constraint. The lifetime seems to need to be determined by the form of the eventual path, I think

Consider such two contrast example

let mut foo: i32     = 22;
let r_a: &'a mut i32 = &'a mut foo;
let r_b: &'b mut i32 = &'b mut *r_a;

In this example, the lifetime in the reborrow constraint for lifetime 'b is 'a that is the lifetime of r_a, the eventual path in the supporting prefixes is r_a, which is a named variable.

Then, we consider the example whose eventual path is a DereferenceExpression

let mut foo: i32     = 22;
let mut r_a: &'a i32 = &'a foo;
let r_b: &'b &'a i32 = &'b r_a;
let r_c: &'c i32     = &'c **r_b;

Again, the eventual path in this example for r_c is **r_b, so what stuff in the path whose lifetime will determine the constraint? Is the reference designated by *r_b or the named variable r_b? It seems *r_b whose lifetime is 'a.

So, for the third issue, I have the conclusion that

If the eventual path is a named variable, the lifetime of the variable determines the reborrow constraint
If the path is a DereferenceExpression, the lifetime of the valued designated by the operand determines the reborrow constraint

Is this conclusion correct? For example

   let mut i = 0;
   let mrf:&'a mut i32 = &'a mut i;
   let rrmrf:&'b &'a mut i32 = &'b mrf;
   let e:&'c &'_ mut i32 = &*rrmrf;

The eventual path of e is just *rrmrf since rrmrf is a shared reference, since it is a DereferenceExpression, the constraint to the lifetime of e should be determined by rrmrf? Namely, 'b :'c

18

It doesn't mean 'y == 'x, it means (in the words I used) "'y is capped to 'x", which means (in the way we normally express these relations) 'x: 'y. Reborrows have to permit smaller lifetimes or you couldn't do stuff like

fn f<'a>(v: &'a mut Vec<()>) {
    let reborrow = &mut *v;
    reborrow.push(());
    println!("{v:?}");
}

If the reborrow was 'a, you wouldn't be able to use v anymore.

The relevent portion of the RFC is:

In that case, we compute the supporting prefixes of lv_b, and find every deref lvalue *lv in the set where lv is a reference with lifetime 'a. We then add a constraint ('a: 'b) @ P

Or in slow motion for the code above,

**r_b // supporting prefix which is a deref of *rb, which...
 *r_b // ...is a reference with lifetime 'a...
      // ...because type_of(*r_b) is *(&'b &'a i32) == &'a i32

// Resulting constraint: ('a: 'c) @ P

I'm failing to follow exactly what you're saying here.

The wording is confusing, I agree. The borrow of r_a is (stored in) r_b. r_b doesn't have to be valid everywhere r_c is valid; there is no ('b: 'c @ P) constraint. They are saying r_b can expire. The fact that r_b can expire means that r_a doesn't have to be borrowed at the commented line. So you can do this:

    let foo: i32 = 22;
    let mut r_a:          & /*'a*/ i32 = &/*'a*/ foo;
    let r_b:     & /*'b*/ & /*'a*/ i32 = &/*'b*/ r_a;
    let r_c:              & /*'c*/ i32 = &/*'c*/ **r_b;

    r_a = &0;
    println!("{r_c}");

If r_a was still borrowed at the println!, that would be an error. You can force that to be the case by using r_b to prevent it from expiring, and indeed, doing so results in an error.

Reasoning about things like this is part of why I prefer to read 'a: 'c as "'a is valid for (at least) 'c". If there was a 'b: 'c constraint, that would mean that 'b would have to be alive (in the lifetime sense) everywhere 'c was alive. That would mean that r_b: &'b &'a i32 would still be alive at the println!("{r_c}"), or any other use of r_c: &'c i32. That would mean r_a is still borrowed and can't be mutated (overwrote) in the code above.

The reborrowing constraint generation only applies to prefixes which are derefs of references with lifetimes, as quoted above and mentioned in passing in example 1.

It's true you can't take a reference longer than the value liveness scope of a variable, but that's not "the lifetime of the variable". That's a check that comes later ("dropping an lvalue").

Yes, the only supporting prefix is **r_b, *r_b is a reference with lifetime 'a, and thus the constraint is 'a: 'c.

If one of the supporting prefixes is a named variable, no lifetime constraints are generated based on that supporting prefix. However, if a named variable drops (or is overwriten etc) at some point where it is still borrowed (e.g. because you tried to borrow a local variable for some externally defined lifetime), that's a borrow check error.

So the lifetime of borrowing a named variable is effectively capped by the value liveness scope of the variable -- or by the next time you mutate it or take a &mut directly to it, etc. But these other effective constraints are not lifetime constraints per se. (I personally find conflating value liveness scopes with lifetimes to be a source of confusion more than a helpful mental model.)

There's a lot of nuance that could be explored in the "reporting errors" section, which talks about deep writes and shallow reads and so on.

2 Likes
19

It doesn't mean 'y == 'x, it means (in the words I used) "'y is capped to 'x", which means (in the way we normally express these relations) 'x: 'y.

'x: 'y, by your utterance, means 'x is valid in everywhere where 'y is valid, however, not the other way around. Consider this example:

fn check_lifetime<'x,'y>(a:&'x i32, b:&'y i32)->&'x i32 
where 'x:'y
{
     b  // error, 'y lives shorter than 'x
}

This is a subtle thing.

'y is capped to 'x

Sounds like 'y can at best be 'x.

So, from this perspective, the "supporting prefixes" does not apply to simple "reference" case, as specified in the first example.

Updated

Seems that the supporting prefixes also applies to the simple case

let x: &'x i32 = ...;
let y: &'y i32 = &*x;

In this case, the eventual path for y is *x because x is a shared reference. According to the law I summarized above, the lifetime used to form the constraint is x's since *x is a DereferenceExpression, x's lifetime is 'x, so 'y should satisfy 'x:'y.

If one of the supporting prefixes is a named variable, no lifetime constraints are generated based on that supporting prefix.

I think you may misunderstand what I mean by "named variable". In this case

let mut foo: i32     = 22;
let r_a: &'a mut i32 = &'a mut foo;
let r_b: &'b mut i32 = &'b mut *r_a;

The supporting prefixes path for r_b are *ra, ra, the eventual path is ra and ra is so-called the named variable(namely, an * IDENTIFIER* in the utterance of Rust reference). It is not a DereferenceExpression. r_a has type &'a mut i32, hence its lifetime is used in the reborrow constraint in the lifetime of r_b, namely, 'a:'b.

20

Yes. It's also true "capped" is too loose of a term when being this technical.

Yes.

I think you're right, so let me have a look at your latest example.

The supporting prefixes are *ra and ra, I agree. And here's what I'm saying happens (based on the RFC):

  • ra: Is not a deref
    • no lifetime constraints are generated
  • *ra: Is a deref of a reference with a lifetime (r_a), and said lifetime is 'a
    • 'a: 'b is generated

That corresponds to what you said, but the thing being dereferenced is not always an identifier.

Let's try something more complicated.

struct Holder<'a>(&'a mut *mut i32);

// Implied: 'c: 'b, 'b: 'a
fn foo<'a, 'b, 'c>(mmh: &'a mut &'b mut Holder<'c>) {
    let _: & /* 'd */ mut i32 = unsafe { &mut **(**mmh).0 };
}

Looking at the supporting prefixes,

  • mmh is not a deref, no constraint
  • *mmh is a deref of a &'a mut _
    • 'a: 'd
  • **mmh is a deref of a &'b mut _
    • 'b: 'd
  • (**mmh).0 is a field access, no constraint
  • *(**mmh).0 is a deref of a &'c mut _
    • 'c: 'd
  • **(**mmh).0 is a deref of a *mut _, no constraint

The multiple constraints are important for the reasons discussed in example 3.

This use of r must extend the lifetime of the borrows used to create both p and q. Otherwise, one could access (and mutate) the same memory through both *r and *p. (In fact, the real rustc did in its early days have a soundness bug much like this one.)

The deref-of-reference supporting prefixes are the important ones, whether they are

  • dereferencing an identifier like mmh
  • or dereferencing something else, like **mmh