Unexplainable Borrow Lock

help
1

I was originally investigating how the compiler performs borrow checking, but I found a strange and counterintuitive behavior associated with nested borrows, so let me share it here. In the following code, apparently, there is no reason to forbid move ptr_global at the very end. Neverthless, it does not get compiled. You can see the compilation log below.

fn _lifetime_bound_test<'a, 'b>(_lhs: &'a mut &'a String, _rhs: &'b String) {
    println!("There is no interaction between the inputs.");
}

fn main() {
    let a = String::from("a");
    let ptr_global = &mut &a; // More formally, ptr_global: &mut 'global &'global String.

    {
        let b = String::from("b");
        let _ptr_inner = &b; // More formally, ptr_inner: &'inner String.
        _lifetime_bound_test(ptr_global, _ptr_inner);
    }

    let _can_be_moved = ptr_global;
}

------------------------------------------------COMPILATION LOG----------------------------------------------------
error[E0505]: cannot move out of ptr_global because it is borrowed
--> src\main.rs:15:25
|
7 | let ptr_global = &mut &a; // More formally, ptr_global: &mut 'global &'global String.
| ---------- binding ptr_global declared here
...
12 | _lifetime_bound_test(ptr_global, _ptr_inner);
| ---------- borrow of *ptr_global occurs here
...
15 | let _can_be_moved = ptr_global;
| ^^^^^^^^^^
| |
| move out of ptr_global occurs here
| borrow later used here

For more information about this error, try rustc --explain E0505.
error: could not compile borrow_lock (bin "borrow_lock") due to 1 previous error.
If I remove the _lifetime_bound_test part, it gets compiled without any problems.

I would really appritiate if anyone could explain what is happening.

P.S.

It seems that &'a mut 'a T (&'a &'a T is fine) is the cause of the borrow lock. In fact, the following code does not get compiled.:

fn locker<'a>(_: &'a mut &'a ()) {
    println!("Do nothing");
}

fn main() {
    let a = ();
    let ptr = &mut &a;
    locker(ptr);
    let _new = ptr;
}

Result:

error[E0505]: cannot move out of `ptr` because it is borrowed
  --> src\main.rs:34:16
   |
32 |     let ptr = &mut &a;
   |         --- binding `ptr` declared here
33 |     locker(ptr);
   |            --- borrow of `*ptr` occurs here
34 |     let _new = ptr;
   |                ^^^
   |                |
   |                move out of `ptr` occurs here
   |                borrow later used here

For more information about this error, try `rustc --explain E0505`.
error: could not compile `borrow_lock` (bin "borrow_lock") due to 1 previous error
2

My understanding of why this occurs is that &'_ mut T is !Copy, so there's two choices for how passing a mutable reference into a function could work:

  1. Give ownership of the mutable reference to the function.
  2. Reborrow the mutable reference (with &mut *ptr_global), and pass the new reborrowed mutable reference to the function.

Note that Rust does 2 by default for the sake of ergonomics (otherwise, we'd need to write &* or &mut * almost every time we pass references to functions), and AFAICT it's equivalent (modulo the diagnostics/errors reported) to 1 in the case that that the lifetime of the reborrow is set equal to the the original reference's lifetime.

The result of either option:

  1. ptr_global was moved out of, and cannot be used again. Maybe diagnostics would be different, but your code still wouldn't compile.
  2. _lifetime_bound_test(&mut *ptr_global, _) forces the reborrow to last for all of 'a. Thus, the reborrow prevents the original ptr_global reference from being used for all of 'a, and since that's equal to its lifetime, ptr_global can never be used again.
3 Likes
3

The type &'a mut &'a String is, almost always, incorrect and will result in a nonfunctional program. So will any other &'a mut T where the lifetime 'a appears within T (unless 'a = 'static).

This is because the presence of this type creates two lifetime constraints:

  • &'a mut ... means that the data pointed to is exclusively borrowed for lifetime 'a, so it cannot be used in any other way until 'a ends
  • &'a String means these references are not valid after 'a ends

The combination of these two constraints leads means that the &Strings are exclusively borrowed “for the rest of their existence”. Thus, the variable ptr_global can only be used exactly once.

To avoid this problem, in general, the lifetimes of mutable references must be allowed to be different from lifetimes appearing in types those references refer to. In the case of function arguments, this usually means leaving them anonymous:

fn _lifetime_bound_test<'a, 'b>(_lhs: &mut &'a String, _rhs: &'b String) {

(Then, if you wish, you can also delete the rest of the lifetime names from this function, because each name is only used once and so imposes no constraints. But that’s irrelevant to the borrowed-forever problem.)

4 Likes
5

Thank you for the response first of all!
I have another question regarding case 2.
Could it be possible that a function without a returned value causes a borrow lock?
If a function returnes some referential type or a value that shares the lifetime with the inputs, it could lock the inputs even after the function call.
However, I cannot imagine how temporary function call that doesn't produce anything triggers a borrow lock.

6

Thank you for the response!
I still don't get how a lifetime works.
As far as I think, the lifetime parameters appearing in the function signature are applicable only to the variables generated inside the function. However, if this were true, there would be nothing wrong to move ptr_global after the temporary function call because _lhs and _rhs are already cleaned up. I think I still don't understand how lifetimes propagate across function calls.
Does the function signature influence the lifetimes of the original values living outside the function body?

7

Here are the complementary results with a little modification to the original script.

CASE 1;

I changed the lifetime parameters appearing in _lifetime_bound_test.

CODE:

fn _lifetime_bound_test<'a, 'b, 'c>(_lhs: &'a mut &'b String, _rhs: &'c String) {
    println!("There is no interaction between the inputs.");
}

fn main() {
    let a = String::from("a");
    let ptr_global = &mut &a; 

    {
        let b = String::from("b");
        let _ptr_inner = &b; 
        _lifetime_bound_test(ptr_global, _ptr_inner);
    }

    let _can_be_moved = ptr_global;
}

RESULT:

Compiled.

CASE 2;

In addition to the modification in case 1, I changed the lifetime of _rhs from 'c to 'b.

CODE:

fn _lifetime_bound_test<'a, 'b>(_lhs: &'a mut &'b String, _rhs: &'b String)
where 
'b: 'a
{
    println!("There is no interaction between the inputs.");
}

fn main() {
    let a = String::from("a");
    let ptr_global = &mut &a; 

    {
        let b = String::from("b");
        let _ptr_inner = &b; 
        _lifetime_bound_test(ptr_global, _ptr_inner);
    }

    let _can_be_moved = ptr_global;
}

RESULT:

error[E0597]: `b` does not live long enough
  --> src\main.rs:15:26
   |
14 |         let b = String::from("b");
   |             - binding `b` declared here
15 |         let _ptr_inner = &b; 
   |                          ^^ borrowed value does not live long enough
16 |         _lifetime_bound_test(ptr_global, _ptr_inner);
17 |     }
   |     - `b` dropped here while still borrowed
18 |
19 |     let _can_be_moved = ptr_global;
   |                         ---------- borrow later used here

For more information about this error, try `rustc --explain E0597`.
error: could not compile `borrow_lock` (bin "borrow_lock") due to 1 previous error

CASE3;

In addition to the modification in case 2, I changed the lifetime of _rhs from 'b to 'a.
CODE:

fn _lifetime_bound_test<'a, 'b>(_lhs: &'a mut &'b String, _rhs: &'a String)
where 
'b: 'a
{
    println!("There is no interaction between the inputs.");
}

fn main() {
    let a = String::from("a");
    let ptr_global = &mut &a;

    {
        let b = String::from("b");
        let _ptr_inner = &b;
        _lifetime_bound_test(ptr_global, _ptr_inner);
    }

    let _can_be_moved = ptr_global;
}

RESULT:
Compiled.

QUESTION

Why does compiler forbid case 2 while letting case 3 be compiled?
At first glance, they have the similar lifetime relation though.

8

QUESTION
Why does compiler forbid case 2 while letting case 3 be compiled?
At first glance, they have the similar lifetime relation though.

well, they don't havea similar lifetime relation at all. in 2 you got 2 &'b String, while in 3 you got one &'a String and one &'b String
you can do

fn _lifetime_bound_test<'a, 'b>(lhs: &'a mut &'b String, rhs: &'b String)
where 
'b: 'a
{
    *lhs = rhs
}

with 2, but not with 3.
and if you can do this, then it would result in use after free in 2, so it cannot be allowed

1 Like
9

yes ,see the function i just posted above for a simple example

1 Like
10

First of all, let's slightly simplify your code. You don't even need the second lifetime/scope, they don't interact and the error is still the same:

fn _lifetime_bound_test<'a>(_lhs: &'a mut &'a String) {
    println!("There is no interaction between the inputs.");
}

fn main() {
    let a = String::from("a");
    let ptr_global = &mut &a;
    
    _lifetime_bound_test(ptr_global);
    
    let _can_be_moved = ptr_global;
}

There 'a is invariant. See in Subtyping and Variance - The Rustonomicon.

How is this affecting your program? In &'a mut &'a String you are basically telling the compiler that String lives as long &mut to it lives. String is the value, then you create a shared reference for the whole duration of the function - &a will not be forgotten untill the end of main. The type of this reference is &'1 String. Then you are taking &mut to it, and by the variance rules the lifetime of an exclusive borrow is exactly the same as the lifetime of the borrowed type, in this case borrowed type is &'1 String and it has lifetime '1, thus the type of the mutable borrow is &'1 mut &'1 String. Then you are calling the function. It is generic over 'a, and 'a in this case is '1.

When you call a function, 2 things may happen: you either reborrow or move the reference. For invariant lifetimes it is basically equivalent: moving out will left the variable uninitialized while reborrow will render it unusable, as the lifetime is still forced to be '1 (it is not covariant there and can't be made smaller, see the link).

So what you get? 'a is '1, so function took your reference for '1. Remember what is '1? Until the end of main.

That's why you can no longer use the value. Function borrowed it. Duration of a borrow isn't tied to how long the function runs. It is just that for most situations they happen-to-be equal. There, you created a reborrow that lasts for a long time, way longer they the function runs, they you passed it to the function, it executed. Reborrow was for a certain lifetime and it is not tied to the function.

How to fix it? You can return the long reborrow back from the function:

fn _lifetime_bound_test<'a>(lhs: &'a mut &'a String) -> &'a mut &'a String{
    println!("There is no interaction between the inputs.");
    lhs
}

fn main() {
    let a = String::from("a");
    let ptr_global = &mut &a;
    
    let p = _lifetime_bound_test(ptr_global);
    
    let _can_be_moved = p;
}

Invariant lifetimes are tricky but if you read Rustonomicon (the link earlier) enough, you will understand how to work with them.

Related: Borrowing something forever - Learning Rust

4 Likes
11

Case 2 is forbidden by the same logic as in my previous post: 'b is invariant lifetime parameter because it is inside &mut. If this code could've compiled, you would be able to write lhs inside rhs, which cause UB after the second string is deallocated. Lifetimes of two strings are different and can't be covered to each other in writable context. This example is present in the link above iirc.

1 Like
12

I was originally investigating how the compiler performs borrow checking, but I found a strange and counterintuitive behavior associated with nested borrows, so let me share it here. In the following code, apparently, there is no reason to forbid move ptr_global at the very end. Neverthless, it does not get compiled. You can see the compilation log below.

fn _lifetime_bound_test<'a, 'b>(_lhs: &'a mut &'a String, _rhs: &'b String) {
    println!("There is no interaction between the inputs.");
}

fn main() {
    let a = String::from("a");
    let ptr_global = &mut &a; // More formally, ptr_global: &mut 'global &'global String.

    {
        let b = String::from("b");
        let _ptr_inner = &b; // More formally, ptr_inner: &'inner String.
        _lifetime_bound_test(ptr_global, _ptr_inner);
    }

    let _can_be_moved = ptr_global;
}

------------------------------------------------COMPILATION LOG----------------------------------------------------
error[E0505]: cannot move out of ptr_global because it is borrowed
--> src\main.rs:15:25
|
7 | let ptr_global = &mut &a; // More formally, ptr_global: &mut 'global &'global String.
| ---------- binding ptr_global declared here
...
12 | _lifetime_bound_test(ptr_global, _ptr_inner);
| ---------- borrow of *ptr_global occurs here
...
15 | let _can_be_moved = ptr_global;
| ^^^^^^^^^^
| |
| move out of ptr_global occurs here
| borrow later used here

For more information about this error, try rustc --explain E0505.
error: could not compile borrow_lock (bin "borrow_lock") due to 1 previous error.
If I remove the _lifetime_bound_test part, it gets compiled without any problems.

I would really appritiate if anyone could explain what is happening.

P.S.

It seems that &'a mut 'a T (&'a &'a T is fine) is the cause of the borrow lock. In fact, the following code does not get compiled.:

fn locker<'a>(_: &'a mut &'a ()) {
    println!("Do nothing");
}

fn main() {
    let a = ();
    let ptr = &mut &a;
    locker(ptr);
    let _new = ptr;
}

Result:

error[E0505]: cannot move out of `ptr` because it is borrowed
  --> src\main.rs:34:16
   |
32 |     let ptr = &mut &a;
   |         --- binding `ptr` declared here
33 |     locker(ptr);
   |            --- borrow of `*ptr` occurs here
34 |     let _new = ptr;
   |                ^^^
   |                |
   |                move out of `ptr` occurs here
   |                borrow later used here

For more information about this error, try `rustc --explain E0505`.
error: could not compile `borrow_lock` (bin "borrow_lock") due to 1 previous error

First of all, thank you for your cooperations!
With your supports and some experiments, I found out that the combination of the lifetime constraints on _lhs: &'a mut &'a String and the fact that an immutable referemce is copied when passed to another caused this behavior. In fact regarding the latter cause, we can see that it compiles fine if we pass ptr_global to the function by explicitly reborrowing it twice, which prevents _lhs from having the same lifetime as ptr_global while letting _lhs have only one lifetime 'a.
I really appreciate you guys.