Avoiding "mutably borrowed" with std::mem::replace()

I've been following this thread, and learning from it:

The following code, based on that thread, fails to compile because nums is mutably borrowed by the time the compiler sees the new tuple being created:

fn fib() -> impl Iterator<Item = u32> {
    let mut nums = (0, 1);
    std::iter::repeat_with(
        move || {
            std::mem::replace(&mut nums, (nums.1, nums.0 + nums.1)).0
        }
    )
}

fn main() {
    dbg!(fib().take(6).collect::<Vec<u32>>());
    dbg!(fib()
        .take_while(|&x| x < 100)
        .filter(|x| x % 2 == 0)
        .sum::<u32>()
    );
}

error[E0503]: cannot use `nums.1` because it was mutably borrowed
  --> src/main.rs:13:43
   |
13 |             std::mem::replace(&mut nums, (nums.1, nums.0 + nums.1)).0
   |             ----------------- ---------   ^^^^^^ use of borrowed `nums`
   |             |                 |
   |             |                 borrow of `nums` occurs here
   |             borrow later used by call

But if the arguments were reversed, it would work:

// replace, but with the arguments reversed
fn swap<T>(src: T, dest: &mut T) -> T {
    std::mem::replace(dest, src)
}
fn fib() -> impl Iterator<Item = u32> {
    let mut nums = (0, 1);
    std::iter::repeat_with(
        move || {
            swap((nums.1, nums.0 + nums.1), &mut nums).0  // this works
        }
    )
}

Given that the target is mutable, it seems unfortunate that it appears first in the argument list.

But hmmm -- it doesn't really feel like it should make any difference. Despite their lexical order, argument evaluation precedes the actual call, so conceptually the new tuple is created first, right? Shouldn't the borrow checker look at it that way?

EDIT: Just to be clear, I am not suggesting that the order of argument evaluation be changed. I am asking whether the borrow checker should only consider the mutable borrow to occur at the time the function call is made, after argument marshaling. The function might mutate the variable, but the marshaling doesn't. I don't know why the mutable borrow would need to be in effect during argument marshaling.

3 Likes

In the failing case, operations are arranged in the following order:

  • Exclusive borrow is created on nums.
  • Code tries to read from nums to create new tuple.
  • Code is going to pass this borrow and this tuple into the function - and that means that the borrow must be alive, so the read attempt must fail.

In the working case, however, the order is different:

  • Code reads from nums to create new tuple. Since integers are Copy, the "read-lock" is held only during creation and immediately released.
  • The exclusive borrow is created.
  • Tuple and borrow are passed into function. Since tuple doesn't borrow from nums itself, there is no conflict.

In other words: your understanding is missing the fact that &mut nums has to be evaluated too.

1 Like

@Cerberuser what you explained about the order of parameters, it can easily imagine this can be extend to a lot of case. Do you have a rule in mind, or a doc, except the Book and the page about Copy and Clone, where this work around is explained? Because I think about so many case that I think a clear rule might be a real avantage for beginners.

I understand that the order of the code is different. But the way you are talking about it, it's as though the mutable borrow is required in order to marshal the pointer in the first place, and that doesn't sound right to me. The borrow is a compile-time concept, and it doesn't actually have to be mutable until the point at which the call is actually performed, right?

It seems to me that the list of operations for the failed case should really be:

  1. Generate code to marshal the address of nums as an argument, which is marked as being shared-borrowed so it can't be changed, and a note is made to mutably borrow it when the call occurs.
  2. Generate code to read from nums to create the new tuple and marshal its address as an argument. Since nums is only shared-borrowed, that's fine.
  3. Generate code to call the function, marking nums as being mutably borrowed.

And that would work fine, wouldn't it? I'm sure that the borrow checker and code generation actually occur in separate phases, but hopefully that helped to get the idea across -- the pointer is not actually mutably borrowed until the function is invoked.

Wrong?

Imagine:

let mut v = vec![42];
f(&mut v[0], v.pop().unwrap());

if Rust changed the order of evaluation to make this code compile (and panic on runtime) then it would be changing the semantics of the code.

EDIT: and if it didn't change the order of evaluation, and just instead "silenced" the borrow checker while evaluating function arguments, then f() would be receiving a dangling pointer as its first argument.


That being said, I agree with:

Indeed! Given that the initial order is arbitrary, it does seem unfortunate that such order was picked.

/// Extension trait to provide `.swap_into()` to all types:
trait SwapInto : Sized {
    fn swap_into (self: Self, to: &'_ mut Self) -> Self
    {
        ::core::mem::replace(to, self)
    }
}
impl<T> SwapInto for T {}

// ...
    (nums.1, nums.0 + nums.1).swap_into(&mut nums)
3 Likes

Thanks -- your SwapInto is interesting.

But just to be clear, I was never suggesting that the order of evaluation be changed. I was asking whether the borrow checker should only consider the mutable borrow to occur at the time the function call is made, after argument marshaling. The function might mutate the variable, but the marshaling doesn't.

Please read Rust: A unique perspective. In this light the problem is that you are first declaring the &mut reference to nums to be exclusive and after that declaration creating a second reference to it, contradicting your declaration of exclusivity. The underlying compiler technological problem is that LLVM can optimize the code for the &mut reference in a way that you will consider a mis-compilation, because Rust has guaranteed to LLVM (due to your &mut declaration) that no such conflicting access occurs.

2 Likes

You should not think of borrows as being "mutable or immutable", but instead as "exclusive or shared" in the same sense as a mutual exclusion lock.

In this frame of thought, the borrow is indeed a compile-time concept, but the mutability of the borrow does not matter. Only the requirement that the borrow holds exclusive access to the referent matters. And as Cerberuser describes, this requirement is not upheld in the case that the exclusive reference is taken before the tuple is created.

We already have ways to delay the enforcement of exclusivity until it is actually needed, the term to look for is two-phase borrows. That's what allows vec.push(vec.len()) to work. We could generalize this notion beyond auto-ref, as seen in the linked issue.

Currently two phase borrows only works on auto-ref, making the dot operator that much more special.

Two phase borrows in action


pub trait Replace: Sized {
    fn replace(&mut self, value: Self) -> Self;
}

impl<R> Replace for R {
    fn replace(&mut self, value: Self) -> Self {
        std::mem::replace(self, value)
    }
}

fn main() {
    let mut x = 0_i32;
    
    // should be equivelent
    // i32::replace(&mut x, x + 1); // but this doesn't compile

    assert_eq!(x.replace(x + 1), 0);
    assert_eq!(x.replace(x + 1), 1);
    assert_eq!(x, 2);
}

http://smallcultfollowing.com/babysteps/blog/2017/03/01/nested-method-calls-via-two-phase-borrowing/

5 Likes

Excellent -- thanks! Marking this as the answer. It makes sense that this would come up a LOT with methods that mutate self.

Yes, of course. That's why my suggestion was not to "silence the borrow checker," but rather to treat the borrow as a shared borrow for the duration of argument marshaling, and only as a mutable borrow when the call is actually made. Again, calling the function requires a mutable borrow, but marshaling the pointer does not (in this case). It just needs a shared borrow so that the marshaling of other arguments cannot cause problems.

It's called a "two-phase borrow," apparently. I had never heard of it.

Does anyone know whether the new Polonius borrow checker will include generalized two-phase borrows?

https://rust-lang.github.io/compiler-team/working-groups/polonius/

I should probably clarify that it is not always possible to defer the mutable borrow (just using a shared borrow) until after argument marshaling. Niko Matsakis offers a somewhat perverse example in his blog post about two-phase borrowing

let mut v: Vec<String> = vec![format!("Hello, ")];
v[0].push_str({ v.push(format!("foo")); "World!" });

So the implementation would have to figure out whether the deferral is actually correct. In the case of our fib() function, it's fine to defer.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.