I'm digging into the details of mutable (exclusive) reference and move semantics. I found the following examples a little bit contradictive based on the compiler message. Let me start with the code examples:
// Example 1.
// A simple illustration of move-only type.
fn main() {
let s = String::new();
let s2 = s;
// Doesn't compile, since String is move-only.
// s has been moved into s2.
s;
}
// Example 2, similar to example 2 but using function to move variables.
fn f(s: String) {}
fn main() {
let s = String::new();
f(s);
// Doesn't compile, since String is move-only.
// s has been moved into `f`.
s;
}
// Example 3.
// `&mut i32` is move-only.
fn main() {
let mut x: i32 = 5;
let y = &mut x;
let z = y; // move happens here
// Doesn't compile.
// Compiler said `&mut i32` is move-only (does not impl Copy trait).
// `y` is moved already.
y;
}
// Example 4.
// The interesting case. Why `y` is not "moved" and invalidated?
fn f(y: &mut i32) {}
fn main() {
let mut x: i32 = 5;
let y = &mut x;
f(y); // Shouldn't y be "moved" here?
// Compiles. Somewhat strange?
y;
}
My question:
It makes sense that a mutable reference type is move-only. But if it's move-only, shouldn't the call f(y) in example 4 "moves" y and invalidates the state? I definitely agree that a simple case such as in example 4 should not "invalidates" y for the ease of use, but syntactically it still seems inconsistent with move semantics. Should I just interpret this as a unique situation to mut references (i.e., thinking of this as a pure syntax sugar for mut ref)? Or there are some better way to think of it? Thanks!
It sounds like you're simply unfamiliar with the Copy trait, so I would strongly recommend reading chapter 4 of The Book, which goes into this and all the other basic / need-to-know concepts of ownership, move semantics and borrow checking in detail.
The direct answer to your specific question is that, to quote The Book (emphasis mine):
Rust has a special annotation called the Copy trait that we can place on types like integers that are stored on the stack (we’ll talk more about traits in Chapter 10). If a type has the Copy trait, an older variable is still usable after assignment
...
So what types are Copy ? You can check the documentation for the given type to be sure, but as a general rule, any group of simple scalar values can be Copy , and nothing that requires allocation or is some form of resource is Copy . Here are some of the types that are Copy :
All the integer types, such as u32 .
...
String does not implement Copy, since it requires allocation.
This is likely to the Deref and DerefMut traits, which allow implicit coercion of references. For instance this feature is why you can use a &String in a function that expects a &str — the type String implements Deref<Target = str>.
Note that this coercion introduces a reborrow of y, it's like if you had written &mut *y explicitly to make a &mut i32 by temporarily borrowing y.
The fact that Deref coercion happens in this case is not actually clear from the rust reference anywhere as far as I can see, but this appears to be what happened.
To compare: I've had some experience writing custom futures, and in that case you're dealing with Pin<&mut Self> instead of a simple mutable reference, and you have to explicitly call as_mut on it to make a reborrow instead of it automatically happening as seen above.
No, I meant it for this thread. I interpreted the question as “why doesn't f(y) cause y to be unusable afterwards when y has type &mut T despite mutable references being neither Copy nor Clone”
The answer to this question seems to be related to deref coersions to me. Maybe it's some other coercion — I don't know, but some sort of coercion or reborrow is happening.
As for the example with pin, it's an example of a situation where it doesn't happen — calling fut.poll(cx) on a fut of type Pin<&mut MyFuture> does indeed consume the pin, and you need an explicit fut.as_mut() to avoid that.
I agree, this is "just" an implicit reborrow, not a Deref(Mut) coercion.
However, I don't think Copy is relevant either; implicit reborrowing is unique to &mut references.
When you put a &mut reference at a coercion site (usually a function parameter) where a &mut reference of the same type is expected, the reference is implicitly reborrowed with a shorter lifetime, rather than moved. Reborrowing is different from copying or subtype coercion, and you can't enable it for any other type; it is built in to the compiler for &mut references.
Implicit reborrowing also does not work for functions that expect a generic type R where R is inferred to be &mut T — the function parameter must be of the form &mut _. You can, of course, still reborrow an argument explicitly, like f(&mut *y).
// Example 3 - modified
fn main() {
let mut x: i32 = 5;
let y = &mut x;
// let z = &mut x;
// ^ fails to compile
// error[E0499]: cannot borrow `x` as mutable more than once at a time
let z = &mut *y;
// compiles fine even though it looks like
// we have two simultaneous mutable references against `x`
*z += 1;
println!("{}", *z); // '6'
*y += 1;
println!("{}", *y); // '7'
x += 1;
println!("{}", x); // '8'
// ^ change the order to get compilation error
// error[E0502]: cannot borrow `y` as immutable because it is also borrowed as mutable
}
It surprised me that this worked.
The compiler declines simultaneous direct mutable borrows but has no problem with
let z = &mut *y;
The output makes it look like mutable references can "stack" where the more recent exclusive access "suspends" the previous one. Once the more recent exclusive access "goes out of use" the previous exclusive access becomes active again.
Is this behaviour intentional?
I'm sure my C/C++ background is getting in the way again - but I'd appreciate if someone could clarify the intent behind this behaviour.
On the surface it looks to me as if z is mutating x behind y's back while y is already holding exclusive access to x.
I'm sure that arguments can be made that this is perfectly "safe" behaviour but I still find it unintuitive.
Edit: thinking some more about it, it looks like exclusive access is being "relayed" like a baton (and then later returned).
I understand what you are saying but that's not what I'm asking. Yes String does not impl Copy (that's why example 1 doesn't compile). What I'm asking is, why doesn't the "seemingly" move operation in example 4 (i.e., f(y)) invalidate the y variable (whose type is $mut T, which does not impl Copy either). From the replies now I see this as a "stack borrow" thing which is unique to mut references (also makes total sense grammatically). So it's not really a move. And in function context it is a deref coercion: f(y) de-sugared to f(&mut *y) which is consistent with @peerreynders's example. Not it makes sense. Thanks all!
This further-digging example helped a lot! Now it makes total sense to me. So it's stack (re)borrow + deref coercion that makes the function call context in example 4 works! Thanks @peerreynders for the excellent example and @alice/@Hyeonu for the explanation!
@alice Adding lifetime annotation also helps to understand why deref coercion happens here. In a function call context, the reference of input params doesn't have to live too long in the caller context, so compiler can add an anonymous scope around the function call. Following the illustrations in 3.3 Lifetimes, Rustomonicon we can informally de-sugar the program
fn f(y: &mut i32) {}
fn main() {
let mut x: i32 = 0;
let y = &mut x;
f(y);
}
to the following:
// Doesn't compile, just a thought model.
fn f<'a>(y: &'a mut i32) {}
fn main() {
'a {
let mut x: i32 = 0;
'b {
let y = &'b mut x;
'c {
f::<'c>(&'c mut *y);
}
}
}
}
Now we can see that, due to a lifetime difference (from 'b to 'c), deref coercion must happen. These are indeed different mut reference "types" if you take lifetime into account, so no direct assignment(i.e., move) is happening here.
The compiler inserts a reborrow in any possible location where a &mut T is "moved" into a place that already has its type known to be &mut _ (where the _ is anything; it could be T, it could be some other U (in which case coercions may also be inserted), or it could even be left up to type inference with _ (in which case it is always inferred to be T)).
So:
let x = 0;
let y = &mut x;
let z: &mut i32 = y; // This is an implicit reborrow (equiv. to &mut *y)
let z: &mut _ = y; // This is an implicit reborrow
let z: _ = y; // This is a move
let z = y; // This is a move
Similar examples using function calls
fn takes_mut_i32(_arg: &mut i32) {}
fn takes_mut_t<T>(_arg: &mut T) {}
fn takes_any_t<T>(_arg: T) {}
takes_mut_i32(y); // this is an implicit reborrow
takes_mut_t(y); // this is an implicit reborrow
takes_any_t(y); // this is a move; sometimes this surprises people and there
// have been countless "bugs" filed about it
Interestingly The Rust Programming Language doesn't seem to discuss reborrows though it surfaces in Rust By Example 9.2.1 Capturing and 15.3.3 Aliasing and Programming RustChapter 5: References - Sharing versus Mutation (which means I should have known better) - though the stacking nature isn't explored at all. The explanations tend be along the lines of:
The mutable reference is no longer used for the rest of the code so it is possible to reborrow
When experienced coders try out a new language the process usually goes something like this: skim the docs as quickly as possible, go over some samples until one has “got the gist,” write some code; the expectation is that details and deep concepts will reveal themselves with use and that writing code is going to be the quickest way to familiarize oneself with the new tool. If you are learning Python, Java, JS, C#, Go, etc. this approach works well, mostly because these languages are basically all the same. If, on the other hand, you approach Rust like this you will probably be frustrated. There is nothing especially difficult about Rust (no more than C/C++, at least), but Rust combines novel concepts and refined ideas from other languages into a surprisingly innovative whole. Its design is cohesive, purposeful and visionary.
...
TL;DNR
Step:
Read The Rust Programming Language (to "get the gist")
Read Rust By Example (and fully understand each example)
Read Programming Rust (to deepen understanding)
Now start writing your own code
Bonus: Try to STOP thinking in terms of the patterns you are used to.
Finally: Give it Time
I'm inclined to believe that it's just the implicit reborrow that is needed to get Example 4 to work. y is already a mutable reference of the matching type.
// Example 4 - modified with explicit reborrow
fn f(z: &mut i32) {
*z += 1;
println!("{}", *z);
}
fn main() {
let mut x: i32 = 5;
let y = &mut x;
f(y); // implicit reborrow
f(&mut *y); // explicit reborrow
*y += 1;
println!("{}", *y); // '8'
x += 1;
println!("{}", x); // '9'
}
I agree with you on that. What you are saying is that, once an explicit "&mut..." occurs in the type, implicit reborrow happens which behaves exactly like deref coercion + stacked reborrow. We can definitely think of this as a whole syntax rule or a grammar sugar. But another way to think about it is as follows:
&mut i32 itself is not a complete type. An lifetime annotation is needed to complete a reference type. (Either explicit by 'a notation or implicit by the compiler.)
Assignment to a ref type from a different ref type (including lifetime difference, i.e., considering "complete" types only) results in (implicit) deref coercion, which behaves as if you wrote &mut *... or any other more complicated coercion results such as &mut *****.....
Stacked-reborrows of &mut T will not invalidate the original mut reference. It will only push it to a "lifetime borrow-stack" so it's temporarily disabled. Once it's back on the (lifetime) stack top, it's revived.
These three rules by themself exists, are independent (and beautiful as well, especially the third one). On top of them, "implicit-reborrow" is another rule that can be deducted. It's definitely OK to make "implicit-reborrow" as a rule by itself but that seems not necessary.
Also consider the following examples:
fn f1(x: &mut i32) {
let y: &mut i32 = x;
x; // Compiles.
}
fn f2<'a>(x: &'a mut i32) {
let y: &'a mut i32 = x;
x; // Does not compile. Cannot move out of `x` because it is borrowed.
}
Your three-point model is very close to the truth, but falls ever so slightly short. I'm going to explain why, but first let me say that there isn't a compelling reason to alter your mental model. You might, like me, find the following interesting, but if your goal is just to write high-quality Rust code, this is likely not a language rule you need to commit to memory.
Close, but not exactly.
Again, close, but not exactly. Implicit reborrowing is a distinct rule that cannot be derived from applying deref coercion, even if you consider the lifetime a part of the type.
Consider this function:
fn no_implicit_reborrow<'a, 'b>(a: &'a mut (), b: &'b mut ()) {
fn same_type<T>(_a: T, _b: T) {}
same_type(a, b);
//let _x = a; // error[E0382]: use of moved value: `a`
let _y = b; // OK
}
The caller of no_implicit_reborrow is what gets to choose 'a and 'b, so the function itself cannot suppose that 'a: 'b or vice versa. Nevertheless, same_type(a, b) compiles, because no_implicit_reborrow can choose T = &'c mut () where 'c is some lifetime in the intersection of 'a and 'b.
Although 'c is a different lifetime than 'a, a is not implicitly reborrowed; it is moved. You can tell this because uncommenting let _x = a; causes compilation to fail. However, b is not moved: it is implicitly reborrowed! This is because type analysis broadly goes left to right: once T is determined to be of the form &mut (), all other parameters of type T are subject to implicit reborrowing. All arguments are subject to lifetime subtyping, but only the first argument is moved.
On the other hand, consider this function:
fn implicit_reborrow<'a, 'b>(a: &'a mut (), b: &'b mut ()) {
fn same_type<'c>(_a: &'c mut (), _b: &'c mut ()) {}
same_type(a, b);
let _x = a; // OK
let _y = b; // OK
}
Again, the caller may choose 'a and 'b, but now same_type has two parameters of the form &'c mut (). The 'c that was notional in the last example is now explicit. The difference this makes is that now both arguments are subject to implicit reborrowing.
(You can also make the first example compile by telling the compiler that T is of the form &mut _ with a turbofish: same_type::<&mut _>(a, b).)
Inside the compiler, the way this works out is slightly different than what I said above. Notably, no lifetimes are actually resolved until after the implicit reborrow has been inserted. I'm sure it's possible to come up with an example that demonstrates that (by, for instance, introducing a borrowck error that wouldn't be detected until you fix the type error). In fact, lifetimes almost don't participate in type checking at all — the compiler initially assumes that all the lifetimes are compatible, and then borrowck comes through and checks those assumptions only after all the types have been resolved (including automatic steps like coercions and implicit reborrowing). However, I don't find arguments from compiler internals compelling; I just mention it to provide background for why the language is defined this way.
There's another difference between deref coercion and implicit reborrowing, which is that deref coercions are transitive (e.g. you can coerce from &mut Box<Box<Box<T>>> to &mut T transparently), but implicit reborrowing is not; however, since most places where reborrowing takes place are also coercion sites where deref coercion can also take place, it's not trivial to come up with an example that demonstrates this... I'm thinking one could adapt my first code example, but instead of working on that, I'm going to go to bed.
While the lifetime is an aspect of the type not all aspects of a type are resolved at the same time. If I understand @trentj's post correctly deref coercion has already taken place by the time lifetimes are even considered.
Also consider the following examples:
fn f2<'a>(x: &'a mut i32) {
let y: &'a mut i32 = x;
x; // Does not compile. Cannot move out of `x` because it is borrowed.
}
I can easily rationalize that particular compilation error because stacked borrows cannot accomodate a situation where the lifetime of x and y have to be equal - the lifetime of y has to be strictly less than the lifetime of x.
To recap:
// Example 4a - the original puzzle
fn f(z: &mut i32) {
*z += 1;
println!("{}", *z);
}
fn main() {
let mut x = 5i32; // `x` binds to a value of type i32
let y = &mut x; // `y` binds to a value of type &i32 with exclusive access to `x`
f(y);
// Rigorously applying move semantics only, the `y` binding should now be "uninitialized".
// The `&i32` value with exclusive access (i.e. `&mut i32`) has been
// moved into `f` - never to be seen again.
// However the next line of code successfully uses the `y` binding.
// If the value had been moved there should be a compilation error.
//
// Also as Example 3 showed a `&mut i32` value isn't Copy
// (i.e. has move semantics rather than copy semantics)
// even though the `i32` value is Copy (i.e. has copy semantics)
*y += 1;
println!("{}", x); // '7'
}
As @trentj outlined your confusion wasn't entirely unjustified - using generics:
// Example 4b - using parameterized types
use std::fmt::Display;
use std::marker::Sized;
use std::ops::{AddAssign, DerefMut};
fn f<T>(mut z: T, v: T::Target) where
T: DerefMut,
T::Target: Display + Sized + AddAssign {
*z += v;
println!("{}", *z);
}
fn main() {
let mut x = 5i32; // `x` binds to a value of type i32
let y = &mut x; // `y` binds to a value of type &i32 with exclusive access to `x`
f(y, 1);
// *y += 1;
// ^ uncomment the above and get the following compiler error:
// move occurs because `y` has type `&mut i32`, which does not implement the `Copy` trait
println!("{}", x); // '6'
}
The actual problem - people coming other programming languages expectExample 4a to work because that's the way references tend to work in traditional languages. In the absence of implicit borrows we would have to write:
// Example 4c - in a world without implicit borrows
fn f(z: &mut i32) {
*z += 1;
println!("{}", *z);
}
fn main() {
let mut x = 5i32; // `x` binds to a value of type i32
let y = &mut x; // `y` binds to a value of type &i32 with exclusive access to `x`
f(&mut *y);
// i.e. forcing a reborrow to make things work
*y += 1;
println!("{}", x); // '7'
}
At this point many developers coming from traditional languages would call f(&mut *y);unergonomic, even though it's entirely consistent with move semantics.
So the implicit borrow present in Example 4a is a concession to developer ergonomics - at least for functions that don't use parameterized types (there the rules are a bit different).
Aside: I think going forward I'm going to classify features like implicit borrows and deref coercion as "you know what I mean programming support".
For example, I would have viewed manual deref in the presence of type inference as a kind of programmer's dead man's switch, a checkpoint system where the compiler could zap me awake because obviously I've stopped paying attention to the details (Bugs Abhor Loneliness).