I think what we discovered here is that not only the compiler would infer how to capture the variables (and consequently the trait of the closure) - with or without the move keyword - to make the body of the lambda work, it even tries to apply "deref coercion" (at compile/desugar time) (it appears) as many times as it needs, so it can capture the right variable correctly.
borrow the borrow of the borrow
The compile doesn't just intelligently capture the variable by-value - best illustrated with this example where an additional move won't save us (it has to keep applying move/deref, until it compiles.
struct ClosureWrapper<'a> {
b: Box<dyn Fn() -> () +'a>
}
fn foo<'a>(v: &'a Vec<i64>) -> ClosureWrapper<'a> {
let borrow_the_borrow = &v;
let closure = || {borrow_the_borrow.len();};
ClosureWrapper{b: Box::new(closure)}
}
fn main() {
let v = vec![1,2,3];
foo(&v);
}
Another weird part of this (undocumented?) behavior is that it compiles in the playground; but it doesn't on Compiler Explorer (https://godbolt.org/z/WjozhxGoW), or on my local machine. All tested on version 1.76.
Indeed activating the edition 2021 override (which I consider more convenient on Compiler Explorer than manually specifying the --edition argument) makes the linked code work.
And now looking at the code, the relevant difference between editions is that the updating capturing reborrows instead of taking references, and reborrow can outlast the reference it was reborrowed through.[1] It's a magical property of reborrows / references.
Just like the improved closure capturing can capture individual fields if that’s all you use of a struct, e.g. a closure using foo.x (in read-only fashion; and after custom Deref::deref calls are desugared) will borrow &foo.x, not &foo.
Similarly thus, of only *foo is used (after custom Deref::deref calls are desugared) then &*foo would be captured instead of &foo. This only applies to types where * does not desugar to explicit calls to Deref::deref, so that’s &T, &mut T and Box<T>. For example Rc cannot do this:
use std::rc::Rc;
fn size_<T>(_: &T) {
dbg!(std::mem::size_of::<T>());
}
fn main1() {
let v = Box::new([1, 2, 3]) as Box<[i32]>;
let f1 = || {
let _ = v.len();
// here, `v.len()` desugars to
// `<[i32]>::len(&*v)`, and thus only `*v`
// is accessed in this whole closure
};
size_(&f1); // ==> 16 bytes, from captured `&[i32]`
let f2 = || {
let _ = &v; // force whole v to be borrowed
let _ = v.len();
};
size_(&f2); // ==> 8 bytes, from captured `&Box<[i32]>`
}
fn main2() {
let v = &mut [1, 2, 3] as &mut [i32];
let f1 = || {
let _ = v.len();
// here, `v.len()` desugars to
// `<[i32]>::len(&*v)`, and thus only `*v`
// is accessed in this whole closure
};
size_(&f1); // ==> 16 bytes, from captured `&[i32]`
let f2 = || {
let _ = &v; // force whole v to be borrowed
let _ = v.len();
};
size_(&f2); // ==> 8 bytes, from captured `&&mut [i32]`
}
fn main3() {
let v = Rc::new([1, 2, 3]) as Rc<[i32]>;
let f1 = || {
let _ = v.len();
// here, `v.len()` desugars to
// `<[i32]>::len(&*v)`, but in there `*v` uses custom
// `Deref for Rc<T>`, further desugaring to
// `<[i32]>::len(&*Deref::deref(&v))` in which
// with `&v` the whole of `v` is borrowed!
};
size_(&f1); // ==> 8 bytes, from captured `&Rc<[i32]>`
let f2 = || {
let _ = &v; // force whole v to be borrowed
let _ = v.len();
};
size_(&f2); // ==> 8 bytes, from captured `&Rc<[i32]>`
}
fn main() {
main1();
main2();
main3();
}
In the original code example thus, the “smarter” capturing does not apply any auto-deref during capturing.
Auto-deref happens before analysis of what’s captured, anyways:
borrow_the_borrow.len();
desugars to
<Vec<i64>>::len(&**borrow_the_borrow)
Note that this is fully desugared, as the * dereferences here are both for &T references, so these are built-in operations, not using Deref::deref.
Subsequent analysis for what needs to be captured then can deduce that for access to borrow_the_borrow, only **borrow_the_borrow is ever accessed (and in a read-only manner), so the closure captures include a shared immutable borrow of **borrow_the_borrow, which is a re-borrow &**borrow_the_borrow. This captured re-borrow then, for borrow-checking, is determined to be allowed to live for as long as the original v: &'a Vec<i64>, and no borrow-checking error occurs.
Note that this is my personal mental model of how the compiler handles these. I haven’t checked if this is actually true, or whether it does something else, like e.g. closure captures could be determined earlier and the logic re-implements some reasoning about the derefs and desugarings, or any other equivalent compiler implementation is possible.
If they're documented anywhere, I don't know where. I learned about that one from this comment (expand the collapsed part). Note that when it talks about closure captures, it's talking about 2018 captures until the "what about RFC 2229" section near the end (which is also where rustc_capture_analysis is mentioned/explained).
(If you keep reading the issue, it turns out they could get rid of the nested closure argument, using some relatively advanced lifetime bound gymnastics.)