I’m trying to understand what has been said in this discussion, it’s not entirely easy to read, so excuse me in case I’m repeating anything that has already been said…
But I need to comment on this: I would like to disagree with this phrasing
The 'env: 'scope bound is not enforced on but provided to the user of the thread::scope API. The 'env lifetime is introduced first, then the 'scope lifetime is introduced and 'env: 'scope restricts the latter.
Here’s my take at explaining this in more detail, which, I guess, has – now that I’ve written it – turned into an almost complete (short) account of my interpretation of how and why thread::scope API works and what language mechanisms are involved it its soundness.
Below, the terms “caller” and “callee” generally refer to the caller and callee of the std::thread::scope(…) function in particular; but I also imagine the “caller” as the “API user of the scoped thread API” in general.
The lifetime 'scope is not a bound/restriction on a caller-chosen lifetimes in the call to thread::scope. Instead, the 'scope lifetime is callee-chosen, as it comes from a for<'scope> FnOnce(…) -> … higher-ranked trait bound. Restrictions on a callee-chosen lifetime does not enforce anything on the caller, but instead provides them with additional capabilities, in particular, being provided with the 'env: 'scope bound, it’s possible to convert &'env T references into &'scope T references, or maybe simply – without coercing – it just allows you to know/prove the bounds like &'env T: 'scope.
Where exactly does the 'env: 'scope bound appear in the signature of thread::scope(…)? There is an (implicit) 'env: 'scope bound comes, as an implied bounnd, from using certain types: the Scope<'scope, 'env> type itself has a 'env: 'scope bound, and there’s also the reference type &'scope Scope<'scope, 'env> that implies a Scope<'scope, 'env>: 'scope bound, too, and thus ultimately 'env: 'scope. This implied bound does weaken the higher-ranked trait bound:
for<'scope> FnOnce(&'scope Scope<'scope, 'env>) -> T
It thus relieves the caller a little bit, as the caller of std::thread::scope(…) will no longer need to ensure the closure fulfills a certain FnOnce bound for all lifetime 'scope, but only for those lifetimes 'scope that fulfill 'env: 'scope.
What does enforce that “nothing captured becomes invalid before all the threads are joined”, though? This restriction come from two factors: The 'scope lifetime is callee-chosen, so the user of the thread::scope-API can not influence what this lifetime is. Intuitively, the 'scope lifetime does thus accurately describe the true scope/duration of the thread::scope call, up to the point where the threads are all joined and terminated. The API detail that enforces something then is the signature of Scope::scope(…) which comes with a F: 'scope bound.
Of course the API does not encode in any way the compiler can understand the precise details of “this 'scope lifetime will be chosen exactly to be the one that lasts until all the threads are joined, right before thread::scope returns”. This kind of thing is too complicated for type signatures.
Instead, by using a for<'scope> … HRTB, an arbitrary callee-chosen lifetime is introduced, which a priori would mean the caller needs to be handle any choice of lifetime, including the one that happens to accurately represent the lifetime that “that lasts until all the threads are joined”.
Demanding the caller to handle any choice of lifetime is too restrictive. After all, the choice could have been 'scope == 'static; with a fully unrestricted callee-chosen lifetime 'scope, the F: 'scope bound becomes (almost) as restrictive as F: 'static. This can be demonstrated easily:
use core::marker::PhantomData;
// a simpler `Scope` without using any `'env` and thus without any `'env: 'scope` bounds
pub struct Scope<'scope>(
// marker for invariant lifetime, mirroring what `std` does
PhantomData<&'scope mut &'scope ()>,
);
impl<'scope> Scope<'scope> {
pub fn spawn<F, T>(&'scope self, f: F) -> ()
// we don't need to return a join handle for this demo
where
F: FnOnce() -> T + Send + 'scope,
T: Send + 'scope,
{
unimplemented!() // we only care about the signature
}
}
pub fn scope<'env, F, T>(f: F) -> T
where
F: for<'scope> FnOnce(&'scope Scope<'scope>) -> T,
{
unimplemented!() // we only care about the signature
}
fn demonstration1() {
let x = 1;
// compiles
std::thread::scope(|s| {
s.spawn(|| {
println!("{x}");
});
});
}
fn demonstration2() {
let x = 1;
// fails
scope(|s| {
s.spawn(|| {
println!("{x}");
});
});
// error message contains hint:
// argument requires that `x` is borrowed for `'static`
}
The error message will even call out 'static, despite there being no explicit F: 'static requirement, instead there’s a F: 'scope requirement for an unrestricted callee-chosen lifetime 'scope. The only thing we know about the 'scope lifetime is that the &'scope Scope<'scope> reference lives long enough, but for any other reference, such as one to x, the F: 'scope bound is essentially as restrictive as F: 'static.
In order for the thread::scope lifetime to be useful, the API teaches us more about the 'scope lifetime than “it’s any arbitrarily chosen lifetime”. Instead it gives the 'env: 'scope implied bound, which provides the information “the 'scope lifetime is any lifetime, but it will be at most as large as 'env”. While previously we were only able to deduce F: 'scope for the &'scope Scope<'scope> reference, and other 'static references, not every &'env T reference will qualify, too.
And 'env is caller-chosen, so we can choose it as small as we need, in particular small enough for a &'env i32 reference in demonstration1 to qualify. But what is to stop us from choosing 'env to be too small? Is there any restriction left at all!? We are the caller for thread::scope; why couldn’t we just chose “'env is small enough so that borrowing x for 'env works” in demonstration3 below?
fn demonstration3() {
// fails
std::thread::scope(|s| {
let x = 1;
s.spawn(|| {
println!("{x}");
});
});
}
The answer is: Caller-chosen lifetimes have one important restriction, too.
If you define a function like
fn foo<'a>(r: &'a u8) {}
The caller of foo can chose almost any lifetime for 'a. But the lifetime 'a has to last at least as long as the call to foo! This is a universal rule in Rust that is always enforced: Lifetimes that are lifetime parameters to a function last as long as the duration of the function! And it makes sense, after all, in a function like foo, the argument r is usable for the entire duration of the function, so it’d better be alive the whole time.
This is the missing piece: The 'env lifetime is a caller-chosen lifetime parameter of the function thread::scope, and thus it must be at least as long as the entire time until thread::scope returns. This is what makes thread::scope’s API sound: No matter what lifetime the caller chooses for 'env, it’s always true that the lifetime 'env lasts longer than the call the thread::scope, while the duration “until all the threads are joined” is always shorter than the call to thread::scope. Hence the 'env: 'scope bound is correct. 'env outlives the time “until all the threads are joined” which is the intended meaning for 'scope.
