Why thread::Scope need an 'env lifetime parameter

help
1

The std::thread::Scope is defined as this:

pub struct Scope<'scope, 'env: 'scope> {
    data: Arc<ScopeData>,
    /// Invariance over 'scope, to make sure 'scope cannot shrink,
    /// which is necessary for soundness.
    ///
    /// Without invariance, this would compile fine but be unsound:
    ///
    /// ```compile_fail,E0373
  /// std::thread::scope(|s| {
  ///     s.spawn(|| {
  ///         let a = String::from("abcd");
  ///         s.spawn(|| println!("{a:?}")); // might run after `a` is dropped
  ///     });
  /// });
    /// ```
    scope: PhantomData<&'scope mut &'scope ()>,
    env: PhantomData<&'env mut &'env ()>,
}

I wonder can it be defined just as:

pub struct Scope<'scope> {
    data: Arc<ScopeData>,
    /// Invariance over 'scope, to make sure 'scope cannot shrink,
    /// which is necessary for soundness.
    ///
    /// Without invariance, this would compile fine but be unsound:
    ///
    /// ```compile_fail,E0373
    /// std::thread::scope(|s| {
    ///     s.spawn(|| {
    ///         let a = String::from("abcd");
    ///         s.spawn(|| println!("{a:?}")); // might run after `a` is dropped
    ///     });
    /// });
    /// ```
    scope: PhantomData<&'scope mut &'scope ()>,
}

the 'scope seems to be the function std::thread::scope body's syntax block.

Then

let n = 9;
std::thread::scope(|s| {
     println!("{}", n);
 });

The n in println!("{}", n); must have held a lifetime longer than 'scope by nature. and in the view of the caller, std::thread::scope is a sync function --- any thread will terminate before the call returns, so the 'env is not necessary.

1 Like
2

Let's see.. the relevant API are of course

pub fn scope<'env, F, T>(f: F) -> T
where
    F: for<'scope> FnOnce(&'scope Scope<'scope, 'env>) -> T,

as well as

impl<'scope, 'env> Scope<'scope, 'env> {

pub fn spawn<F, T>(&'scope self, f: F) -> ScopedJoinHandle<'scope, T>
where
    F: FnOnce() -> T + Send + 'scope,
    T: Send + 'scope,

}

On first glance, 'env indeed seems completely unused. Let's create a stub to work with this practically:

use std::marker::PhantomData;

pub struct Scope<'scope, 'env> {
    scope: PhantomData<&'scope mut &'scope ()>,
    env: PhantomData<&'env mut &'env ()>,
}

pub fn scope<'env, F, T>(f: F) -> T
where
    F: for<'scope> FnOnce(&'scope Scope<'scope, 'env>) -> T,
{
    todo!()
}

impl<'scope, 'env> Scope<'scope, 'env> {
    pub fn spawn<F, T>(&'scope self, f: F) -> std::thread::ScopedJoinHandle<'scope, T>
    where
        F: FnOnce() -> T + Send + 'scope,
        T: Send + 'scope,
    {
        todo!()
    }
}

fn not_main() {
    let n = 9;
    scope(|s| {
        println!("{}", n);
    });
}

Rust Playground

Works great! Next, what happens if we remove the 'env parameter?

use std::marker::PhantomData;

pub struct Scope<'scope> {
    scope: PhantomData<&'scope mut &'scope ()>,
}

pub fn scope<F, T>(f: F) -> T
where
    F: for<'scope> FnOnce(&'scope Scope<'scope>) -> T,
{
    todo!()
}

impl<'scope> Scope<'scope> {
    pub fn spawn<F, T>(&'scope self, f: F) -> std::thread::ScopedJoinHandle<'scope, T>
    where
        F: FnOnce() -> T + Send + 'scope,
        T: Send + 'scope,
    {
        todo!()
    }
}

fn not_main() {
    let n = 9;
    scope(|s| {
        println!("{}", n);
    });
}

Rust Playground

Aaaand... it still compiles! Maybe this whole 'env thing really is unnecessary? But wait! We didn't even use spawn yet! Back to the first version, let's spawn something!

fn not_main2() {
    let n = 9;
    scope(|s| {
        s.spawn(|| {
            println!("{}", n);
        });
    });
}

So the above still works fine in the version with 'env.

But if you copy it over to the other version, we get a compilation error. Surprising, isn't it?

error[E0597]: `n` does not live long enough
  --> src/lib.rs:28:28
   |
25 |       let n = 9;
   |           - binding `n` declared here
26 |       scope(|s| {
   |             --- value captured here
27 | /         s.spawn(|| {
28 | |             println!("{}", n);
   | |                            ^ borrowed value does not live long enough
29 | |         });
   | |__________- argument requires that `n` is borrowed for `'static`
30 |       });
31 |   }
   |   - `n` dropped here while still borrowed

For more information about this error, try `rustc --explain E0597`.

so we can conclude that 'env was somehow necessary.

Why is it requiring 'static all the sudden though?

You are correct in stating that in principle, a single 'scope lifetime should suffice. The main role 'env currently plays in the API is that it appears in the function signature of scope:

pub fn scope<'env, F, T>(f: F) -> T
where
    F: for<'scope> FnOnce(&'scope Scope<'scope, 'env>) -> T,

and what it does here is equally subtle and crucial, and relates to how HRTBs work in Rust. The HRTB (higher-ranked trait bound) of for<'scope> FnOnce(&'scope Scope<'scope, 'env>) -> T, comes with some implied bounds. The type &'scope Scope<'scope, 'env> has a validity requirement of Scope<'scope, 'env>: 'scope (as a special case of the general validity requirement of T: 'a for the type &'a T). This boils down to the implied bound 'env: 'scope.

An implied bound in a HRTB acts as a restriction. Instead of asking the closure to work for literally all lifetimes “'scope”, it only requires it to support those lifetimes that meet the implied bound, 'env: 'scope, so only lifetimes up to at most 'env.

Now, inside of the scope-closure, this 'scope lifetimes is placed on the spawned closure, which can only capture borrows that live at least as long as 'scope. For the outer closure to support all lifetimes, it could only capture lifetimes that are 'static, because 'scope == 'static would be one permitted subsitution for making the higher-ranked bound concrete. With the restriction using 'env, it can support borrowing shorter-lived lifetimes only living as long as 'env, too.

Relying on this implied-bounds handling on HRTBs is indeed feeling slightly hacky. Really, one might prefer if there was just special syntax for this, e.g.
for<'scope, where 'env: 'scope> FnOnce(&'scope Scope<'scope>) -> T,
or something like that. The 'env lifetime wouldn't actually need to be part of the Scope<…> type at all. (You could even still do that just now, by instead giving the closure some second dummy: &'scope Dummy<'env> argument that creates the same implied bound; but that would be more annoying to work with, you'd want to just write scope(|s| …) not scope(|s, _| …) every time you use this API, right? So having the parameter on Scope<…> instead is the more ergonomic API design here.

You are correct in asserting that the single lifetime 'scope would be enough if the intuitive observation that “the 'scope is the function std::thread::scope body's syntax block” was true; but there is no magic here, just function signature. And we don't have magical syntax for saying “this lifetime must correspond exactly to the duration that this function is called”, so the 2-lifetimes solution as it stands now is the best approximation to make it work with the tools available, i.e. in a way the compiler can “understand”.

As a follow-up question, you might wonder “why do we need 'scope then, why do we need an HRTB at all, can't we just use only 'env everywhere?”

That question has a different answer. IIRC, the main thing the HRTB here gives us is the guarantee that the scope cannot possibly return its Scope handle to the outside, nor any of the ScopedJoinHandles; and not leaking those to the outside is relevant for soundness.

16 Likes
3

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.