Can't move a closure into a spawned thread

help
1

This simple function won't compile. Please help me understand why! (I'm still relatively new to Rust!)

fn foo<F: Fn() + Send>(f: F) {
    thread::spawn(move || (f)() );
}

The full error is:

error[E0310]: the parameter type `F` may not live long enough
 --> src/main.rs:7:5
  |
7 |     thread::spawn(move || f);
  |     ^^^^^^^^^^^^^^^^^^^^^^^^
  |     |
  |     the parameter type `F` must be valid for the static lifetime...
  |     ...so that the type `F` will meet its required lifetime bounds
  |
help: consider adding an explicit lifetime bound
  |
5 |     F: Fn() + Send + 'static,
  |                    +++++++++

For more information about this error, try `rustc --explain E0310`.

(In my "real" code, I don't think I can use a 'static lifetime.)

In my (limited) understanding, f should be fully owned by the newly spawned thread. f should be moved onto the thread's stack, and so should only have to live as long as the thread lives. So I don't understand the error!

This (even simpler!) code doesn't work either (with the same error):

fn foo<F: Send>(f: F) {
    thread::spawn(move || f );
}

In contrast, something like this works fine:

fn foo(v: Vec<u8>) {
    thread::spawn(move || v);
}

(For completeness, what I'm actually trying to do in my "real" code is more like this:)

fn foo<F: Fn(Context) + Send + Clone>(f: F) {
    let mut handles = Vec::with_capacity(4);
    for _ in 0..4 {
        let context: Context = get_context();
        let f_clone = f.clone();
        let handle = thread::spawn(move || (f_clone)(context) );
        handles.push(handle);
    }
}
1 Like
2

The problem is that, as far as the compiler is concerned, threads started via spawn can potentially run until the program exits. This requires the ’static bound to prevent potential use-after-free issues.

To start a thread that needs access to non-’static data, you’ll need to use something like thread::scope instead.

Edit: For example,

fn foo<F: Fn(Context)->String + Send + Clone>(f: F) {
    thread::scope(|s| {
        let mut handles = Vec::with_capacity(4);
        for _ in 0..4 {
            let context: Context = get_context();
            let f_clone = f.clone();
            let handle = s.spawn(move || (f_clone)(context) );
            handles.push(handle);
        }
        for h in handles {
            dbg!(h.join().unwrap());
        }
    });
}
2 Likes
3

You must add + 'static bound. When you allow any type F, this includes allowing a temporary reference that will be invalidated as soon as your function returns.

Send alone doesn't promise that data will live for as long as any thread.

5 Likes
4

Have you tried the suggestion that the compiler is giving you?

5

Thanks so much for the very quick replies!

Unfortunately, I can't use thread::scope because I need my function foo to return immediately (after creating a threadpool that runs in the background).

Have you tried the suggestion that the compiler is giving you?

I had originally thought that I couldn't use 'static (because I thought that 'static would have unacceptable consequences for the code that calls foo). But it sounds like the only answer is to use 'static and to deal with the consequences on the calling code!

Thanks again!

6

For completeness, this works, and is even closer to my "real" code:

use std::{
    sync::mpsc,
    thread::{self, JoinHandle},
};

fn run_closure_on_multiple_threads<F>(f: F) -> Vec<JoinHandle<()>>
where
    F: Fn() + Clone + Send + 'static,
{
    const N_THREADS: usize = 4;
    (0..N_THREADS)
        .map(|_| {
            let f_clone = f.clone();
            thread::spawn(move || (f_clone)())
        })
        .collect()
}

fn main() {
    let captured_string = String::from("hello");
    let (tx, rx) = mpsc::channel();

    // Spawn threads:
    let handles = run_closure_on_multiple_threads(move || {
        tx.send(format!(
            "{} from {:?}!",
            captured_string,
            thread::current().id()
        ))
        .unwrap()
    });

    // Join threads:
    handles
        .into_iter()
        .for_each(|handle| handle.join().unwrap());

    // Print contents of the channel:
    rx.into_iter().for_each(|s| println!("{s}"));
}

This code prints out:

hello from ThreadId(2)!
hello from ThreadId(4)!
hello from ThreadId(5)!
hello from ThreadId(3)!

Thanks again for the very quick replies!

1 Like
7

You can see the signature of thread::spawn:

pub fn spawn<F, T>(f: F) -> JoinHandle<T>
where
    F: FnOnce() -> T + Send + 'static,
    T: Send + 'static,

It needs F to be 'static.

In your code above, you actually want Joinhandles to be joined immediately after created, this is better to be implemented by thead::scope.

Maybe you are actually questioning whether 'static is essencial for thread::spawn.

/// - The `'static` constraint means that the closure and its return value
///   must have a lifetime of the whole program execution. The reason for this
///   is that threads can outlive the lifetime they have been created in.
///
///   Indeed if the thread, and by extension its return value, can outlive their
///   caller, we need to make sure that they will be valid afterwards, and since
///   we *can't* know when it will return we need to have them valid as long as
///   possible, that is until the end of the program, hence the `'static`
///   lifetime.

The fact is thread::spawn is designed return a JoinHandle<T>(JoinInner<'static, T>), and F spawned may be running in the whole lifetime of the program and never be joined.

If you digging into the source code of thread::spawn, you may see thread::spawn_unchecked_.

    /// # Safety
    ///
    /// The caller has to ensure that the spawned thread does not outlive any
    /// references in the supplied thread closure and its return type.
    /// This can be guaranteed in two ways:
    ///
    /// - ensure that [`join`][`JoinHandle::join`] is called before any referenced
    /// data is dropped
    /// - use only types with `'static` lifetime bounds, i.e., those with no or only
    /// `'static` references (both [`thread::Builder::spawn`][`Builder::spawn`]
    /// and [`thread::spawn`][`spawn`] enforce this property statically)

If you believe in yourself, enable #![feature(thread_spawn_unchecked)].

The drawbacks of 'static F:
Edit: Thanks to @kornel
Since you always use move to create a closure, this ensures capturing all the ownerships the closure needs, the closure doesn't capture any references with a lifetime shorter than 'static , it can be used in contexts where a 'static lifetime is required, such as passing it to thread::spawn.

The behavior of closure is quite hard to learn, something above may be wrong, feel free to let me know!

1 Like
8

move doesn't extend any lifetimes. Ownership and lifetimes are orthogonal. For example &mut must be moved to remain mutable, but that doesn't make its target live any longer. You can own MutexGuard<'_> or Cow<'tmp>, but end up with dangling pointers if you force them to live for an arbitrary lifetime.

Note that any lifetime bounds, including 'static, apply only to references and types containing references. They do nothing when applied to self-contained types. This means that String is not 'static, but rather it isn't affected by any lifetime bound. And on the other hand, owning a type doesn't mean you control its lifetime. You can own Box<&'tmp i32>, but not be able to use it beyond the 'tmp scope of its content.

5 Likes
10

Thank you, all, for a super-helpful conversation. This has really helped clear up a silly misunderstanding of mine!

The root cause of my confusion was that I didn't understand what 'static meant in the context of fn foo<F: Fn() + 'static>(f: F).

I had (mistakenly) imagined that foo would only accept closures which capture 'static variables, such as string literals (&'static str), or static BAR: Vec<u8> = vec![0, 1, 2]. So, for example, I didn't understand why rustc allows a (non-static) String to be moved into the closure that's passed to foo (because the String isn't static).

But, if I've understood correctly, Fn() + 'static is actually quite simple: It just means that the closure can't capture references, unless those references are 'static. It's perfectly fine to move owned types (such as a Vec or String) into the closure (as long as those types don't borrow non-'static references). The penny really dropped for me when I read kornel's last paragraph, especially: "lifetime bounds, including 'static , apply only to references and types containing references. They do nothing when applied to self-contained types.".

Thank you again!

If anyone else has a similar confusion to me, and stumbles into this thread in the future, then here are some relevant links that also helped me:

3 Likes
11

This seems to be very poorly documented. The only place I know of that explicitly describes the two different uses of 'static lifetimes (for references and trait bounds) is in rust-by-example.

2 Likes
12

This is another good resource for these sorts of questions/ideas.

2 Likes
13

"Doesn't capture borrows" is a very good mental model. There are exceptions but they're pretty rare/niche.

There's a brief blurb here too, though you have to make the leap from "no non-'static[1] lifetimes in the fully-elaborated type" to "this is a property of types that has nothing to do with the liveness of values or ownership per se".

My notes on that RBE page:

By "all references in T" they mean that bounds are recursive on their generic parameters.

Foo<'a, 'b, T>: 'x

holds if and only if all of these hold:

'a: 'x, 'b: 'x, T: 'x,

(Note that not all lifetime-carrying types actually contain references.)

  1. and non-higher-ranked ↩︎

1 Like
14

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.