Safe, Stable and Scoped Threadpool


#1

A bugfix that enabled the proposed v2 of the scoped thread API recently
landed, so its now possible to write libraries for scoped threads.

This is my take on it, a scoped threadpool that runs on stable rust:

https://crates.io/crates/scoped_threadpool

(Reddit discussion: https://www.reddit.com/r/rust/comments/3hyv3i/safe_stable_and_scoped_threadpool/)

PSA: Only has a safe interface on nightly right now, see reddit thread.

Features:

  • Thread creation is independent from job execution, so
    you can cache the pool itself, even in a global variable. (It combines nicely with lazy_static for example)
  • Safe interface due to not using RAII for joining threads. (If there
    is other unsafety with scoped threads in general I’m not currently aware
    of it)
  • Stable interface due to using thread::spawn internally. (The only unsafe block is for transmuting a job to 'static lifetime)
  • Jobs can access data of any lifetime outside of the scope during which they run, re-enabling parallel processing of stack data.

This is just the bare bone functionality right now, and no care has
been given yet to optimize it in particular (and I’m missing the
expertise for it), so feel free to offer suggestions for improvements.
:slight_smile:


#2

Cool! Is there any chance this could be merged with (or the other way around) the threadpool crate? It would be really nice to be able to re-use the same threadpool for both scoped and non-scoped (static lifetime) threads.


#3

Sure, what feature are you missing specifically? :slight_smile:


#4

pool.execute(...) instead of pool.scoped(|scope| scope.execute(...)).


#5

I think the goal would be to have one, awesome, threadpool crate. Lots of similar crates should be avoided if possible.


#6

A healthy ecosystem needs to have a bit of competition. While only having one crate that does a thing is conveninet in the short term, I think it hurts growth and leads to a worse situation in the long term.


#7

I didn’t say competition is bad. But collaboration is probably even better!