So I just bumped into the crossbeam crate and how it allows scoped threads (i.e. threads that are guaranteed to be joined before exiting the scope from which they were spawned). This seems like a cool building block for an idea I have in mind.
However I worry a little about performance, as threads in general are not known to be cheap to spawn (green threads / goroutines notwithstanding). If nobody has any data on how crossbeam's scoped threads perform I'll need to benchmark that myself, but I'm hoping that someone already has this information
You are right, threads have a noticeable cost when spawning. Depending on what you are trying to do, it could be useful a thread pool or, even simpler, using rayon. The latter provides a good way of handling parallelization and it is much more fine grained than a raw thread pool (i.e. it uses a work stealing algorithm).
Rayon is indeed "the competition", and by the looks of it, it wins along these dimensions:
developer effort: Rayon is Iterator-based. This makes it easier (and often trivial, though this is not necessarily the case) to convert code written with stdlib iterators to use rayon.
initialization runtime cost: Threads have a nontrivial spawn cost, which must be paid for each thread spawned. Rayon has an upfront initialization cost too (thread pool, setting up work stealing etc) but it seems that this is paid only once.
Here is a link to the cookbook chapter on concurrency. You'll see both rayon and crossbeam in there.
They each have their uses. Rayon is well suited to data-parallel tasks, whereas crossbeam is better for just doing a different task on another thread or setting up a pipeline. I default to using crossbeam-channel now over the channels in std just because of the bound channel feature.
