Simple / trivial thread work queue?

Without pulling in all of rayon / crossbeam, is there a quick / simple way to do:

  1. here is a work queue
  2. start N threads, do work, return value
  3. wait for all threads to finish

?

Addendum: Basically, something like:

  let x: Vec<T> = ...;
  let y = x.par_map(f, num_threads).collect::<Vec<_>>();

Make a vector of jobs, put it in an Arc/Mutex, spawn N threads, and have each thread keep popping items from the vector until it finished. To get back the items, probably use a channel (or return a vector of results from each thread), though you'll need separate handling if you need them in the same order as they were in originally.

1 Like

Assume:

  • f: T -> U
  • x: Vec<T> (input)
  • y: Vec<U> (output)
  • U does not implement default

is there a nice way to preallocate y, then make each job take a (&T, &mut Uninit<U>) so it directly writes to the right location ?

On nightly, there's Vec::spare_capacity_mut()

I am thinking of maybe going with Vec in std::vec - Rust

pub unsafe fn from_raw_parts(
    ptr: *mut T,
    length: usize,
    capacity: usize
) -> Vec<T, Global>

The question here now is, how do I even do a "malloc" for 'n' elements of 'T' (with proper alignment) ?

Ah, probably Layout in std::alloc - Rust from Allocating - The Rustonomicon

You can just do Vec::with_capacity and then use raw pointers to make the uninit references.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.