Workerpool with individual resources

Hi everyone,

I have an object with an Iterator trait that always hand me a reference to the same Vector, only the content of said vector changes. I'd like then do some parallel computation based on the vector's data.

I'd like to do so while reducing the amount of memory allocated, and avoid any vector copying, so here is what I'm envisioning:

  • create a number n of workers
  • each worker has its own local vector
  • at each iteration, a worker is fed a reference to the iterator's vector, and do an elementwise copy of the iterator's vector to its own local vector
  • the worker can then do its computation on its own local vector

How should I go about this?

Have a look at rayon if that can help you:

https://crates.io/crates/rayon

On keeping memory consumption down, rust allows you to set stack size for threads. Not sure if rayon let's you configure that though.

This is technically impossible. As soon as the worker has a reference to the vector, the vector cannot be mutated in the root scope. You'd have to create the copy in the root thread and then hand the ownership of the copy to the other thread.

Maybe it'd possible to do the copy in the root scope? The worker would hand a mutable reference to its vector through a channel, and not the other around?

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.