Splitting vector for concurrent processing


#1

I have a long vector with data. I want to split it into equally-sized chunks and then process each chunk in a separate thread. Here’s an example:

use std::thread;
use std::sync::Arc;

fn main() {
    let data = vec![0, 1, 2];
    let data_chunks = Arc::new(data.chunks(2));
    
    let data_clone = data_chunks.clone();
    thread::spawn(move || {
        data_clone;
    });
}

However, this fails with an error saying “`data` does not live long enough”. What would be the right way to split a similar vector for processing in multiple threads?


#2

This works:

use std::thread;
use std::sync::Arc;

fn main() {
    let data = Arc::new(vec![0, 1, 2]);
    for i in 0..data.chunks(2).count()
    {
        let data_clone = data.clone();
        thread::spawn(move || {
            for _ in data_clone.chunks(2).nth(i)
            {
                //process stuff
            }
        });
    }
}

Don’t know if there is a more idiomatic way. Once thread::scoped is fixed, this will be much easier.


#3

Crossbeam has support today! http://aturon.github.io/crossbeam-doc/crossbeam/struct.Scope.html#method.spawn