Worker threads and 'global' work queue


#1

This is my first attempt at threading in Rust. I’m trying to create a number of worker threads and a single control thread.

The simple map/reduce examples I’ve seen so far don’t deal with a central work queue, they hand off work as the thread is created and then join all the child threads.

let foo: Vec<i32> = Vec::new();
let data = Arc::new(Mutex::new(foo));

There is a vec holding work to be done. Each worker thread locks the vec, checks for work and if it finds anything takes it and starts processing it. If there is no work the thread sleeps for a second before checking again.

This is how I am creating my worker threads:

for i in 0..5
{
    let dd = data.clone();
    let index = i;

    thread::spawn(move || 
    {
        println!("spawned thread {}", index);

        loop
        {
            let mut d = dd.lock().unwrap();

            if d.len() == 0
            {
                println!("no work for thread {}, sleeping", i);
                thread::sleep(Duration::from_secs(1));
            }
            else
            {
                let x: i32 = d.pop().unwrap();

                println!("thread {} has work!  {}", index, x);
                thread::sleep(Duration::from_secs(x as u64));
                println!("thread {} work complete!", index);
            }
        }
    });
}

The control thread just posts work into the vec every second.

I’ve posted my current attempt in the playground here (playground won’t run it because it takes too long):
https://play.rust-lang.org/?gist=98c65f2600a3e2a34aa163d4d919a4de&version=stable

Are my locks for the vec not being released inside the worker thread loop? I tried adding additional braces for scope, but it didn’t seem to make a difference.

I had to clone() the data for each thread. At first I thought each thread was getting its own copy of the underlying vec, but it does appear to be a single shared copy. (I think, still not 100% convinced on this?)

What am I doing wrong?


#2

You need to drop(d) the MutexGuard if you’re going to sleep. It’s dropped automatically at the end of the scope but that happens after you sleep.


#3

Thanks! Can’t believe I missed that.