Struggling to sync Mutex<Vec> between threads

Goal and environment

I have been following the book, and wanted to share a Vec between threads so that some integers are pushed into it in somewhat arbitrary order. I tried Vec, Arc, Mutex for this.

The environment I am using is

OS: MacOS 12.5.1 on M1 chip
Shell: zsh 5.8.1 (x86_64-apple-darwin21.0)
Compiler: rustc 1.64.0 (a55dd71d5 2022-09-19)
Cargo: cargo 1.64.0 (387270bc7 2022-09-16)

Code tested

Here is my code.

use std::sync::{Arc, Mutex};
use std::thread;

fn main() {
    let contents = Arc::new(Mutex::new(vec![]));

    for i in 0..10 {
        let contents = Arc::clone(&contents);

        thread::spawn(move || {
            let mut contents = contents.lock().unwrap();

            contents.push(i);
        });
    }

    println!("Contents: {:?}", contents);
}

Result & expected

I expected all 10 integers to be in contents, albeit random order.
However the result sometimes missses some integers, as follows.

thread % for i in $(seq 0 10); do target/debug/thread; done
Contents: Mutex { data: [0, 1, 2, 3, 4, 5, 6, 7, 8], poisoned: false, .. }
Contents: Mutex { data: [0, 1, 2, 4, 6, 3, 5, 8], poisoned: false, .. }
Contents: Mutex { data: <locked>, poisoned: false, .. }
Contents: Mutex { data: [0, 1, 2, 4, 3, 5, 6, 7, 8], poisoned: false, .. }
Contents: Mutex { data: [0, 1, 2, 3, 5, 4, 6, 7, 8], poisoned: false, .. }
Contents: Mutex { data: [0, 1, 2, 3, 4, 5, 7, 6], poisoned: false, .. }
Contents: Mutex { data: [0, 2, 4, 6, 1, 7], poisoned: false, .. }
Contents: Mutex { data: [0, 1, 2, 3, 5, 4, 6, 7, 8], poisoned: false, .. }
Contents: Mutex { data: [0, 1, 2, 3, 4, 5, 6, 7, 8], poisoned: false, .. }
Contents: Mutex { data: [0, 1, 3, 2, 5, 6, 7, 8, 4], poisoned: false, .. }
Contents: Mutex { data: [0, 1, 2, 3, 4, 5, 6, 7, 8], poisoned: false, .. }

I thought Vec::push might take so long that it sometimes does not finish even after the Mutex goes out of scope, and added short sleep at the end of the closure, as follows.

use std::sync::{Arc, Mutex};
use std::thread;
use std::time::Duration;

fn main() {
    let contents = Arc::new(Mutex::new(vec![]));

    for i in 0..10 {
        let contents = Arc::clone(&contents);

        thread::spawn(move || {
            let mut contents_local = contents.lock().unwrap();

            contents_local.push(i);

            thread::sleep(Duration::from_millis(1000));
        });
    }

    println!("Contents: {:?}", contents.lock().unwrap());
}

However the problem persists. As sleeps of 1, 10 ms did not help, I tried with 1 second in each thread but still, some of the integers are missed, as follows.

thread % for i in $(seq 0 10); do target/debug/thread; done
Contents: [0, 1, 2, 3, 4, 5, 6, 7, 8]
Contents: [0, 1, 2, 4, 3, 5, 6, 7, 8]
Contents: [0, 1, 2, 3, 4, 5, 6, 7, 8]
Contents: [0, 1, 2, 3, 4, 5, 6, 7, 8]
Contents: [0, 1, 3, 2, 4, 5, 6, 7, 8]
Contents: [0, 1, 3, 2, 4, 5, 6, 7]
Contents: [0, 1, 2, 3, 4, 5, 6, 7, 8]
Contents: [0, 1, 2, 3, 4, 5, 6, 7, 8]
Contents: [0, 1, 3, 4, 2, 5, 6, 7, 8]
Contents: [1, 0, 3, 5, 4, 2, 6, 7]
Contents: [0, 1, 3, 2, 4, 6, 5, 7]

Due diligence

I wanted to resolve the issue myself but I have been unable to search for relevant problems, particularly for a novice like me. I have searched google and this forum for some combinations of sync, vec, mutex, thread-safe with no success yet. Any link may be of help to me.

Your main thread is exiting before the worker threads all finish. You should just use a Vec to store all of the join handles from each thread::spawn and join all of them at the end of main so you can guarantee all of the threads you spawned finish.

You could also put a sleep at the end of the main function, but you're still gambling on thread scheduling not screwing that up, so you should avoid that.

2 Likes

Thanks @semicoleon !
I have added joining the handles and now it works. Great!

While you got a solution, I still want to point out another misunderstanding.

That's impossible by construction.

The call to push is inside the worker thread. This means that the worker thread has a reference to the vector. This in turn means that it has a copy of the Arc (otherwise lifetimes would be invalid and the program would be unsound – this is prevented by the 'static bound on the worker callable). But if the thread has a copy of the Arc, then that will keep the mutex and its contents alive at least until the Arc is dropped. And the particular copy of the Arc isn't dropped until the end of the worker closure, which isn't reached until Vec::push() returns. So if Vec::push() were to take a long time, that would simply cause the mutex to be locked for a longer time – nothing else. It couldn't "go out of scope" while the push is happening.

1 Like

Hi @H2CO3 ,
Thank you for carefully reading my post and correcting another misunderstanding.
Your answer helped me understand the inner working more, and although I still do not get what the part (otherwise lifetimes would be invalid and the program would be unsound – this is prevented by the 'static bound on the worker callable) means exactly, the rest makes full sense to me. Someday I will get a good grasp of the lifetimes.
Many thanks!

Since a thread created by thread::spawn() may live for an arbitrarily long time, it's invalid for it to have any references to its environment. The function creating the thread might have long returned when the thread is still running, which would cause any captured references to be dangling. The type parameter of thread::spawn(), representing the worker callable, has a 'static lifetime bound (you should read the signature of all functions you are using!), which prevents it from capturing any non-owning references.

1 Like