Threads in Rust

Hi,

I am new to Rust and I am trying to add up all previous numbers to a given number. To do so, I am trying to implement it with threads, since it can be an expensive operation after a million numbers. I created a Struct to hold a vector with all numbers but I am having problems with lifetimes when I try to pass this reference to the threads. I read something about Arc to do this, but it does not work.

I leave the code here in case someone can help me and explain me why it does not work as expected:

Thank you in advance.

use std::sync::Arc;
use std::thread;

pub struct Rectangle {
sides: Vec,
}

impl Rectangle {
pub fn new() -> Rectangle {
Rectangle {
sides: (1..1000000000).collect(),
}
}

pub fn accumulate(self) -> i128 {
    let mut result = 0;
    let mut futures = Vec::with_capacity(4);
    for chunk in self.sides.chunks((self.sides.len() + 3) / 4) {
        let atomic_chunk = Arc::new(chunk).clone();
        let future = thread::spawn(move || {
            for number in atomic_chunk.iter() {
                result += number;
            }
        });
        futures.push(future);
    }

    for future in futures {
        future.join();
    }
    return result;
}

}

Note that these aren't futures, they're threads. Those are two distinct concepts.

The best way to do this would probably be with rayon, which adds a ParallelIterator trait, allowing you to .par_iter().sum() and get all the plumbing for free.

The main problem is that thread::spawn requires 'static, because the lifetime of the thread is completely unconnected to the lifetime of the calling scope. This means it cannot borrow anything from the calling scope. (Rayon and crossbeam allow you to scope threads, so that you can borrow from the containing scope.)

3 Likes

@CAD97, Thank you for your reply.

I understand what you mean, but I was reading articles like: http://gradebot.org/doc/ipur/concurrency.html and it looks like it is possible to pass values to threads using Arc. I think the problem must be in the first for loop, but I still don't understand why this solution works for some cases and not for mine.

Only unbound (+Send) variables can be moved to threads. (Like mentioned using a library with scope allows bound variable to be used.) (&'static counts as unbound.)

Your chunk is a &'_[i128]. Placing it in another structure Arc does not remove the bound.

1 Like

Do you mean does not compile at all?

It would help others hep you if:
a) You put code tags (I put ``` on a line before and after the code) when posting so that it is readable.
b) Included any compiler error messages.

Anyway, if I understand correctly this can never work. You have lots of "result" variables in there. One in the accumulate() function and one local to each thread.

Consider this simplified example:

use std::thread;

    pub fn accumulate() -> i128 {
        let mut result = 0;
        let mut futures = Vec::with_capacity(4);
        for i  in 1..=4 {
            let future = thread::spawn(move || {
                for n in 0..1000_000 {
                    result += i;
                }
                println!("thread result: {}", result);
            });
            futures.push(future);
        }

        for future in futures {
            future.join();
        }
        return result;
    }

    fn main () {
         println!("accumulate result: {}", accumulate());
    }

Which produces the following result:

$ cargo run
...
2000000
1000000
3000000
4000000
0

There was already discussion about when is 'static required, when you transfer a reference rather than ownership to the thread. Compiler suggests why it might be harmful - if you place Arc on reference (which is chunk) it still does not transfer ownership over underlying data to thread:

    17 |         for chunk in self.sides.chunks((self.sides.len() + 3) / 4) {
       |                      ^^^^^^^^^^-----------------------------------
       |                      |
       |                      borrowed value does not live long enough
       |                      argument requires that `self.sides` is borrowed for `'static`
    30 |     }
       |     - `self.sides` dropped here while still borrowed

To make it working Arc should own the initial vec. Another issue in the code is access to mut result from multiple threads will lead to race condition - this either requires Mutex which is slow or calculate sum in every individual thread and return it as closure result, which can be obtained when joining threads back. See working example on playground

You could also use an atomic.

But the distributed fold is the better approach.

Yeah, or ndarray crate which was built for this specific math task and also has ndarray::parallel

Rust's threads are great, but I agree with @CAD97 that you probably want to consider using Rayon. Here's a version of your accumulate function using Rayon.


pub fn accumulate(self) -> i128 {
    use rayon::prelude::*;

    self.sides.par_chunks((self.sides.len() + 3) / 4)
        .map(|chunk| chunk.sum::<i128>())
        .sum()
}

par_chunks is an iterator over the chunks of a slice, like the one you were using originally, but it's a parallel iterator that can distribute work across multiple threads.

With each element, we map from the chunk to the sum-of-the-chunk, and then sum all the sums.

Of course, if you're trying to learn how to use threads directly, this won't help, because the threads are managed behind the scenes.

Yeah, that's what I was trying. I tried rayon after @CAD97 recommended it and it resolves the problem easily, but @dunnock 's code is what i was looking for. Thank you all :slight_smile: