Idiomatic way of threads mutating sub-slices

1

Alright, I'm having a heck of a time getting Rust to be happy here...

Let's purport that we have a Vec which is kept alive by a Arc. I'd like for N threads to operate on Vec::len() / N non-aliasing sub-slices.

I know that Vec has a chunks_exact_mut() but I'm having a hard time working with the borrow checker here.

Any advice on how to get this to compile?

2

You can use crossbeam::scope for accessing non-'static types in new threads.

pub fn main() {
    let mut v = vec![0; 1024];

    crossbeam::scope(|s| {
        v.chunks_exact_mut(1024 / 4).for_each(|chunk| {
            s.spawn(move |_| {
                chunk.iter_mut().for_each(|v| {
                    *v = 1337;
                });
            });
        });
    }).unwrap();
    // all threads are automatically joined at the end of the `scope`
    
    dbg!(&v);
}

(playground)

3 Likes
3

Heh, I'll have to dive into that dependency later.

I'm learning Rust for fun in my free time so I was hoping to do this without using any external crates.

In C++, this would all be so trivial...

4

Another, arguably more ideomatic approach is to use the rayon crate, e.g.

use rayon::prelude::*;

pub fn main() {
    let mut v = vec![0; 1024];
    
    println!("using {:?} threads", rayon::current_num_threads());

    v.par_iter_mut().for_each(|v| {
        *v = 1337;
    });

    dbg!(&v);
}

(playground)

2 Likes
5

Unfortunately, the standard library does not contain the necessary primitives to do this in safe Rust. It used to have a Crossbeam-like “scoped thread” feature, but this was removed shortly before the Rust 1.0 release because of a fatal design flaw, discovered at the last minute. Replacements were developed in external crates, but haven't been moved back into the standard library (yet?).

Also note that the rayon and crossbeam developers include many members of the Rust team, so while they are not part of the Rust toolchain, they are not exactly third-party either.

7 Likes
6

Rust is deliberately a "batteries not included" language, so many things require use of external crates. std is maintained in lock sync with the rustc compiler, because std is the only crate that can access and use unstable compiler-internal attributes and methods.

Library traits, types and impls/methods that do not require that degree of compiler intertie are intended to exist in other crates, rather than in std. As a result, those items in other crates can evolve via semver, whereas Rust's "forever" stability guarantees make it almost impossible to evolve anything in std.

4 Likes
7

I'm not knocking the crates or the usage of them.

I was just remarking that I'm doing this more as a learning exercise so if I use other crates, I don't really learn anything about Rust's threading or its lifetimes and borrowing.

8

Since you've got the main thread waiting to join() on the others, it can keep the data alive without the the others ever needing to see the Arc. This isn't something the compiler is currently able to reason about, though, so you'll have to reach for unsafe to make it work, either manually or via one of the other crates. If you do it manually, I imagine it's similar to the C++ solution:

struct SendablePtr<T:?Sized>(pub *mut T);
unsafe impl<T:?Sized> Send for SendablePtr<T> {}

pub fn main() {
    let mut ptr = std::sync::Arc::new(vec![0; 1024]);

    let mut handles = 
        Vec::<std::thread::JoinHandle<()>>::with_capacity(4);

    let v = std::sync::Arc::get_mut(&mut ptr).unwrap();

    // SAFETY: From this point, `v` and `ptr` MUST remain untouched in this
    //         thread until all child threads have terminated.
    //
    //         Child threads MUST NOT allow a pointer / reference to exist after
    //         they terminate.
    v
        .chunks_exact_mut(1024 / 4)
        .for_each(|chunk| {
            let chunk = SendablePtr(chunk as *mut [_]);
            handles.push(std::thread::spawn(move || {
                unsafe { &mut *chunk.0 }.iter_mut().for_each(|v| { *v = 1337; });
            }));
        });

    handles
        .into_iter()
        .for_each(|handle| { handle.join().unwrap(); });
    
    dbg!(ptr);
}

(Playground)

9

Ah, so that's what I was missing.

I had tried a raw mutable pointer before but the compiler said that it wasn't safe to Send across thread boundaries. So this is what I should've done, just directly overcome it using a custom type.

That works beautifully and simply, thank you!

10

Here’s the relevant part from the Rustonomicon:

Raw pointers are, strictly speaking, marked as thread-unsafe as more of a lint ... It's important that they aren't thread-safe to prevent types that contain them from being automatically marked as thread-safe. These types have non-trivial untracked ownership, and it's unlikely that their author was necessarily thinking hard about thread safety.

11

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.