Rayon: Vectors containing dyn Traits (e.g. functions)

help
1

Hi there, I'm trying to determine if what I'm attempting is possible. My intent is to, using rayon, run a number of functions concurrently that take no input and produce the same result. Given:

// Imagine this being some arbitrarily expensive function.
fn bar(n: u32) -> u32 {
    n + 1
}

The first step I'd take is to naively shove closures in a vec![] and call it a day:

let foo = vec![|| bar(1), || bar(2), || bar(3)];

but

    |
118 |     let foo = vec![|| bar(1), || bar(2), || bar(3)];
    |               ^^^^^---------^^^^^^^^^^^^^^^^^^^^^^^
    |               |    |
    |               |    the expected closure
    |               expected slice, found array of 3 elements
    |
    = note: expected struct `Box<[[closure@src/main.rs:118:20: 118:29]], _>`
               found struct `Box<[fn() -> u32; 3], std::alloc::Global>`

Let's help Rust with the inference:

let zoo: Vec<dyn FnOnce() -> u32> = vec![|| bar(1), || bar(2), || bar(3)];

but

error[E0277]: the size for values of type `dyn FnOnce() -> u32` cannot be known at compilation time
   --> src/main.rs:117:41
    |
117 |     let zoo: Vec<dyn FnOnce() -> u32> = vec![|| bar(1), || bar(2), || bar(3)];
    |                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
    |
    = help: the trait `Sized` is not implemented for `dyn FnOnce() -> u32`
    = note: slice and array elements must have `Sized` type

Box to the rescue?

    let foo: Vec<Box<dyn FnOnce() -> u32>> = vec![
        Box::new(|| bar(1)),
        Box::new(|| bar(2)),
        Box::new(|| bar(3)),
    ];

Okay! This actually compiles! Although I'm not sure why, given the threat of Sized in the previous compile error and that I can't see Sized listed in Box's docs. Well, no matter, sally forth:

use rayon::prelude::*;

foo.into_par_iter().for_each(|f| f());

but

error[E0599]: no method named `into_par_iter` found for struct `Vec<Box<dyn FnOnce() -> u32>>` in the current scope
   --> src/main.rs:140:9
    |
140 |     foo.into_par_iter().for_each(|f| f());
    |         ^^^^^^^^^^^^^ method not found in `Vec<Box<dyn FnOnce() -> u32>>`
    |
    = note: the method `into_par_iter` exists but the following trait bounds were not satisfied:
            `[Box<dyn FnOnce() -> u32>]: Sized`
            which is required by `[Box<dyn FnOnce() -> u32>]: rayon::iter::IntoParallelIterator`
            `[Box<dyn FnOnce() -> u32>]: rayon::iter::ParallelIterator`
            which is required by `[Box<dyn FnOnce() -> u32>]: rayon::iter::IntoParallelIterator`

Arg. So why is it complaining about Sized again now?

For now we can ignore that the for_each doesn't do anything, and that I'm likely invoking the f() incorrectly. At the moment I can't see why Rayon wouldn't want elements of Box like this. Is what I'm attempting fundamentally impossible? Thank you kindly.

2

when you use dyn T, the size is not necessarily known. By boxing it, it can be any size and still be reached, albeit, at a slower speed.

When you use rayon, you need to make sure your inputs are Send + Sync. Try adding that to your requirements with dyn FnOnce() -> u32 + Send + Sync + 'static

Edit: according to: rayon::iter::IntoParallelIterator - Rust
Vec<T> impls IntoParallelIterator so long as T: Send. So , you don't need the sync bound

3

It turns out that adding only Send in that fashion was sufficient, and it worked! Thank you!

I suppose the lesson here is that once I enter dynamic dispatch land via dyn, I need to be explicit about what combined trait this actually is. So it wasn't obvious to Rust that my Boxed closures were also Send, and so it refused to continue unless I told it things were okay.

And as for actually testing, this was enough:

    foo.into_par_iter().for_each(|f| println!("{}", f()));

It seems the Box gets stripped off automatically, probably via Deref, etc.

4

Yeah, Box gets deref'd to &T. Since the size of a pointer/reference is either 32 or 64 bit constant, we can return an &dyn T or &mut dyn T (with some limitations. For example, having a trait T with a self-refferential associated type can't be returned as &dyn T or &mut dyn T)