Calling a trait object within a Rayon par_iter() closure?

help
1

Newbie question here - new to both Rust and system-level languages so would very much appreciate some help.

Use case is that I'm generating a map of financial bars from csv files of ticks and serialising to disk.

I'm looking to make pluggable generators for different types of bar.

My naïve plugin has a single trait:

pub trait Generator {
    // Process a tick and return any new bars
    fn next(&self, tick: Tick, tick_dt: NaiveDateTime) -> Option<BTreeMap<i64, Bar>>;
}

To speed up the process, the generator runs each financial symbol in parallel:

use rayon::prelude::*

fn run_generator(
    generator: Box<dyn Generator>, 
    symbols: &Vec<Symbol>, 
    start_year: u16, 
    end_year: u16) {
    
    symbols.par_iter().for_each(|symbol| {
        for year in start_year..=end_year {
            ...         
            let option = generator.next(tick, tick_dt);
        }
    }
}

The par_iter().for_each fails to compile because the Generator trait cannot be shared between threads as the Sync trait is not implemented.

The problem appears to be that any borrowing is seen as unsafe, but I have to pass a borrowed &self.

So how are trait objects used within a parallel loop? This is a pattern I was planning to use quite extensively in the app, so I'd appreciate any pointers (pun intended) towards a practical solution.

2

In order to share a reference between threads, the type needs to implement Sync. Your trait object won't by default. One way to fix this is to add the trait object bounds like this: Box<dyn Generator + Sync>. Adding additional traits like this to dyn only works with marker traits.

However, this has the disadvantage of possibly needing to add it anywhere that you're using dyn Generator, so instead you might consider making it a requirement for the Generator trait itself:

pub trait Generator: Sync {
    [...]
}
3

dyn Generator is not Sync indeed, but dyn Generator + Sync is. So, if you replace Box<dyn Generator> with Box<dyn Generator + Sync>, this error should go away.

4

That works, folks - I realised I must be overlooking something simple. Much appreciated!

I have checked out The Book on concurrency, but it's quite thin. Clearly, I need to improve my understanding. Are there any learning resources you can recommend - ideally something with lots of practical examples and exercises? This is relatively unfamiliar territory for me...

5

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.