Asynchronous parallel iterator

help
1

Hello,

I'm a bit stuck with this one. Most of my past experiences with concurrent programming was done with Rayon high level functions. Now, I want to do something more sophisticated and I don't where to start.

The problem is the following.

I have an infinite interator that produces somewhat complex states and an iterator of computations I want to run for those states. The first computation will be run with the first state, the second computation with the second state and so on. I don't know how many computations there are, I just have an iterator.

I would like a main thread which compute the next state and spawn the computation on an available thread. However, I can't compute the state n without knowing the state n - 1. So I need to generate the states sequentially.

I hope that this is clear. I've create a playground where I mock what I want to do.

I'm interested in any tools, advices or insight that could help me go foward with this problem.

Thanks!

2

You can use par_bridge. playground

fn do_something_awesome<C>(computers: C, iter: Iter) -> u64
where
    C: Iterator<Item = Computer> + Send, 
{
    computers
        .zip(iter)
        .par_bridge()
        .map(|(computer, state)| {
            computer.heavy_computation_that_i_would_like_on_a_different_thread(state)
        })
        .sum()
}

Everything before the bridge happens sequentially, and everything after it is parallelized.

1 Like
3

It seems to work fine. Thank @alice.

4

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.