I’m trying to find, or to create, a helper tool which basically operates on a stream of data, processes the data in parallel (CPU intensive and slow operation), and output the result of the processed data.
Important is that the output data is kept in order of first in (as a FIFO), and that input is blocked if the current number of processed elements is too long.
Rayon seemed almost like a good choice but couldn’t understand how to make it operate on a stream where not all elements are present when starting the processing.
My current solution involves some MPSC channels and a thread pool, but feels sub-optimal and can’t handle back pressure right now.
Is there some crate out there like this, or any proposals of how to implement this in a Rusty way?