I've been trying to figure out the best approach to handle async tasks in a queue with the ability to work on a maximum number of tasks concurrently.
I currently have a channel
which when receives pushes a future in to a futures::stream::FuturesOrdered
. I then call fut.next()
when fut.len()
reaches the max, and tasks concurrently. This was heavilly based of this answer.
This works very well, however instead of waiting for the tasks to buffer/reach the max before calling fut.next()
, I instead want to continuously take and run available tasks whilst there is availability (up to the max limit).
I'm unsure the best approach for this, especially as it doesn't seem I can share FuturesOrdered
between threads (I would have one thread to continuously fut.push(rx.event)
and a separate to process the fut.next())
. Maybe using a stream and take(max)
would give me more flexibility?