Futures and task prioritization


#1

As the discussions around futures are spread in many places, I prefer to post this on this high-visibility forum, where I know core developers like @alexcrichton and @aturon are likely to be around. If there is an existing discussion thread on this matter somewhere, please point me towards it.

It took me a long while to wrap my mind about the general design of futures-rs, but my current perception can be summarized as follows: you start by building a state machine associated with a certain asynchronous task using future/stream combinators, and in the end you pass it down to an event loop, which takes care of “running” the state machine until its final state is reached by polling it repeatedly as the relevant input events arrive (which, in turn, triggers execution of relevant event handling code and state transitions).

On its side, the event loop receives a stream of OS events in an implementation-dependent way (in a Tokio setting, this would be mio, which itself hooks into epoll on Linux), and associates them with futures/streams by leveraging the fact that a certain future/stream has registered interest into a certain OS event.

Now, if my understanding is correct, here is what I am wondering about: as long as host system resources are plentiful, input events can be handled in a FIFO fashion, and it will optimize the overall latency and throughput very well. But as soon as the system is under load, we may need to prioritize some tasks over others in order to fulfill our latency constraints. For example, an audio application which asynchronously receives audio frames from the network and asynchronously feeds them into a sound card driver may want to prioritize delivering audio frames to the driver over receiving them from the network in order to avoid sound card buffer underruns (which result in very ugly glitches in the audio output). This requires reordering “sound card driver wants audio frames” events before “server received audio frames” events, for example by using something like a priority queue for OS-application communication instead of a FIFO queue.

Have this kind of task prioritization issues been considered so far by the futures-rs designers? And if so, how would you plan to handle them?


#2

I think one of the main goals of Tokio reform is to separate/decouple the notion of a reactor (event loop) from an executor. So in that model you’d have an event loop that simply notifies you of readiness and then you could execute the future (or continuation of it) on a custom executor that has a priority queue internally. At least that’s my understanding.


#3

I think the cleanest approach will be to split high-priority tasks into a separate thread with its own event loop and set high priority for the whole thread.


#4

So in my example scenario, you would imagine having one network-focused event loop running on a low-priority thread, one audio-focused event loop running on a high-priority thread, and then the network thread would somehow pass down its decoded audio frames to the audio thread via some synchronization mechanism?


#5

I am not completely sure, but I think it should be possible to pass connection to a different event-loop. So ideally you’ll have something like this: low-priority network thread receives connection, recognizes that it’s an audio stream, passes descriptor of this connection to high-priority event-loop which handles reading data from the network and immediately sending it to audio device.

Even if it’s not possible (or not currently convenient) you can work around it by specifying that incoming connection to port A will be general low-priority messages and connections to port B high-priority audio data.


#6

That would handle prioritization of audio-related network traffic over other network traffic. This is not exactly what I was thinking about (more like, in an hypothetical world where operating system asynchronous APIs were consistent, how to prioritize sending audio over receiving it, as missing the deadline for an audio frame is worse than dropping a network packet). But I think the underlying problem is of a similar nature and could be handled similarly.

So indeed, using the operating system’s built-in task prioritization mechanism by dispatching events to threads of different priorities would be one good solution.


#7

I’d be hesitant to leave scheduling up to the OS’s whim. You lose a lot of control over that decision and the OS may not do the desired thing under load; thread priority is perhaps too coarse and the kernel (naturally) has zero understanding of what it is you’re actually trying to do. There’s also the issue of portability and the differences between various kernel CPU schedulers. I think it’s better to have as much control over this as possible and minimize OS interference.


#8

I would also prefer to keep things at the application level, but I accept that relying on the OS’ built-in task prioritization mechanism is a possible solution. Especially as the OS scheduler has some superpowers which may not be available in user space, such as “real-time” scheduling on Linux.