I am trying to figure out the best approach to prioritisation of futures (requests being processed by a server). If I radically simplify the problem, I can say I have got a server (let's say actix web server with async request processing functions) which processes incoming requests with async/await on a set of threads (could be just one thread). Each request can be one of 2 types. For example I would like to separate as the following, but can change the categories in the future: 1) requests with incoming data requiring empty response and 2) requests with empty incoming data but requiring responses full of data. Type of a request can be recognised from very early on when it is dispatched in a requests processing function (which is async). In most cases requests processing involves either reading or writing files on disk.
When a host has got plenty of resources, I do not care about the prioritisation as all requests are processed with relatively low latency.
Now when a host starves for resources (or when my server is required by a side-monitor to slow down a bit), I would like to make sure that requests of the 1st type take priority for execution. Specifically, futures constructed for the processing of the 2nd type requests are not executed until 1st priority requests are all processed (or about to be completed processing). In other words, I would like to make sure the latency for 1st type requests is minimal as possible; the latency for the 2nd type of requests can be unbounded (waiting even indefinitely, if necessary).
I searched rust for futures priority polling, tasks prioritisation queues, but still struggle to find the best approach and the best combo of libraries, which could help me to achieve the required behaviour.
Could you please advise how you would do it and what libraries you would use within the async function, which processes the requests?