Does anyone have any advice for tuning Tokio besides changing the number of threads?
I was wondering if for latency sensitive processing if there are some neat tricks to minimize the time it takes to wake up tasks. I was considering having a small heartbeat task, just to keep the scheduler hot.
Thanks
When I want low latency I dedicate a real thread to each thing that needs to respond quickly and run that at a higher priority or preferably dedicate a core to run it on. I might be wrong but I don't see anyway to do those things with Tokio.
You can do that with multiple runtimes. Put one runtime on a dedicated thread with high priority and run those tasks there.
Ah, indeed. thread_priority - Rust
minimize the time it takes to wake up tasks
What kind of tasks, what are they waiting on? Why are they not waking up as fast as you expect them to? Optimizations probably depend on your specific scenario.
Some wakeup delays will also depend on OS configuration rather than tokio itself. For example linux groups timer wakups in 50µs increments by default to improve efficiency.
I've been focusing on websockets (inbound & outbound), but I am using a few other types of await points, such as timers or REST calls. The goal is to reduce tails, which I believe are coming from the threads getting parked.
The attraction is the available libraries that are available in the tokio ecosystem - I'd rather not have to create my own IO drivers or async executor that busy-spins.
I've considered doing the thread priority, and even seeing if I can hijack the tasks, by scheduling tasks within my own lightweight executor that owns a tokio runtime (just so I have the io and timer drivers alive).
With this strategy, do you stick a load balancer in front of multiple runtimes?
You would usually put different kinds of tasks in runtimes with different priority. Load balancing only makes sense when it's the same kind of task.
Besides using a single thread runtime with scheduler priority, are there settings in the runtime that would be applicable?
As for choosing values, benchmark 