I am learning async programming in rust andreferring the Programming Rust book chapter on the topic. I got the internals of future, poll, wakers, executors etc. I have a doubt which has stuck with me for long and still unresolved even after going through other sources in the literature. What I understood is that async programming is nice as functions are non-blocking and we can later poll for the result. I also understand that non-blocking computations inside async functions are also executed on the main thread and returning back as soon as await is reached giving us concurrency on main thread. My questions is that someone has to wait for that blocking operation right?, for example some background thread has to wait for response of a http network call and then execute waker so that main thread polls again for the result. So isn't it that each async task while non-blocking on main thread still has a background thread blocked for the result of the async operation. If this is all true then we still have thread spawned per task in background which in worst case would pose same problem as that of spawning thread for each task in first place (which people argue are heavy and instead we should opt for async programming for I/O bounded tasks).
On Linux there's a special OS interface called epoll that lets you give the OS a big list of sockets and wait for an event on any one of them, without using a separate thread per socket. Other OS have similar interfaces.
There typically isn't such an API for files, so what you are describing is true for file IO. For this reason, the Tokio tutorial says the following:
Reading a lot of files. Although it seems like Tokio would be useful for projects that simply need to read a lot of files, Tokio provides no advantage here compared to an ordinary threadpool. This is because operating systems generally do not provide asynchronous file APIs.
@alice So if I consider sockets then my poll method should internally be using epoll ? and if so then is waker invoked by epoll itself ?
The way it works in Tokio is that whenever a Tokio TcpStream wants to make an operation, it registers the socket with epoll and then stores the file descriptor/waker pair in a hashmap somewhere inside Tokio. Then, when Tokio has no work to do (or periodically if there's constantly work), Tokio will ask epoll which sockets have events, and then look up the wakers from the table and wake those tasks.
@Alice I was watching talk by Steve Klabnik : The Talk You've Been Await-ing for - YouTube
What does he mean when he says we can hand over a callback to epoll and it is going to call wake ?
I don't think you can literally hand a callback to epoll. You would have to store the callbacks in a hashmap similar to how I explained Tokio stores the wakers.
In Windows there's far too many options, each with different trade-offs:
- you can pass an event handle to each async operation in an
OVERLAPPEDstructure, then block onWaitForMultipleObjectson those events, very similar to epoll, which you can use to simply run multiple IO requests in parallel on the same thread, or to build a complex async runtime, though there are some headaches to deal with (like a max of 16 events in any one call) - You can simply ask it (for some requests) to call a callback on an OS managed threadpool, so there's no blocking at all, Unixes to my knowledge have never done this, for better or worse (it's of course inherently multithreaded, with all the risks of such)
- Most efficiently, you can use "IO completion ports", which is essentially a message queue you pull completion messages out of in a loop that the OS posts to when it's done, and it's up to you to associate each event back up to the request that started it to continue with the result. This means you can fire async requests off in a bunch of different worker threads and only one thread just pulling the next event off and dispatching new jobs to those workers, a great match to an async runtime. Linux has been working on getting this with
io_uringfor a while, it looks like it landed in 5.1, yay!
Note that tokio (and pretty much every other cross platform runtime, eg libuv used by node) doesn't actually use OS native async for filesystem requests (last I checked), they simply do as you assumed and shuffle the blocking call off to a victim thread to take the blocking call. This is both slower due to the communication overhead, but also can cause serious issues in weird edge cases, including deadlock (for example, reading and writing to a file handle that's actually a pipe). This is because Linux has for a very long time not had usable OS async file IO. Hopefully when io_uring can be assumed this will change!
This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.