"Be aware that most operating systems do not provide asynchronous file system APIs. Because of that, Tokio will use ordinary blocking file operations behind the scenes. This is done using the spawn_blocking threadpool to run them in the background."
So file io is actually blocking io + thread pool? Did I mis-understood something?
Not all IO is the same. Different kinds of FDs/handles have different amounts of support for non-blocking access. File systems are generally the least compatible; while some platforms do have some amount of async file read/write support, it hasn’t been available and broad enough for Tokio to already be supporting it.
However, looking at Tokio's issue tracker, it seems there is recent progress on async file IO via io_uring on Linux:
Even then, there are many more filesystem operations (e.g. deleting a file) which are not expressed in terms of reading/writing bytes and must each have non-blocking support implemented in the OS and async support implemented in Tokio, before Tokio could remove that disclaimer entirely from their documentation.
It seems for tcp network Tokio "is" using epoll as a non-blocking io backend.
Update: But why tokio only support non-blocking IO for tcp/network fd? Why not also support them with normal file fd? At least in linux, tcp and normal file fd can be handled in the same way.
It's a bit surprising to see that you mention asio without mentioning that it could only do asynchronous operations on Windows and, sometimes, on Linux (on many Android versions io_uring is not supported, e.g). It's written very explicitly in the documentation!
Besides the fact that C++ libraries normally don't support async io on files while Rust does the best it could do? No.
Register a epoll/poll/kqueue fd for all other file, tcp fd handled by tokio.
Set "non-blocking" flags for a file fd when open or create it.
Register the file fd into the epoll/poll/kqueue with a callback (in c-style) function, wrap it into the "Future::poll" in rust
Schedule the file IO with epoll/kqueue event, thus we can get a non-blocking, single-thread IO on multiple files and tcp streams.
But I realized that, maybe user should register the epoll/kqueue by themself, and do all the above work manually. Tokio's default implementation is using thread pool.
Being "ready" means that the requested operation will not block;
thus, poll()ing regular files, block devices, and other files with
no reasonable polling semantic always returns instantly as ready
to read and write.
Tokio does not use regular files with epoll because it doesn't work. According to epoll, files are always "ready" for reading or writing, even if such operations actually block. Other than io_uring, there is simply no such thing as "asynchronous file IO", and so we do not support things that do not exist.