History and status quo of async/non-blocking I/O in Rust?

help
1

Hi,

I recently started learning Rust coming from a Node background. It first took me by surprise that Rust's std crate only sync/blocking operations for I/O and that async I/O only lives in user land right now. On the other hand... Node did the same in the early days by removing things like Promises into user land. (Even though they kept callback style for async I/O in the built-in modules.)

I tried to research what is the history and status quo about async I/O in Rust and would like to now, if I missed something.

The de-facto standard for async I/O is currently mio. But mio is very low-level. A widely used higher level async I/O create which builds up on mio is rotor. rotor has an example implementation how it can be used for HTTP called rotor-http. But the de-facto standard for using HTTP in Rust is hyper. Just recently hyper added support for async I/O by using rotor.

Is this correct? mio is the current standard for low-level async I/O, rotor is the current standard for higher level async I/O build on top of mio and hyper is the current standard for async/non-blocking HTTP requests/responses build on top of rotor?

Other questions
Is there a library like hyper for async file system I/O?
What is the Futures API mentioned here?

1 Like
2

@alexcrichton gave a talk that was (partially) about futures-rs.

3 Likes
3

Thanks.

4

Is there a library like hyper for async file system I/O?

Not that I could find. I had a use-case for it, so I started researching and experimenting with it.

The unfortunate bit is that OS support for async file I/O is spotty and inconsistent. This is actually one of the few areas where, IMO, Windows has a better story than Linux.

The POSIX specification does include an API for asynchronous reads and writes on files, however the implementations are inconsistent between the BSD family and Linux. The BSDs have true asynchronous file read/writes in the kernel whereas Linux is kind of disappointing: it just moves all I/O to an implicitly spawned userspace thread. For my use-case, this was unacceptable because I was wanting to have all CPU cores working as much as possible, not being cross-scheduled with this hidden thread.

Windows, on the other hand, supports true kernel-level async I/O (or at least it pretends to, see below), so you can have all scheduled userspace threads working on your task that needs data from files (in my case, hashing potentially large libraries of images and comparing them).

You can see my prototype for this in my img-dup repo, which I unfortunately haven't touched in a while.

Instead of relying on the inconsistent implementation of async I/O on *nix, I decided to use posix_fadvise() instead, which lets me notify the kernel of which files to bring into cache so they can (hopefully) be read without blocking. I haven't benchmarked this solution yet, so it might perform worse than just using the async APIs.

Windows' implementation of file async I/O actually fits better into the mio event loop paradigm, because you're just looping between polling for messages and dispatching them (a common construct in Windows programming). I find this to be much more elegant than the POSIX API.

5

Be aware that Windows async IO has surprising randomly nonasync behaviors: Tech Notes: Nonblocking disk IO

In practice your best bet cross platform is using threads.

6

Yeah, it just seems to be a symptom of trying to get kernels designed for a synchronous world to do asynchronous things. No one has really tried to get it right yet. Windows might not have true async I/O until the next generation of kernels after NT (whenever that comes around).

7
  1. A long time ago, Rust did have goroutine-style async IO; at the time, Rust had a large runtime, and the async IO support was removed along with the runtime when Rust's niche was reimagined several years ago.
  2. If you want to do asynchronous IO, your best bet is probably libuv. Node uses it of course, Rust used to use it. I wrote a FFI binding to libuv a while ago expecting to use it in a project that never materialized; there's been some increasing interest in this lately.
  3. Asynchrony is well known to be good for network traffic (C10k and all that) but its relevance for disks is … frequently overestimated.
8

A while ago I set out to build the fastest (in terms of throughput) way to read files (on Linux). I assumed non-blocking would be the way to go, so I wrote Filebuffer. It is a low-level building block for non-blocking file reading that does two things:

  • It can memory-map files. This is fast because it avoids unnecessary copies and system calls. If data is in the disk cache, file reading is as fast as memory access.
  • It can query whether a piece of a file is in physical memory, and and suggest to the operating system to bring parts of a file into physical memory. On top of this you’d build non-blocking IO: check if the part you are going to access is in physical memory, and if it isn’t request the OS to bring it into RAM and go and do something else in the mean time.

What I found though, when throughput is your goal, is that the Linux kernel is already pretty smart in detecting access patterns and prefecting file data from disk. So all this checking if data is available, and even only giving prefetching hints, are just wasted system calls.

The reason that there is very little gain, is that IO is already async, only at the system level, not at the application level. Assuming that you are doing memory-mapped IO, if you page fault, the OS will start to load the data from disk and in the mean time schedule a different application that has computations to do. This other application would have had to run anyway, so anything that can be done while your application is waiting for data is a win: it means that your application can use the time it is scheduled more effectively. And it doesn’t have to be a different application entirely; it can also be a different thread of your application. Having a few threads that do blocking IO can be extremely fast. (It is more favourable for high throughput than low latency though.)

My conclusion was: just mmap the file and let the OS deal with the rest. A 0.1% win isn’t worth two days of further tuning. Keep in mind that this was about througput-optimised reading only. If what you want to do is low-latency writes the results could be completely different. Also, on Windows it is not possible to check whether data is in physical memory, so this style of doing non-blocking IO doesn’t work there.

8 Likes
9

On Windows a file can be opened with either FILE_FLAG_RANDOM_ACCESS or FILE_FLAG_SEQUENTIAL_SCAN which are hints to the OS regarding how you're going to be accessing the file. Those flags apply even when using memory mapped files, so if you want to maximize throughput and you're reading everything in linear order, FILE_FLAG_SEQUENTIAL_SCAN is basically optimal.

10

Thanks for the heads-up, I wasn’t using these. Filebuffer does support prefechting on Windows via PrefetchVirtualMemory, but it can’t check if data is in physical memory because I am not aware of a mincore equivalent on Windows. I didn’t run any benchmarks, but I would expect that you are right, and with FILE_FLAG_SEQUENTIAL_SCAN there is no advantage in manual prefetching.

11

I have a question here... :slight_smile: Just don't know if it's an error on my side. He mentioned await beeing "blocking" but my perspective (JS Promises + async/await) is it is just sugar for futures/promises (so they are just as "blocking" as futures):

async function loadPosts() {
  const res = await loadUrl('http:/...');
  const posts = JSON.parse(res);
  const subPosts = await loadSubPosts(posts);
  posts.subPosts = subPosts;
  return posts;
}

can be written as:

function loadPosts() {
  return loadUrl('http:/...').then(res => {
    const posts = JSON.parse(res);
    return loadSubPosts(posts).then(subPosts => {
      posts.subPosts = subPosts;
      return posts;
    });
  });
}

edit: it's @ ~37 min
edit2: ok, he's refereeing to the concurrent execution of futures:

async function downloadCount() {
  const [res1, res2] = await Promise.all([
    loadJson('http:/...'),
    loadJson('http://...'),
  ]);
  
  return res1.downloads + res2.downloads;
}
12

I think most transpilers for async/await actually compile down to generators/use state machines, but in this specific case nested closures (Promise nesting) might work as well – in JS, that is, in Rust you'd have to think of ownership and moving variables into closures!

By the way, regarding generators, @erickt wrote a bit about how do implement that in Rust here: Stateful: A Rust Experimental Syntax Extension for Generators and More - Chasing Rabbits

13

Anyone knows what's holding this back? Is there something about the design of await that makes it tricky even though erickt already built a prototype for yield? As far as I understand the async/await state machine conversion is very similar to the one used by yield (conceptually). Is this not implemented purely because it is extremely complex to do it in rustc? Or maybe it is waiting for MIR? If there are a lot of unanswered questions about the design then I'd love a thread about that. I think rust would be fantastic with async/await, the sooner we get it, the better:)

1 Like
14

Yeah, sorry, i was talking about the result (future object) of async/awaits & and_thens.

15

@elszben: stateful is still in active development :slight_smile: @NSil has started contributing, and we even have an irc channel #stateful now :slight_smile: Current state is that we've got generators mostly working. This lets you write such things like:

    #[generator]
    fn gen<T: 'static>(items: Vec<T>) -> T {
        for item in moved!(items) {
            yield_!(moved!(item));
        }
    }

We're still working on async/await. If anyone wants to help out, please let me know or join the #stateful channel!

16

I'd like to help out but I know very little about the compiler internals. I could be less than helpful at the beginning. I've checked out the irc channel but it seems to be dead. So it may not be the best way to discuss things (it's probably just a timezone issue though). Since you are not working alone, I assume that you discuss things with NSil. If so, could you please move the discussion to some public (and persistent) forum? Maybe then I (and possibly others) could participate.