Allocation and async code

1

I have no real reason beyond my curiosity about this. We are allowed to allocate and free memory in async code which can ultimately result in a system call. What guarantees does the allocator have to provide? Is it possible for an allocation or deallocation to indefinitely block a thread (and all the tasks waiting on it)?

2

Rust does not define what blocking is.
It is down to a particular executor[1] (eg tokio) and/or the person writing the code.

The documentation for poll

An implementation of poll should strive to return quickly, and should not block.

Note it does not say "must".

If you deadlock you have a bug in your code but just like with Mutex it is considered safe.

  1. The code that calls poll() ↩︎

3

There are two different sorts of blocking-in-async hazards to consider:

  1. Blocking on some other event that must occur in the application (e.g. calling a blocking channel recv() function) generally must be entirely avoided in async code, because it is a risk of deadlock — the async code that has blocked might be preventing other async code from making progress to cause that event to occur (if that other async code is running concurrently in the same task or in a task assigned to the same execution thread).

    But, in principle, you might block on some outside event that is not driven by the application itself. The memory allocation can be seen as that kind of blocking — as long as your code isn’t part of the kernel or a debugger, you know that it will eventually make progress.[1]

  2. Blocking in async code may reduce the performance of the application, because this executor thread is blocked during a period of time it could instead be making progress on other tasks.

    This is why, for example, you shouldn’t use std::fs operations from within async code — they (normally) can’t deadlock but they can be slow.

We typically decide that blocking on memory allocation is acceptable by (1) because the allocation is served by the kernel and hardware, not other parts of the application, and is acceptable by (2) because its duration is sufficiently short to not matter. But an application with stricter timing requirements might have a different opinion — that’s why there’s no precise definition of the blocking you shouldn’t do.

  1. Even things within the application might be blocked on if you specifically know that there are no causal dependency cycles that could lead to deadlock. ↩︎

8 Likes
4

(As I'm sure you know, but to clarify for the thread)

This one is a double edged sword: it's often far more efficient even from a global perspective to use std::fs over what implementations like tokio::fs have to do since OS support for async fs is still not there (maybe not now? I'm not sure where io_uring support is at, either at the OS level or in Rust async. What about MacOS? No idea what it looks like over there)

The first main problem with this are that straightforward uses of std::fs can block nearly indefinitely even when using it normally and everything is going right: instead of using the straightforward std::fs::read() and it blocking for several seconds on large enough files, or the obvious next step of just opening the file and reading chunks at a time, you need to do the rather unintuitive reading chunks and also yielding to the executor. It's far more natural to use the wrappers.

But worse, the real issue is that "fs" is a terribly overloaded concept now, and opening a file or even reading a single byte can be arbitrarily slow depending on where that file is and what hooks (notably antivirus) are doing - in the worst case you might be waiting for a network attached storage drive to fail to be found after network timeouts.

Really it's not worth the hassle dealing with it most of the time, just use the tokio::fs wrappers and keep in mind that it's much worse overhead so that buffering reads is even more important.

5

Even better is to spawn a blocking thread and do file I/O there. Otherwise you are at the mercy of tokio's hidden buffers and might end up doing thousands of system calls instead of one.

Writing is more fraught because calls to tracing (or just plain println and dbg) might end up as blocking writes (you can deadlock if you are writing to a pipe). The library writer has no control over what tracing subscriber is used.

There's no real way to avoid allocation though when you spawn a future, hence my question.

6

True, though it's easier to deadlock yourself that way. Everything is a trade-off!