are there any linux filesystems that support non-blocking directory reads?
this would be massively helpful for FUSE filesystems, since they can block for absurd amounts of time (sometimes several MINUTES!), but i can't seem to get them to ever return EWOULDBLOCK.
fuse filesystems can return whatever errno they want, but i'm not sure they can detect that the file descriptor has O_NONBLOCK set...
i actually wrote all the code for async directory reads before realizing they don't actually work..
is there any way i can get this working, short of patching the kernel?
To my knowledge no such thing is available. Last year a patchset for adding getdents to io_uring was sent but it hasn't been merged. So your only option is to throw threads at the problem, as tokio does.
You could try limiting the concurrency per device id so that a single slow filesystem can't gum up an entire threadpool.
it feels like a simple O_NONBLOCK would be much easier to implement than O_ASYNC or io_uring, just check the kernel cache and return early if it isn't available right now.
The getdents syscall doesn't take a flags argument, so they'd have to introduce a new one. Which is a non-trivial task since it understandably leads to a lot of discussion about new API design.
O_NONBLOCK for getdents would not be guaranteed to make forward progress if the entries are not already in the cache. O_NONBLOCK with a read-like operation checks if data is already available and if not does nothing, without causing any background thread to read the requested data to be spawned. This is also the reason why O_NONBLOCK doesn't have any effect for regular files. It really only makes sense on things like sockets and pipes where another process (potentially on a different system) can cause data to become available for reading without the process trying to read having to do anything.
The read equivalent actually is preadv2(..., RWF_NOWAIT). And as @bjorn3 says you still need to handle the case where no data is immediately available which means offloading to a threadpool or some native async mechanism. And since polling isn't applicable to filesystems the latter case would be io_uring anyway.