How to cheaply read non-blocking from a File?

I know this question is asked all the time, but I still can't find an answer: How to (cheaply) read non-blocking from a file? I found three ways, but all of them don't work (well) in my case:

  • Let a blocking read run in a separate thread and communicate over channels and just leave the thread and file descriptor be if it hangs. This doesn't work for since my program runs for a long time (it's a daemon, so potentially days, months, years ...) and there is a limit on the open file descriptors allowed.
  • Use tokio or another async library. I don't want to do this because at least tokio is - I think - pretty big and I don't want to have so much overhead for so little.
  • Extract the file descriptor and then use linux syscalls to put the file descriptor in non-blocking mode. I don't want to do that because I would need to use unsafe for this.

Is there any other possibility or do I have to use one of the last two options?


Assuming that by "file" you mean special files like pipes, sockets and the like you can use mio::Poll to have a cross-platform abstraction for readiness notifications. Once a fd is ready you can keep reading until you get a wouldblock error.

For regular (on-disk) files cross-platform support for nonblocking IO is spotty because they can't be handled with a readiness-model, you need a completion-based model such as io_uring or iocp which aren't available on all operating systems. That's why async runtimes have a fallback to use threads instead.

1 Like

By "file" I mean all files which find / would list. How does this work for that?

Well, that's mostly regular files. In that case io_uring is the only sane way to get non-blocking io on linux. There are also half-baked solutions like aio or rwf_nowait. But none of them are portable.

If you need a portable solution then use threads, that's what the async runtimes do under the hood anyway on most platforms.


This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.