Best way to use `sendfile` in a Rust codebase (with tokio or threads)

[dpc]

I'd like to use a zero-copy transfer from a file to tcpstream using sendfile.

However seems like this call is inherently blocking. Is there anything better to do about it than run in a spawn_blocking?

[alice]

You're not going to get around spawn_blocking for file IO unless you use io_uring. The sendfile call doesn't change anything regarding this.

[dpc]

I thought so.

Though while I kept searching, I found that async-io (smol?) had some method of using non-blocking calls and a separate thread doing async IO wakeups via epoll.

Seems like sendfile works with non-blocking FDs: c - sendfile() completion to non-blocking socket - Stack Overflow

Though it makes me wonder - given that sendfile is particularly useful for large transfers, is using epoll + non-blocking not going to just introduce bunch of context switches to the user-space for no good reason (just to get woken up and restart the operation?) I would expect the TcpSocket to get "not-ready" very frequently...

[alice]

Maybe sendfile works with non-blocking fds, but this is only relevant for the writes to the outgoing socket. The reads from the file are still going to be blocking.

@dpc

Though while I kept searching, I found that async-io (smol?) had some method of using non-blocking calls and a separate thread doing async IO wakeups via epoll.

Not completely sure what you're referring to here, but this is a problem with the APIs exposed by the kernel. You're not going to find any library that can do this, except possibly for things based on io_uring.