Async file IO using memmap

I am wondering about the possibility of implementing async file IO using memmap.

The idea would be to be able to perform several file reads simultaneously ( and hopefully faster than doing them one-by-one ).

Has anyone experimented in this area? Or maybe there is even an existing crate (with a higher level functionality than memmap). I am a little worried than using memmap directly appears to involve using unsafe.

tl;dr You can't.

With the file-backed mmap the actual IO to the disk happens on page fault event which is the same way how the virtual memory pages are loaded from/stored to the swap space when the system memory pressure is high. So the question is same as "Can I dereference some pointers asynchronously?" and the answer is no.

2 Likes

Ah, I see. I guess you could use some threads? But it seems messy, and perhaps inadvisable.

Edit: Hmm. I had a read here. Rust is mentioned at slide 47.

If you're interested in io_uring and Rust, GitHub - tokio-rs/tokio-uring: An io_uring backed runtime for Rust and GitHub - ringbahn/ringbahn: safe bindings to io-uring might be good projects to look at.

3 Likes

I am actually on windows, it was more I was trying to puzzle out why File IO is currently not async, in a naive way. I think I now understand that async File IO is not currently easily available for somewhat obscure technical/platform reasons. Which is quite ok, it was more curiosity on my part, I was trying to understand why mostly. I guess sometime in the next few years it may become easily available.

1 Like

I just noticed this:

" * 57:33 - Replacing epoll with io_uring"

# Tokio Ecosystem with Alice Ryhl

published a week ago. Off to have a listen!

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.