So, just trying to summarize the discussion on Read so far, I think we would like to complement its current only mandatory method…
fn read(&mut self, buf: &mut [u8]) -> Result<usize>
…with a second method based on internally managed storage that looks like this (all names are open to bikeshedding)
fn read_managed(&mut self) -> Result<&[u8]>
Note that with this API, you cannot call read_managed() if you are still holding a slice that you got from a previous call to it. That is intentional: it is a prerequisite for “sliding windows” use cases.
The implementation must guarantee that the underlying data is actually present in RAM, that no data races can occur with concurrent memory mappings, and that reads from the underlying slice will not block. If you want to use full-blown mmap(), in all its blocking and unsafe glory, then that is OS-specific and you should go for the memmap-rs crate.
One thing which read() can do and read_managed() cannot is to tune the buffer size. To some extent, this is on purpose: we sometimes want it to be automatic. But it is an important tuning parameter for performance vs memory footprint, and therefore it should be possible to control it. We could do it this way:
fn transfer_granularity(&self) -> usize;
fn max_window_size(&self) -> usize;
// winsize must be higher than zero, multiple of granularity, smaller than max
fn set_window_size(&mut self, winsize: usize) -> Result<()>;
To which extent these methods could/should have “sane defaults” for existing Read implementations is left as an exercise to the reader. If there are no sane defaults, then we may want to add these facilities as an extra “ReadManaged” subtrait of Read, rather than directly into the Read trait.
Another question that must be resolved is whether such an API could be used in no_std environments. I suspect that it couldn’t, which would be another argument in favor of the ReadManaged supertrait approach.
I’m not sure how epoll-style nonblocking readout and readout from N different sources should be handled, maybe this could use inspiration from mio for network I/O. Does anyone have opinions on that?
(One opinion which I personally have is that not every I/O source may have a nonblocking mode. For example, I’m not sure if CPU I/O ports can always be nonblockingly peeked in low-level operations. If that is the case, we may not want to provide a nonblocking interface for everything that has a Read/ReadManaged implementations, but only in cases where it makes sense.)