What's the state-of-the-art, on stable, way to read data from a file into a buffer without requiring a redundant initialization of the read-buffer?
Edit: I should probably clarify, I need to read a file block-by-block (i.e. not read the entire file in one go, so I can't use fs::read()).
I believe std::fs::read doesn't redundently initialize the buffer read in std::fs - Rust.
Indeed -- thanks for the tip!
Unfortunately it seems to end up using unstable features (in default_read_to_end), so I can't use it in my crate. Very interesting to see what some stabilizations may bring us though.
If your buffer is fixed-size you can use MaybeUninit, if not, I think you can use Vec::reserve to allocate uninitialized memory. You will probably need to write some unsafe code but I imagine it should be possible on stable rust.
Are you sure the overhead of initializing the buffer is a problem. For example when you create the buffer with something like
let buf = vec[0u8; 4096];
Rust should use alloc_zeroed from the global allocator GlobalAlloc in std::alloc - Rust
Which is optimized for that. For example most common OS will only give zeroed pages when asked for more memory. So if the allocator understands that it can do nothing and still return an initialized Vec. Although performant allocators reuse memory so they would need to zero reused memory.
Once you have an allocated buffer you might be able to just reuse it, so even if the allocator had to zero memory that only occurs once.
It's not a problem per se -- in this case I'm just reflecting on what a customer said about their use-case: During peak periods, they may be transferring TB's of data daily. There is some point at which I start to feel uncomfortable performing redundant initializations -- while I can't say exactly where that limit is, I do know that when we're dealing with over 1TB per day that limit has been passed a long time ago.
So it's not a technical requirement, more of a ... mental health one. 
Just the thought of wasting all those cycles.. shudders
Oh, absolutely -- I do that as much as possible. But what I'm asking about here are places where that's not possible (at least not the way the code currently works).
This is what peak performance looks like:
reader.by_ref().take(len).read_to_end(&mut vec)