Idiomatic way to read n bytes into a Vec without appending

1

I need to read n bytes (where n is a runtime value) into a Vec from a TcpStream. That is, the beginning of the Vec. I want to overwrite the first n elements of my Vec. The Write impl for Vec only appends, so I'm forced to check that Vec has enough room and resize if not, then read_exact into a mut slice of the Vec. Is there a more idiomatic/efficient way to do this?

2 
let mut buf = vec![0u8; 8192];
let n = strm.read(&mut buf[..8192]).unwrap();
// .. buf[..n] contains the read data ..
// look out for n == 0, which could mean EOF
3

Use Cursor::new。

Creates a new cursor wrapping the provided underlying in-memory buffer.
Cursor initial position is 0 even if underlying buffer (e.g., Vec) is not empty. So writing to cursor starts with overwriting Vec content, not with appending to it.

Example:

use std::io::Write;

fn main() {
    let mut v = vec![1, 2, 3, 4];
    let mut c = std::io::Cursor::new(&mut v);
    c.write(&[5, 6]).unwrap();
    
    dbg!(v); // 5, 6, 3, 4
}
3 Likes
4

Doesn't this fail if need to read, e.g., 8193 bytes? Also, I need a guarantee that n bytes were read, which read does not do.

5

How would I pass the TcpStream to write though?

6

.take(n) + std::io::copy, maybe?

1 Like
7

Ah, sorry -- I misunderstood your question.

The way to solve these problems (efficiently) is to use bucket brigades / rope buffers (unless you know an absolute maximum buffer size, and it is reasonable to preallocate it), where you create a list of buffers you read into. If your first read does not fit into the first buffer, you allocate a new one and keep reading into it. This is to eliminate the need to copy the buffer each time you need to grow it.

Also, if you already have a bunch of buffers allocated in a list you can use vectored reads to scatter the reads among multiple buffers in one read call.

I know there are crates to help you do these things in the async world, and I'd be surprised if there aren't such tools for the blocking std world as well.

8

I have actually just decided it's time to move to async, so what crate would you recommend for this?

9

This actually works perfectly for a sync environment. Thanks! EDIT: In fact, it also works in async land, so pending fancy buffer lists I will be using this.

10

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.