How can I peek a TcpStream exactly?

Consider a protocol where the first few bytes indicate the total size of a frame (containing the size of size itself). I'd like to peek the size of a frame and use the result to decide the size of buffer to create, and then read_exact to the buffer to read the entire frame (both size and contents). But there is no peek_exact to make sure I read a fixed number of bytes. What should I do? Wrap one myself with reference to read_exact?

I hope this helps.

    const FRAME_SIZE: usize = mem::size_of::<usize>();
    let mut frame_size_buffer = [0_u8; FRAME_SIZE];
    let stream = TcpListener::bind("0.0.0.0:4000").unwrap();
    for client in stream.incoming() {
        let mut client = client.unwrap(); //Handle result error in your code
        client.peek(&mut frame_size_buffer).unwrap(); //Handle result error in your code
        let your_frame_size = usize::from_le(unsafe { mem::transmute(frame_size_buffer) });
        let mut read_exact = Vec::<u8>::with_capacity(your_frame_size);
        client.read_exact(read_exact.as_mut_slice()).unwrap(); //Handle result error in your code
    }
1 Like

This would make a protocol lose cross-architecture compatibility.

Calling peek without checking amount of bytes peeked doesn't ensure that the entire buffer was actually peeked, like read. This is the reason I asked if there is a peek_exact similar to read_exact.

Just call usize::from_le_bytes(frame_size_buffer) with transmutting handled properly inside.

This will create a Vec<u8> with capacity=your_frame_size but length=0, which make read_exact read nothing. Correct way is create a Vec<u8> with frame size of zeroed memory with vec![0; your_frame_size] and call read_exact with its mut slice.

As a reference, my current impl is like below:

pub fn recv(&mut self) -> io::Result<Vec<u8>> {
    // TODO avoid copy with "peek_exact"
    let mut len_buf = [0; 4];
    self.conn.read_exact(&mut len_buf)?;
    let len = u32::from_be_bytes(len_buf).try_into().unwrap();
    let mut frame = vec![0; len];
    frame[0..4].copy_from_slice(&len_buf);
    self.conn.read_exact(&mut frame[4..])?;
    Ok(frame)
}

What's I'm looking for is like below:

pub fn recv(&mut self) -> io::Result<Vec<u8>> {
    let mut len_buf = [0; 4];
    self.conn.peek_exact(&mut len_buf)?;
    let len = u32::from_be_bytes(len_buf).try_into().unwrap();
    let mut frame = vec![0; len];
    self.conn.read_exact(&mut frame)?;
    Ok(frame)
}

Seems that peeking can't "avoid copy", but can make the code clearer. I think if there is peek, there should be peek_exact.

Your current implementation seems like a perfectly good implementation already. Yes, it doesn't peek, but peeking is just a “user-space” abstraction (happening in your process, not a fundamental operation on the socket) anyway — the operations executed by the CPU will be more or less the same as if there was a peek_exact.

Also: A lot of network reading code works by having a single buffer it reads bytes into (similar to Rust's BufReader, though that doesn't have quite enough control) and then copying data out of that buffer once it has a complete message buffered. It would not be hard to write something to do that.

2 Likes

read_exact works by calling read, checking how many bytes it read, and then calling read again if needed (with an updated buffer start/length), until either the required number of bytes have been read, or an error/eof is encountered.

This works very well because each call to read will block until at least one byte of data is available (or until there is an error/eof), so while the loop might have to go around several times, it never has to busy-wait calling read over and over without making progress.

The same behaviour isn't possible for peek. If you peek and get back 2 bytes when you wanted 4, then retrying the peek will not block even if no more data has been received yet: it will just return the same 2 bytes as the first time.

So, if you want to peek a specific number of bytes, then you would have to retry it in a loop (probably with some kind of sleep/delay in the loop to avoid busy-waiting) to get the behaviour that you want - it's not possible for the standard library to do it any better than that.

Probably the best option is to just copy the four bytes as you are currently doing, and not use peek at all.

6 Likes

SO_RCVLOWAT might work, though I don't know if it also applies to peek.

1 Like

BufReader allows peeking the buffer.

Yes, but there is no way to ask it to fill the buffer to an exact (or even minimum) length.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.