How do I read from a reader (TcpStream) until a certain amount of bytes, EOF, or until a 4-byte delimiter ("\r\n\r\n")

As of why: I am learning about HTTP and is writing a simple router library. I currently read a constant amount of bytes from the TcpStream. and find where the end of the header is (if there is even any).

Preferably I want to know why it stopped reading (EOF, Reached N bytes, or Delimiter).

I feel like there should be a better way to do this. I appreciate any help!

What I would do is wrap the TcpStream in a std::io::BufReader and keep calling fill_buffer() until the buffer contains the delimiter you are looking for.

From there, you can parse the headers before the \r\n\r\n and call consume() to remove them from the buffer, and anything left will be your body.

3 Likes

I've been trying to that (then i took a break.). due to an error

here is the example code:

let mut r = TcpListener::bind("0.0.0.0:8080").unwrap();
    for x in r.incoming() {
        let y = x.unwrap();
        let mut reader = BufReader::new(y);
        let rbuf = reader.fill_buf().unwrap();
        reader.consume(rbuf.len())
    }

so the error here is:

error[E0499]: cannot borrow `reader` as mutable more than once at a time
   --> src/example.rs:69:420
    |
119 |             let rbuf = reader.fill_buf().unwrap();
    |                        ------ first mutable borrow occurs here
...
137 |             reader.consume(rbuf.len());
    |             ^^^^^^         ---- first borrow later used here
    |             |
    |             second mutable borrow occurs here


since the rbuf's type here is a &[u8]. i am not sure why the error even happens. as it is not a mutable reference to reader

although the function fill_buf() requires a mutable reference. it should have stopped using that reference a long time ago before reader.consume()

would you care to explain why this happens and how should i fix it? as i am exactly not familiar with rust (less than 2,000 lines of code). rust is kinda weird

Borrows are tied together via their lifetimes. If you have a method with signature (&mut Self) -> &T, then the returned immutable borrow will still have the effect that the original receiver (self) is considered mutably borrowed.

No, this is required for soundness. Explanation.

If you need the length, cache the length. Not the slice, not anything that borrows from the reader, only the length.

As far as I can tell, std::io::BufReader does not have a method fill_buffer().
It does have a method fill_buf() but that does not have the behavior you're counting on here, see BufRead::fill_huf():

Returns the contents of the internal buffer, filling it with more data from the inner reader if it is empty.

This means repeatedly calling fill_buf() does not enlarge the buffer, it simply keeps returning the initial read over and over.

It would be great to have something like a read_more() method on BufRead but fill_buf() is not it.

since i am writing a library relating to http stuff. i certainly need the data. what should i do in this case?

also. thanks for your answer to this question. i understand rust's references a little more now

after digging through std's docs. that might not be the case? as there is a function that specifically requires the type not to return the same data

BufRead::consume()

It seems you are using BufReader in a very weird way or you didn't understand my advice. If you need the data, then use it, then consume more of it once you are done with the current chunk. There's nothing preventing you from doing that. Playground. You can also just access the buffer temporarily, since the BufReader provides a .buffer() method.

sorry. i understand it now. using BufRead for the main problem feels a little clunk though. thanks!