How to read to a vec as a buf

help
1

So I this is what I have right now

let listener = TcpListener::bind("0.0.0.0:9377").unwrap();
for stream in listener.incoming() {
    thread::spawn(move || {
        let mut stream = stream.unwrap();
        let mut buffer = [0;4096];
        stream.read_exact(&mut buffer).unwrap();
        let response = String::from_utf8_lossy(&buffer[..]);
    });
}

and this works but you see the buffer is 4kb but I want this to be bigger (like big) because I want to make my own web host platform thing and I need it so people can upload files and with me only having a buffer of 4kbs is not a lot that has to be a really small file and I want it so people can upload videos. Is there any way I can maybe put it in a vector so it can be dynamically sized so I don't have to make a 1gb array?

2

I have also tried doing this but it does not work and just hangs on stream.read (the 2nd one)

let mut stream = stream.unwrap();
let mut buffer = [0;4096];
stream.read(&mut buffer).unwrap();
let response = String::from_utf8_lossy(&buffer[..]);
let response = response.split("\r\n\r\n").next().unwrap();
let mut size:usize = response.len()+4;
for i in response.lines() {
    if i.starts_with("Content-Length: ") {
        size+=i[16..].to_string().parse::<usize>().unwrap();
    }
}
let mut buffer = vec![0;size];
stream.read(&mut buffer).unwrap();
let response = String::from_utf8_lossy(&buffer[..]);
3

Calling read without checking the length that it returns is always wrong.

Perhaps you're looking for read_to_end?

4

When I try and do this, and I try and connect with a web browser it hangs and once I tell the browser to stop looking for a response then it finishes the parsing. (this is what I mean 2021-12-17 09-37-40)

5

Right. Your browser won't send an EOF after they send the request because they want to reuse the connection for future requests, but read_to_end waits until an EOF arrives. The only way to do that correctly is to parse the (possibly partial) request that you got from the browser so you can tell whether the entire request has arrived or not, and stop reading only once the entire request has arrived.

6

that is what I tried to do with the 2nd set of code but that does not work because you cant read the stream twice (I think that is the error), I have no idea how I would be able to read it then see if it is the end they try and read more then see if it is at the end etc, one thing I did try and do was read_line to see if I ever got to the end but it would freeze once I would get to the end and not return an error, I have tried to use the get length then read line together, I dont know if I did it wrong but it did not work too.

7

Ok, well, I can immediately tell you that the previous snippet is wrong because it does not check the length returned by the call to read.

8

Try something like this:

let mut request = Vec::new();

let mut buf = [0; 4096];
while !request.contains(b"\r\n\r\n") {
    let len = stream.read(&buf)?;
    request.extend_from_slice(&buf[..len]);
}

// .. parse request here
2 Likes
9

When the HTTP request contents a content-length header (which I believe is typical but not guaranteed), this can be used to size the buffer.

10

The content-length header specifies how much data the request has after the \r\n\r\n string. For example, if you're uploading a file, it measures the length of the file, and does not include the request itself. It's not present for a bare request that doesn't include posting extra data such as a file upload.

11

Thanks for the correction, Alice. I thought the problem was sizing a buffer for a file upload, but it looks like you're right and there is also a problem with the size of the headers alone.

I think your solution is correct. Could also take a look at how other crates (e.g., hyper) handle this.

12

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.