How to tell when connection is done sending bytes?

help
1

Hello, I'm attempting to collect all the bytes into a String

    tokio::spawn(async move {
        let mut buf = [0; 5];
        let mut string = String::new();

        loop {
            // When all bytes are received, it seems to stop here and wait for more bytes.
            let n = match socket.read(&mut buf).await {
                Ok(n) => n,
                Err(e) => {
                    dbg!(e);
                    return;
                }
            };

            // An attempt to break the loop here.
            if n == 0 {
                break;
            }

            string.push_str(String::from_utf8(buf[..n].to_vec()).unwrap().as_str());
        }

        println!("{}", string);
    });

How can I tell when the bytes are done sending so that I can exit the loop?

2

If socket.read returns Ok(0), then the connection must've been closed. (That means, if you don't want to close the socket, then you should communicate the total string size on the socket in advance.)

Note that the packets you receive might have fragmented UTF-8 characters on the ends, which is why you shouldn't convert on the fly for full UTF-8 compatibility.

3

I tried setting the buffer size to 5 as shown above and I would get like Ok(5) about 5 times, then Ok(2) for the remaining two bytes, then it fails the n == 0 test, starts the loop again, and gets stuck at socket.read(&mut buf) again. I'm assuming socket.read is holding waiting for more bytes. So I'm unable to determine when I can exit the loop.

Note that the packets you receive might have fragmented UTF-8 characters on the ends, which is why you shouldn't convert on the fly for full UTF-8 compatibility.

Noted, thank you.

4

socket.read() will return Ok(0) when the other side closed the connection. If your sender keeps the connection open, the receiver side will wait on socket.read().

5

I see. That makes sense why curl in the terminal hangs, and only after pressing ctrl + c, does the bytes write. But the question remains... How do I know when I've received the full request bytes so I can form it into a request struct?

Then I can properly process the request and return a response.

6

TCP provides a stream interface and has no native way of signalling the end of a message to the other side. This has to be specified and implemented by the protocol transported over TCP.
As you mentioned curl, let's assume you are implementing an HTTP server. The HTTP request header is terminated by an empty line (see also HTTP - Wikipedia), so you want to read from the socket until you encounter this empty line.

7

If you want to be able to send multiple requests (in series, paralleyl is a different situation) over the single connection, then you need a framing protocol of some kind. There are many choices. Http uses a particular approach, based around lines. SMTP assumes only ascii, and is built around lines. Other protocols uses a fixed initial encoding that includes the request length in bytes.

8

You need always respect timeout.

9

if you can know the size of the data before sending it i recommend you send that at the beginning of the message. operating on known lenghts and enforcing it makes it easy to handle and helps quickly recognize malformed data from a malicious actor

10

The phrase you're looking for here is "Framing"

tokio_util has some basics to make this easier, but in general you're now in the business of "implementing a protocol" so it's worth learning what's going on and how it works and what functionality you need and why.

As an example, a very common extensible approach is a "type, length, data" frame, where you collect enough data for two ints, one that says what the data shape is and the other is the number of bytes in the data frame. Then you wait for that number of bytes to arrive, chop that off the buffer, parse and dispatch it based on the type.

You'll want to use a circular buffer type like: BufMut in bytes::buf - Rust

11

Yea that is what I'm doing currently. I'm just reading the first 1024 bytes, reading line one, checking that it's a proper http request line, then validating the rest.

12

For HTTP you need to handle either content-length or possibly content-transfer-encoding. It's actually kind of a lot of work to correctly parse arbitrary incoming HTTP requests, since the same socket can get used for multiple requests in a row if you're doing it properly (eg handling the connection header) and the client doesn't need to wait for you to respond to one before it sends the next, not to mention all the weird ways you can attack a server by sending data slowly to starve it of connections.

For hobby stuff, just explicitly responding with Connection: Close on requests and trusting the client to open a new connection every time could be fine, though I'd still validate against content-length.