How to Stream n Bytes to a File

help
1

Been struggling with this for several days. I'm wanting to stream at most n bytes to a file from the HTTP body and return an error if the user sends more than the negotiated size. A stub of what I have so far assuming a 10MB limit:

pub async fn handle_chunk_upload(
    ...
    body: Body,
) -> Response {
    ...
    let body_with_io_error = body
        .into_data_stream()
        .take(10_000_001) // <- was expecting this to limit the size, but it limits # of items.
        .map_err(io::Error::other);
    let mut body_reader = pin!(StreamReader::new(body_with_io_error));

    let bytes_written = tokio::io::copy(&mut body_reader, &mut file)
        .await
        .expect("couldn't copy bytes");

    if bytes_written > 10_000_000 {
        remove_file(full_path)
             .await
             .expect("could not delete oversized file");

        return resp
            .fail("bytes received exceeded negotiated size")
            .status_code(StatusCode::BAD_REQUEST)
            .into_response();   
    }
    ...
...

I'd like to be able to do this from within the request scope so I can programmatically set this limit, do validation, and send back the appropriate error message.

Thanks in advance!

2

Take a look at axum::body::to_bytes as well as StreamBodyAsOptions.

One of them should do the trick.

3

I looked at axum::body::to_bytes before but I wanted to not buffer the entire body into memory. I'll take a look at StreamBodyAsOptions. Thanks!

4

So it turns out that the stream reader's take method does support limiting by bytes:

let body_with_io_error = body
    .into_data_stream()
    .map_err(io::Error::other);

let mut body_reader = pin!(
    StreamReader::new(body_with_io_error)
    .take(10_000_001) // <- this works for limiting on bytes
);

A question and a problem:
Q: Do I need to pin this? Am I getting any performance or safety out of pinning, here?

P: My HTTP testing client hangs when I send a larger file. The stdout looks like the axum server isn't hanged up and has send the response to the client. I'm assuming that I need to either flush the body stream or explicitly terminate the TCP connection to unstuck the client. How should I proceed here?