Processing a file by chunks? (Sending a file using tokio/Framed)

I need to send (large) files over the network in an application that uses tokio-util's Framed. For Reasons(tm) want to send the files in chunks. Our Encoder has one of these:

impl Encoder<Bytes> for Codec {
  type Error = crate::err::Error;

  fn encode(
      &mut self,
      data: Bytes,
      buf: &mut BytesMut
  ) -> Result<(), crate::err::Error> {
    buf.reserve(data.len());
    buf.put(data);
    Ok(())
  }
}

(Obviously other types can be added if needed, but this feels like good potential candidate).

Basically my question boils down to "How do I read a file in chunks into Bytes buffers? Or is there a better way to do it?".

The general handwavy idea (this obviously doesn't work):

async fn send_file_data(
  mut file: File,
  conn: &mut Framed<TcpStream, clntif::Codec>
) -> Result<(), Error> {
  let mut buf = Bytes::something();
  loop {
    let len: usize = file.read(&mut buf)?;
    if len != 0 {
      conn.send(Bytes::something_else(&buf)).await?;
    } else {
      return Ok(())
    }
  }
}

(Chunk size will be around 64K - 256K, so I'm keen on not using [u8; X]).

Hm... Just noticed BytesMut has a BorrowMut<[u8]>.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.