Using vec![] or vec![0; 256] in async function

help
1

Hi

There is the following code snippet which works as expected. It was able to send data to the remote server on the TcpStream connection.

async fn write_pipe(mut write_half: tokio::io::WriteHalf<tokio::net::TcpStream>) -> std::io::Result<()>
{
    let mut tokio_stdin = tokio::io::stdin();

    loop
    {
        let mut message: Vec<u8> = vec![0; 256];
        tokio_stdin.read(&mut message).await;
        write_half.write(&mut message).await;
    }

    Ok(())
}

However, if I were to change the line let mut message: Vec<u8> = vec![0; 256]; to let mut message: Vec<u8> = vec![];, I will not be able to send any data on the TCP connection (nothing shows up on the remote server's console output which is running nc).

Why is that the case?

What if I do not know the full size of the message beforehand?

2

AsyncReadExt::read does not grow the message buffer. So if the buffer has a length of 0, which is the case when you create a vector with vec![], you won't be able to read any bytes from tokio_stdin, henceforth not forwarding any data to your write_half.

5 Likes
3

Thanks for the answer! What can I do then if I do not know the size of the message that the user wants to send beforehand?

4

According to the tokio docs you probably shouldn't use tokios stding:

This handle is best used for non-interactive uses, such as when a file is piped into the application. For technical reasons, stdin is implemented by using an ordinary blocking read on a separate thread, and it is impossible to cancel that read. This can make shutdown of the runtime hang until the user presses enter.

For interactive uses, it is recommended to spawn a thread dedicated to user input and use blocking IO directly in that thread.

Instead you should probably use stds stdin

use std::io;

fn main() -> io::Result<()> {
    let mut buffer = String::new();
    io::stdin().read_line(&mut buffer)?;
    Ok(())
}
5

AsyncReadExt::read_to_end appends the read data to the buffer, growing it when necessary. AsyncBufRead::read_line would be an appropriate method for reading interactively from stdin. There are also the copy and copy_buf functions for conveniently copying bytes from a reader to a writer.

3 Likes
6

Thanks! I will check on that.

7

Thanks for the quick reply! I will check on those as well.

8

To expand just a little on @jofas's answer, the async I/O APIs are designed to mirror the sync stdlib I/O APIs, for consistency. So you can expect the sync APIs to behave in the same way in terms of which methods can grow the buffer and which cannot.

3 Likes
9

Not directly related to the question, as it has been already answered, but there are a couple more things you could improve in your snippet.

  1. You should declare your buffer outside IO loop. Now each time you loop you allocate and deallocate heap memory. Maybe compiler can optimize this out, but I don't know it and I don't think this should be relied upon.
  2. You have a logic bug. AsyncReadExt::read (similar to std's Read) returns Result<usize>, where Ok(n) means that n bytes were written to the buffer. But you are writing a whole buffer to the write_half each time, which will result in incorrect data being send. Using tokio::io::copy[_buf] as @jofas has suggested will also fix this.
2 Likes
10

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.