Why does captured stdout + stdin need a new thread in std::process::Stdio?

[b-m-f]

Hi,

I was reading Stdio in std::process - Rust to understand how I can run a process and pass some data into it via stdin.

The example in the docs is creating a new thread to pass in the data. Not doing that also locks the program in my experiments.

Does anyone have a good explanation or a resource to learn more about why that is necessary?

Example from Docs

use std::io::Write;
use std::process::{Command, Stdio};

let mut child = Command::new("rev")
    .stdin(Stdio::piped())
    .stdout(Stdio::piped())
    .spawn()
    .expect("Failed to spawn child process");

let mut stdin = child.stdin.take().expect("Failed to open stdin");
std::thread::spawn(move || {
    stdin.write_all("Hello, world!".as_bytes()).expect("Failed to write to stdin");
});

let output = child.wait_with_output().expect("Failed to read stdout");
assert_eq!(String::from_utf8_lossy(&output.stdout), "!dlrow ,olleH");

[haccerman]

well, if you don't use a thread in this example, you are trying to write to stdin when the program hasn't been started yet, so it can't handle the input that you put in, therefore it needs to be in a separate thread. write_all will write until all of the buffer is written, which is not the case if the program isn't started.

[kpreid]

@haccerman

well, if you don't use a thread in this example, you are trying to write to stdin when the program hasn't been started yet,

In this case the program has already been started, because spawn() was called. The usual problem in this case is:

1. You're not reading from the output pipe, so
2. the output pipe’s buffers fill up, so
3. the child process’s write to stdout blocks, so
4. the child process doesn't read any more, so
5. the input pipe fills up, so
6. your write blocks.

This only applies to programs that do “streaming” processing, writing as they read, but for an arbitrary program you have to assume that they might. This won't happen on small pieces of data that fit in the kernel’s pipe buffer size, but it will when the input and the output are large.

So, in the general case, you have to write concurrently with reading. This doesn't necessarily mean spawning a thread — you can also use async or raw non-blocking IO — but spawning a thread is the way to do it with only std facilities.

[haccerman]

bruh