Capture process output as it comes in

[jlindsay]

Hello everyone,

I am using std::process::Command to execute a very long running process. I can get that to work without issue, but my problem is that I can only figure out how to capture the process's output after the process has completed. But I would like to be able to print each new line of output from the process as they come in. For example, the process will issue periodic progress updates, e.g. "Progress: 1%" and I'd like to print this out as it comes in, not after the process has completed. I've read all sorts about pipes etc on various forum, but don't see anything that addresses my issue. Any suggestions on how to implement this?

[cuviper]

If you pipe stdout, as shown in docs here, then you can get that field from the Child handle and read it dynamically like a file.

[jlindsay]

Thank you for your reply @cuviper. I must be going 'Rust blind' looking at all these process::Command docs because I still can't understand how to implement this. It's funny because I was able to make this work in Python without any issues at all. I've been coding in Rust as my primary language now for a number of years but this is clearly a corner of the language that I cannot fully understand.

[semicoleon]

It's not exactly pretty, but here's a simple example that I think does what you want

use std::{
    error::Error,
    io::{Read, Write},
    process::{Command, Stdio},
};

fn main() -> Result<(), Box<dyn Error>> {
    let mut child = Command::new("curl")
        .arg("https://google.com")
        .stdout(Stdio::piped())
        .spawn()?;

    let mut stdout = child.stdout.take().unwrap();

    let mut buf = [0u8; 100];
    let mut do_read = || -> Result<usize, Box<dyn Error>> {
        let read = stdout.read(&mut buf)?;

        print!("{}", std::str::from_utf8(&buf[0..read])?);
        std::io::stdout().flush()?;
        Ok(read)
    };

    let mut last;
    while child.try_wait()?.is_none() {
        last = do_read()?;
        println!("\nalive: {}", last);
    }

    println!("{}", child.try_wait()?.unwrap());

    // make sure we try at least one more read in case there's new data in the pipe after the child exited
    last = 1;

    while last > 0 {
        last = do_read()?;
        println!("\ndead: {}", last);
    }

    Ok(())
}

[jlindsay]

This works for me @semicoleon. Thanks for posting this. It may not be the prettiest solution for this problem that I've seen, but it certainly is functional!

[semicoleon]

Oh I forgot to mention that you should be careful with this loop

while child.try_wait()?.is_none() {

Since it will essentially busy wait even if there's no output for several loop iterations. You may want to do some sort of back off if the process isn't outputting data constantly to avoid wasting a bunch of CPU time on checking if the child has exited yet.

[jlindsay]

Also, can you explain why the closure is necessary? Why can't you just read in the while loop directly? I tried and it just hangs, which then makes me wonder why it needs to be this way.

[semicoleon]

The closure shouldn't be required, I just didn't want to write the same code in both loops. I'm not sure why just copying the closure body into the loop would hang