Display *and* capture STDOUT and STDERR from a Command

Hi there!

I'm having an issue that I already had a long time ago, but as I didn't manage to get help on that one at the time and I'm still stuck on it, I'll re-open a new thread here :slight_smile:

I'm currently making an application where at some point I want to let a long command running and display its output in the terminal, while still being able when the command ends to retrieve all its output in a string.

How can I do this? Given the command will use escape sequences to display colors and reset lines in the terminal, I need something that would work a bit like an Stdio::inherit_and_pipe().

Thanks in advance for your help!

1 Like

You could spawn the process (with Stdio::piped()) and read from its stdout, printing and pushing to a Vec<u8> whenever it outputs something. For printing you should write into std::io::Stdout since println! expects UTF-8 strings and you just want to pass the bytes on to stdout. I think this should handle colors and such correctly as well.

If you are already running a subcommand, you could just pipe both of its output streams through tee like this:

./foo 1> >(tee foo.stdout.txt) 2> >(tee foo.stderr.txt >&2)

and then read foo.stdout.txt and foo.stderr.txt when it finishes.


If you really want to do it programmatically, you could read and write in a loop, but be aware that this basically only works if everything is unbuffered everywhere: Playground

fn main() -> Result<()> {
    let mut cmd = Command::new("cat")
        .arg("-u") // ensure unbuffered
        .stdin(Stdio::piped())
        .stdout(Stdio::piped())
        .stderr(Stdio::piped())
        .spawn()?;

    let mut stdin = cmd.stdin.take().unwrap();
    let mut stdout = cmd.stdout.take().unwrap();
    let mut stderr = cmd.stderr.take().unwrap();

    let lines = [
        "first line written to stdin",
        "second line written to stdin",
        "third - and longest - line that has been written to stdin",
    ];
    let mut readbuf = vec![0; 64];

    for s in lines {
        stdin.write_all(s.as_bytes())?;
        println!("wrote: `{}`", s);

        stdout.read(&mut readbuf)?;
        println!("read: `{}`", String::from_utf8_lossy(&readbuf));
    }

    Ok(())
}
2 Likes

What you probably actually want to do here is use the tracing crate with a FmtSubscriber to stdout and a appender subscriber to a file.

Thanks everyone for your answers!

I'll go with the programmatic solution as I don't to involve any filesystem for just running a command.

I'll try that and check if it works properly :slight_smile:

Ok so I just tested this out and it seems like it works but not exactly like I want.

The thing is, it seems to work perfectly when just checking STDOUT or STDERR, but I want to display both STDOUT and STDERR.

I came up with a loop where I do a stdout.read(&mut buff), print it with io::stdout().write(&buf[0..bytes_read]) for STDOUT, and the same for STDERR. This works like a charm except that if there is, let's say, a message displayed on STDOUT, then another, but nothing on STDERR, the loop will block while trying to .read() on STDERR.

And AFAIK there doesn't seem to be any non-blocking reading method in std for this kind of situations.

Do you have any idea how I could make this work?

I'm actually thinking to write the same function as you're describing. Currently, I have two versions of the programs that work as expected: one is written in Ruby, and one is written in C++.

The way how it works in Ruby/C++ version is using a select to listen the IO ready event. (I think epoll/poll/kqueue/iocp will work too, but I didnt use them as select is the first one I saw when I search in Ruby)

Now I would like to port the functionality to Rust out of practicing Rust but I didn't have much spare time to do it yet.

I did a simple search and it seems that Rust has some binding to the epoll that you might be able to try.

The pseudo-code is

select(read_ios) // this will block until some io is ready
if it is stdout, then read from stdout
if it is stderr, then read from stderr

The easy workaround would be to spawn a thread to do the reading for one of the outputs. I think it's a little unfortunate std doesn't have anything ready-made for this, not sure if there's some problem with implementing it or it just hasn't come up.

I'm not familiar how to do this in Rust, but with C/C++, you would need to set your stdout and stderr file descriptors into non-blocking mode, and use the select system call (or poll, etc.) to multiplex between them. That way, you could respond to them as equals, not ever forcing stderr to always follow stdout, and it would fully handle the case where a command only outputs to one or the other descriptor. It sounds like you will need to do the same (somehow) in Rust.

Apologies for not providing a canned solution, but hopefully this will at least kindle some ideas on how to break free of your dilemma going forward.

Generally speaking, when working with stdout/stderr from child processes, I found the duct crate to be helpful.

2 Likes