How to process output of running command and clean up when done?

I found this example in The Rust Cookbook:

use std::process::{Command, Stdio};
use std::io::{BufRead, BufReader, Error, ErrorKind};

fn main() -> Result<(), Error> {
    let stdout = Command::new("journalctl")
        .stdout(Stdio::piped())
        .spawn()?
        .stdout
        .ok_or_else(|| Error::new(ErrorKind::Other,"Could not capture standard output."))?;

    let reader = BufReader::new(stdout);

    reader
        .lines()
        .filter_map(|line| line.ok())
        .filter(|line| line.find("usb").is_some())
        .for_each(|line| println!("{}", line));

     Ok(())
}

which is almost exactly what I want, except that the code fails to wait() on the child process. I want to run a subprocess, capture its stdout to a pipe to which I can attach a BufReader, and, when the process completes, invokes wait() in order to clean up.

Before I start inventing a new wheel, I thought I should if this has been done already.

My new wheel might look like a new struct which spawns the subprocess when created, and which provides it's own implementation of the Read trait, which reads from the subprocess's stdout until it receives an EOF, and then either invokes wait() on the child upon detection of EOF, or when the structure is Drop'd.

I feel like this must have already been done somewhere in the standard library, or perhaps a dedicated crate.

Instead of using spawn, use output. That will cause it to "wait" for the child process to end.

Thank you @RedDocMD. I want to process stdout as it is generated, and not wait for it all to accumulate in an unnecessarily large Vec<u8>.

I believe your code should actually end up waiting for the subprocess to end, since your for_each call on the iterator should actually wait for the stdout pipe to be closed before proceeding which should happen when the subprocess ends.

Example playground

@drewkett , I don't think it does...

spawn returns Result<Child>.
The documentation for Child states:

There is no implementation of Drop for child processes, so if you do not ensure the Child has exited then it will continue to run, even after the Child handle to the child process has gone out of scope.

Calling wait (or other functions that wrap around it) will make the parent process wait until the child has actually exited before continuing.

I want to keep reading and processing lines from the child process. There may be LOTS of lines, of which I only need a small fraction. I don't want to tie up a bunch of memory and time waiting to accumulate all of the output of the child process... I just want to examine the lines as they are produced and select those which interest me. I could pipe the result of my command through grep, and maybe I'll do that in the end... but I thought it would be more efficient to process the results directly in my application, and not fork off another command (grep).

But, from reading the documentation, it seems like I need to call wait on the child process. It seems like I shouldn't be the first person who has had to deal with this... so I thought I would ask before I invent my own solution.

Use duct, specifically duct::cmd!("cmd", "arg1", "arg2").reader()? (doc) to get a Read that you can read incrementally from.

2 Likes

To be clear I would not write a function the way that code is written, since it’s not very explicit, but the for_each call won’t complete until the iterator reaches the end which won’t happen until the underlying pipe closes. And the underlying pipe shouldn't close until the process dies.

A way to do what you want in stdlib more explicitly would be to spawn a thread, move the stdout pipe to that thread for reading and wait for the child process on the main thread (and then wait for the spawned thread).

@louy2 - Thanks! duct is exactly what I was looking for!

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.