Problem when interacting with a std::process::Command

I'm doing a challenge where a linux executable outputs to stdout a simple expression that I need to solve and send back the answer to its stdin under a certain time constraint.

Running the program shows something like this:

Challenge: 1+1
Setting alarm...
Solution? 

Note that there is no newline \n after ? .

For this I am using std::process::Command::spawn():

    let mut child = Command::new(program)
        .stdin(Stdio::piped())
        .stdout(Stdio::piped())
        .spawn()
        .unwrap();
    match (child.stdin.as_mut(), child.stdout.as_mut()) {
        (Some(stdin), Some(stdout)) => {
            // 1) Read from subprocess' stdout
            // 2) Evaluate
            // 3) Write to subprocess' stdin
            // 4) Read from subprocess' stdout
        }
        _ => panic!("Could not get stdin or stdout"),
    }

Unfortunately I cannot achieve the steps:

  1. Read from subprocess' stdout
  2. Evaluate
  3. Write to subprocess' stdin
  4. Read from subprocess' stdout

First issue: if I don't wrap the stdin into a BufWriter I get a Broken pipe with write!(stdin, "{}", answer).unwrap();. Is that normal?

After using the BufWriter I get to step 4. But then everything fails...

If I try to read one byte at a time, I only get the first two lines of step 1. I cannot get the Solution? . If I run unbuffer instead and move program to Command's arg(), I can read up to the Solution? . But in both cases, there isn't anything left in stdout!

I know that the program should outputs something. Timing the whole execution is around 50 ms, and the program times out after 200 ms (with SIGALARM). In that case the program should reply success or failure based on the answer.

All this tells me there is something fishy with buffers:

  1. stdin needed to be wrapped in a BufWriter
  2. Using unbuffer allows me to read (one byte at a time) until the last character of step 1.
  3. It's as if the program never received my input and just exited.

Does anyone would have a clue as to how I can solve this??

Thanks!!

This definitely doesn't sound right. Are you sure the BufWriter isn't just hiding whatever error is happening by not writing to the underlying stream?

As I understand it, BufWriter will buffer characters to write until it gets a certain amount, and then flush. If you don't reach that amount, and never flush it, it will never actually write those bytes - so if there's an error using the underlying stream, you just wouldn't get it.

If you use flush() after writing the bytes to the BufWriter, do you get the same error?

Are you sure the BufWriter isn't just hiding whatever error is happening by not writing to the underlying stream?

This is what I suspect...

Since the post I've tried to see how the program exited. It does gets killed by the SIGALARM, so that's probably why all this IO is failing. Maybe it doesn't get my reply fast enough and kills itself. :frowning:

1 Like

Yeah, if it has a timeout like that, that could be what's causing it. "Broken pipe" seems a lot like the program has either died, or intentionally closed it's own stdin (which it is allowed to do).

I think this might be related to your not being able to read all of Solution? . If you try to read this using something like a buffered reader, or using something like getting a full line, then it won't work since there is no newline at the end. It's hard to say what the exact problem is without seeing your code for reading it, though.

But guessing, I think what happened might have been:

  • the program runs
  • you wait for reading "Solution? ", but can't because the program hasn't output "\n"
  • the program times out and exits
  • it exiting closes it's stdout, so you are now able to read "Solution? "
  • you try to write to stdin, but it fails, since the program has exited

I'd recommend testing with a different command line program to see if you can solve this without the timeout present. As a debugging tool, maybe you could have the child program also output its status to stderr so you can see what both programs are trying to do at the same time?

According to the man page:

Normally, unbuffer does not read from stdin. This simplifies use of unbuffer in some situations. To use unbuffer in a pipeline, use the -p flag. Example:

process1 | unbuffer -p process2 | process3

e: Unfortunately I didn't actually try unbuffer -p before I posted. I'm not seeing this resolve the issue with buffering at all. And it looks like the solution (below) is unrelated to this command anyway!

it exiting closes it's stdout , so you are now able to read "Solution? "

Interesting, I haven't though about the flushing of the buffer at exit! This makes a lot of sense.

In the mean time I was able to pass the challenge. It turned out my Rust was not fast enough and the program did timed out. I though it was timing out after 200 ms, but I mixed it up with 200 microseconds. My time measurement, while probably not that precise, was reporting between 10 and 50 milliseconds run. I though that was plenty, but it's way too large for the 0.2 ms requirement!

To achieve the challenge I had to:

  • Defer any print statement to after the answer was read back. Locking stdout and formatting the string was probably costing too much.
  • Tweak a bit how data was read from the program's stdout. I wanted to avoid buffering as much as possible so I used Read::read() in a loop using a pre-allocated buffer and keeping track of how many bytes were read manually. It's a bit too low level for my liking as indices in the buffer must be manually tracked but it worked fine.

I though of optimizing the calculation of the solution since it's using Boxes a lot, but it was actually fast enough to get to the answer.

But the answer was wrong! Why? Undefined behaviour, yay! The challenge is a "simple" expression of integer additions, subtractions and multiplications. It can simply be eval()ed in Python. But Python is able to use big-ints for arbitrary precision. So I've used the num-bigint crate to perform the calculation. It turns out that the program has signed integer overflow, which is undefined (in C)! The behaviour turned out to be wrapping, so changing to i64 and wrapping_{add,mul,sub} allowed me correctly calculate the solution on every run. Other solutions I found online used C or C# but seemed unaware of undefined behaviour. Rust is just amazing! :heart:

2 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.