Bash Shell pipe redirects

help
1

Greetings Rustaceans!,

I have been trying to write a shell pipeline for the past couple of days.

This is what I would like to have,

'cmd1 args[...] <inp1> | cmd2 args[...] out_from_cmd1 <inp2> ' | 'cmd3 args <inp3> out_from_pipe1'

Please note the quotes around pipe1 and pipe2.

what I have tried:

let pipe1 = { Exec::cmd("samtools").args(&["fasta", "-f", "0x4", &sam_path]) | Exec::cmd("fasta").args(&["spl it", "interleaved", "-", &anchor_len.to_string()])}.stream_stdout().unwrap();

Hyphen ("-") in the second command is a placeholder for output from the first command. That worked so far!. Now to extend it to the third part, I have added

let test = { Exec::cmd("samtools").args(&["fasta", "-f", "0x4", &sam_path]) | Exec::cmd("fasta").args(&["split ", "interleaved", "-", &anchor_len.to_string()])| Exec::cmd("bowtie").args(&["-f", "-p1", "-v0", "-m1", "-B1", "-- suppress", "5,6,7,8", &genome_path, "-"])};

Well it compiles, no error. But the third command is not receiving any input. I have tried adding .stream_stdout().unwrap() after the second command, that doesn't compile.

I have same pipeline written in python:

testout = shell_stdout(
 'samtools fasta -f 0x4 %s | fasta split interleaved - %d | '
  'bowtie -f -p1 -v0 -m1 -B1 --suppress 5,6,7,8 %s -'
  % (sam_path, anchor_len, genome_path))

def shell_stdout(command):
       return subprocess.Popen(command, stdout=subprocess.PIPE, shell=True, executable='/bin/bash', bufsize=-1).stdout

Any help will be greatly appreciated. This is my first post in any forum. Please bear with me, If I have written anything stupid.

1 Like
2

No worries! I don't think you have.

I do need to ask a clarifying question though: What is Exec? Are you using some sort of library for this?

1 Like
3

This looks like subprocess. Did you mean to omit }.stream_stdout().unwrap(); in the third example?

4

Thanks!

Yes, I am using https://crates.io/crates/subprocess

1 Like
5

Thanks for the reply!

Yes! I am using subprocess. As I have mentioned in the query, I was trying to unwrap the stream_stdout after second command and before giving to the third command("bowtie"), that doesn't compile.

Also trying to use
use std::process::{Command} and manually collect stdout and feed stdin. But any luck yet.

6

I was trying to unwrap the stream_stdout after second command and before giving to the third command("bowtie"), that doesn't compile.

Why add stream_stdout to the second command instead of the third? The stdout of the second command is already being piped to the third command. Have you tried something like:

let test = { Exec::cmd("samtools").args(...) |
             Exec::cmd("fasta").args(...) |
             Exec::cmd("bowtie").args(...) }.stream_stdout().unwrap();
2 Likes
7

Thank You! That worked! Somehow I missed that. Though it works now, I realized it's a bad design and performance is not so great. I have decided to remove complex dependencies rather write native Rust code there.

Thanks again for your help!

1 Like