Std::process is escaping a raw string literal when I don't want it to

#21

it shouldn’t be getting

notepad "test \"something in quotes\" "

it should be getting

notepad "test "something in quotes" "

because that’s the literal string I passed. If the quotes needed escaping that’s my job as the caller to do in the string I’m sending as arguments.

Different programs interpret layered quotes differently, some process them in sequence, others from the outside in. The library shouldn’t be assuming it knows better than me as the programmer (or should at least allow an args_raw() option)

And to be clear I’m not trying to lay that responsibility at your feet, it’s not your fault. I’m just working out how to clearly convey it’s actually a bug.

#22

No, absolutely not. For two reasons:

  • The way you build a string in Rust has absolutely no meaning. r#"""# and "\"" and "\x22" are 100% identical in all cases, there’s no trace of any difference anywhere. They all contain exactly one byte ", in all cases.

  • Command::arg() by design explicitly takes role of preserving the argument with any content you put in, and deliberately doesn’t require the programmer to escape anything.

It’s fair to argue that arg() is impossible to implement on Windows, because the entire concept of a command argument technically doesn’t exist on Windows. But Rust’s stdlib chooses the most sensible alternative of assuming arguments do exist in the way CommandLineToArgvW defines them. So in a sense Command only supports running executables using CommandLineToArgvW, and doesn’t support others.

#23

These seem to contradict, can you clarify?

#24

The idea is that if I put abc"'<>&x in arg(), then the executed command will get exactly abc"'<>&x in its env::args().

I can give abc" to arg() and don’t have to give abc\". And I can run .arg(" ").arg(" ") and the receiving executable will get env::args() with [" ", " "], and not [" "] or nothing.

Note that arg cares about content of the string not syntax, so

let mut s = String::new();
s.push(34 as char);
cmd.arg(&s);

is same as cmd.arg("\"").

#25

Can provide an example of this happening? Because I’m pretty sure it’d actually get

abc\"'<>&x
#26

Make an exe from this:

fn main() {
   for arg in std::env::args() {
      println!("the arg is >{}<", arg);
   }
}

And then try running it from another program using Command

Rust supports separate arguments, so it will parse them with quoting.

cmd.exe /c and echo don’t support separate arguments, so they won’t understand what quoted arguments mean.

#27 
    use std::env;
    use std::io;

    fn main() {
    for arg in std::env::args() {
        println!("the arg is >{}<", arg);
    }
    
    let mut input = String::new();
    if let Ok(_discard) = io::stdin().read_line(&mut input) {}
    }

image
image.png1058x740 62.1 KB

Edit: changed the arguments to make the problem more clear
also putting text here to make this post more findable
test.a test.b "test.c test.d" \"test.e test.f\"

#28

notice the difference between the escaped and unescaped quotes?

compare the command line that gets passed here, to the one that get’s passed to notepad.exe above

Notice that \"test.e test.f\" come as separate arguments, not a single argument separated by a space. This is how rust CALLS other programs if I try to pass the "test.c test.d" style, which creates problems.

#29

Interpretation of quoted arguments in Windows is optional, and left up to each individual application’s choice. I haven’t checked if notepad.exe chose to support quoted arguments. I wouldn’t be surprised if it assumed arguments are files, so quote isn’t a legal character, and has some half-assed custom parser. Same happens with dir.

The recommended interpretation of arguments is this:

#30

Note that \"test and "\"test" are completely different things.

#31

the problem is there’s no way to send the "test.c test.d" style when using std::process::Command, even if I explicitly make a string literal with that exact text

Try it, use your args lister code, call it from another rust program you make and try to get the 1 argument with a space style

#32

.arg("test.c test.d") sends "test.c test.d"
.arg(r#""test.c test.d""#) sends "\"test.c test.d\""

#33

try to get your args list to spit out

the arg is >a "b c" d<
#34

I’ve spent many days studying it very precisely, because I’ve written bug-for-bug-compatible reimplementation of CommandLineToArgvW:

Check out these tests:

The syntax is bizarre, but that’s Microsoft’s fault :slight_smile:

#35 
Command::new("rust-produced.exe").arg("a \"b c\" d")

will execute

rust-produced.exe "a \"b c\" d"

and the command will parse it and then print

the arg is >a "b c" d<

To be clear, the similarity between executed command and Rust string syntax is a coincidence. One doesn’t influence the other at all:

let mut s = String::new();
s.push('a');
s.push(' ');
s.push('"');
s.push('b');
s.push(' ');
s.push('c');
s.push('"');
s.push(' ');
s.push('d');
Command::new("rust-produced.exe").arg(s)

assert_eq!(&s, "a \"b c\x22 d");
assert_eq!(&s, r#"a "b c" d"#);

will also execute rust-produced.exe "a \"b c\" d"

#36

Try /S with cmd

#37

and

My a "b c" d example is actually incorrect, as can be seen in the post above when I send \" rust’s argument parser swallows the \" and gives ", and if I send just " it swallows that and doesn’t hand them inside. There’s probably an alternate way of calling std::env::args that gives the actual raw arguments, but that doesn’t disprove the core of the problem which is that there doesn’t seem to be any way to reliable send nested double quotes in command line arguments to non-rust cli programs on windows.

there’s no way (that I’ve yet seen) to get rust to call this, which should give a list of installed software (that I could then digest looking for office versions for a microsoft audit)
powershell -command "Get-WmiObject -class Win32_Product"

instead it will call this, which just echo’s back a string
powershell -command \"Get-WmiObject -class Win32_Product\"

It may very well be it derives from microsoft’s api, not from rust’s internals, but that doesn’t mean it’s not real, nor that it’s not a bug

#38

@kornel I found your updated post over on internals.rust and it seems like you’re agreed we may need a way to send exact strings without any additional (is “sugaring” the word?). I’ll move the discussion over there

Also sorry if my tone got contentious, it felt like I was having trouble convincing you it was actually escaping the quotes and that I wasn’t imagining things

#39

Sorry about the misunderstanding in the beginning. Let’s continue looking for solutions in the other thread.

#40

Hmm. @Xpyder, this works for me:

use std::process::Command;

fn main() {
    let command = "Get-WmiObject -class Win32_Product";
    let output = Command::new("cmd")
        .args(&["/c", "powershell", "-command", command])
        .output()
        .expect("failed to execute");

    let stdout = String::from_utf8_lossy(&output.stdout);
    println!("{}", stdout);
}

as does calling PowerShell directly:

let output = Command::new("powershell")
    .args(&["-command", command])
    .output()
    .expect("failed to execute");

The command takes dozens of seconds to complete, however.

Please note: my example uses .output() instead of .spawn() to wait for a process to exit.

Basically, if you have a command line to execute:

powershell -Command "$className = \"Win32_OperatingSystem\"; Get-CimInstance -ClassName $className"

you have to split and unescape it yourself:

&["powershell",
  "-Command",
  r#"$className = "Win32_OperatingSystem"; Get-CimInstance -ClassName $className"#
]

P. S. As a side note, it seems like Get-CimInstance is preferred over Get-WmiObject. (PS 3.0+)

upd: Ah, I see @kornel suggested exactly the same in his first comment.