Running cargo test as a command

I have a binary mybin with two tests, foo and bar. If I run

cargo test foo --bin mybin --  --nocapture arg1

on my Mac, only test foo runs, as I expect. I now want to add an optional argument, so I try

cargo test foo --bin mybin -- --nocapture arg1 ""

and both tests run. If I change it to

cargo test foo --bin mybin -- --nocapture arg1 " "

only test foo runs.

I know I can handle the optional argument differently, but I'd like to understand what's going on.

And while I have your attention, why does --nocapture come after the --? It makes argument handling harder.

Answering this will answer your first question, actually. In order to run #[test] code, rustc links a “test harness” into the test binary which provides its own main() that runs all the test functions. The test harness code is responsible for capturing output, so it needs the --nocapture flag, whereas cargo test, as far as I know, only concerns itself with getting the test binary built and run and doesn't parse any flags that are to be forwarded.

The test harness is also responsible for filtering tests based on the test name argument(s) it is passed — cargo test merely has a special case for forwarding one non-option argument in addition to the arguments after --. So, the reason that adding "" runs all the tests is probably that it is filtering tests against the empty string (they all match), and " " matches no tests since test names don't contain spaces.

If you want to be able to pass global options to your test code, you'll need to do something like one of these options:

  1. Make them unambiguously unequal to test names
  2. Somehow strip them out before the test harness sees them
  3. Configure your target with
    [[bin]]
    name = mybin
    harness = false
    
    to disable the harness code and only run your own code — but note that this means there is no longer an automatic distinction between cargo test --bin mybin and cargo run --bin mybin; you have to implement the “run tests” behavior yourself with #[cfg(test)] on your main() (and there's no way to discover #[test] tests yourself, as far as I know, so you'd need your own mechanism for declaring and running tests).

Thanks for the explanation, but the more things I try the more confused I get.

cargo test foo --bin mybin

runs all tests in mybin that match the pattern foo.

cargo test "" --bin mybin

runs all tests in mybin as you explain. So does

cargo test foo --bin mybin -- ""

which doesn't make sense to me. Even more confusing to me

cargo test x --bin mybin --

doesn't run any tests, but

cargo test x --bin mybin -- foo

runs test foo. That appears to mean that I can't pass an argument to my program that is a substring of tests I don't want to run.

I'm so confused, I decided to check the arguments being passed to my binary.

cargo test x --bin mybin -- --nocapture foo

Test foo runs, and the program gets "x, --nocapture, foo". Not what I expected.

Because cargo test has a special case to pass one non-option argument to the binary it's running.

I believe it's taking each of the arguments as a substring pattern and ORing them: thus, running all tests matching "foo" and all tests matching "" (which all tests do). The intended way to use this functionality would be to run cargo test -- foo bar to run the tests foo() and bar(), or even foo::a(), bar::x(), bar::y()

1 Like

First of all, the reference for cargo test is a good read.

Are you trying to supply arguments to your test? If yes, you're not getting to anywhere, I believe. My understanding is that cargo test <test_options> -- <binary_options> doesn't pass command line arguments to each individual test. Note that binary_options are meant for the test binary(not your application binary) which was compiled by the toolchain to execute all the tests. So again <binary_options> is NOT for your test function or binary.

Checkout the chapter regarding testing binary crate -- Test Organization - The Rust Programming Language

This is one of the reasons Rust projects that provide a binary have a straightforward src/main.rs file that calls logic that lives in the src/lib.rs file. Using that structure, integration tests can test the library crate with use to make the important functionality available. If the important functionality works, the small amount of code in the src/main.rs file will work as well, and that small amount of code doesn’t need to be tested.

Thanks for the explanation. I've read each of those links several times, but I didn't catch the bit about argument handling.

I am testing a multi-process application, and I need to pass parameters to the test. The main process starts a number of workers, each running the same binary. The arguments to that binary give them information they need to communicate with each other. I have it working now by skipping the arguments to the test harness.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.