Ideas for writing portable tests executing external programs?

[droundy]

I'm currently working on getting more CI working for bigbro, and am struggling a bit to get the doctests to work. Since bigbro is all about running external commands, I need some external commands to actually run! I have a set of commands that I've been using on linux, and most of them work (with tweaking) for Mac, but windows is another story.

Any suggestions for useful portable commands? Or for a nice way to organize the doctests to use non-portable commands? I've already switched to using std::env::temp_dir() to increase portability, but the trouble is that the documentation gets more challenging to understand as I start adding more checks in there, so I don't see a match to do different things on different platforms a great option.

As a more concrete question: sha1sum seems to be portable, but I need to find a file to take the checksum of. Any suggestions for an easy way using the rust standard library to identify some file, any file? Currently I'm just using /usr/bin/python, which obviously won't work on windows.

For the curious, you could see how (poorly?) my testing is going:

https://ci.appveyor.com/project/droundy/bigbro

Travis CI - Test and Deploy Your Code with Confidence

[kornel]

Perhaps you could use your own commands built with Rust?

You could add a project with commands to the workspace and declare it as a dev dependency, which would avoid adding the commands to crates.io and release version of your project.

[droundy]

@kornel

Perhaps you could use your own commands built with Rust?

Would that be possible to do this without creating a new crate, e.g. by creating the commands as tests or something? Any idea how to get them into the PATH when running doctests?

[kornel]

IIRC binaries of the crate are built before tests. But to separate test dummies from real project binaries it may be best to use another crate anyway. With workspaces it's easy.

[droundy]

I'm trying to put test binaries into a separate crate with a workspace, but am running into trouble where they are not getting built. My test crate (which has a src/bin/echo.rs) is listed in the main Cargo.toml as:

[dev-dependencies]

test-utils = { path = "test-utils" }

If I cd into test-utils and run cargo build then the echo executable is built and put in the root target/debug/echo, so the workspace is working properly, but when I run either cargo build or cargo test from the root, the test-utils crate is not built, and echo is not Any suggestions for debugging this?

[kornel]

Try adding extern crate test_utils; to tests/crates that need it.

[droundy]

Even though test-utils doesn't have a library?!

[kornel]

If I understand that correctly it should work for integration tests (I'm assuming that's tests/foo_test.rs):

How do you test binaries? (not libraries)

As part of my learning project of porting Apple's shell_cmds to rust, I would like to properly test things. Here's the thing. Some of the utilities are mostly just custom parsing of command-line arguments, which means I'm trying to test the binary itself. In my project, I have library code like this: src/command.rs. I can put unit tests of library code right in this file. That's fine. Then I've got binaries that look something like this: /src/bin/command.rs -- this binary is where the comm…

Cargo: how do I make a integration test A depend on binary B?

I've got a rust project that currently consists of a single [[bin]] target (but that could trivially be split up into a bin and a lib - main.rs does some trivial initialization and then calls a single method in a sub-module). I'm about to write some code that will involve running a sub-process. The easiest way I can think of to test this is to create a binary that just writes its command-line args and working directory to a temp file. The test can then make the code that should run the sub-proc…

[droundy]

Those links are helpful, but don't address the separate crate issue (with a workspace), which is what I'm running into. Probably it's not so super easy, and I should just generate excess binaries in my single crate.

[droundy]

Also, any suggestions for setting the PATH when running tests? Specifically in doc-tests?

[kornel]

Maybe @alexcrichton will be able to help how to test with binaries from a sub-crate in a workspace.

[alexcrichton]

Ah unfortunately I don't think there's a way to do that right now, have binaries from another member of the workspace available when you test the main crate. This may be possible with cargo test -p foo -p bar to "test both crates" but actually just pick up binaries from one crate.