How to use external crate in Cargo integration tests?


#1

Hi. Here is the situation: I have a library with multiple integration tests run by Cargo, and I want to use some common utilities in the tests. This question has already been answered several times:

In all cases the suggested solution is to move the common code into a subdirectory and include it into the integration tests as a module. I am currently using this and it works fine, but has one issue: if the common code has self-tests as well, these tests are repeatedly run in each integration test. This is expected as Cargo compiles each tests/*.rs file as a separate crate, and each crate contains a copy of a module with #[test]s.

I suppose it is possible to avoid meaningless repetition of self-tests if the common utility code is moved into a separate crate which is then linked into every integration test. However, Cargo does not seem to have a way to link some β€˜third-party’ crate into the integration tests, it only links them against the library being tested.

I also tried to include the utility code into the tested library itself, marking it with #[cfg(test)]. The idea was to compile the utility code only for tests and import it from the integration tests. However, Cargo seems to build the tested crate normally, without the test option, so the utilities are skipped and are absent in the resulting crate.

Ideas, anyone? Thanks in advance.


#2

I think dev-dependencies can do that: http://doc.crates.io/specifying-dependencies.html#development-dependencies


#3

Heh, nice! Dev-dependencies do exactly what I wanted to do. Thank you for the pointer.

Here is the library layout:

libexample
β”œβ”€ Cargo.toml
β”œβ”€ lib.rs
└─ tests
   β”œβ”€ test0.rs
   β”œβ”€ test1.rs
   └─ utils
      β”œβ”€ Cargo.toml
      └─ lib.rs
Source code

Cargo.toml

[package]
name = "example"
version = "0.1.0"

[lib]
name = "example"
path = "lib.rs"

[dev-dependencies]
utils = { path = "tests/utils" }

lib.rs

#[cfg(test)]
extern crate utils;

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Number(pub u32);

pub fn double(n: Number) -> Number {
    Number(2 * n.0)
}

#[cfg(test)]
mod tests {
    use super::*;
    use utils;

    #[test]
    fn test() {
        assert_eq!(double(Number(2)), Number(utils::double(2)));
    }
}

tests/test0.rs

extern crate example;
extern crate utils;

use example::Number;

#[test]
fn test() {
    assert_eq!(example::double(Number(0)), Number(utils::double(0)));
}

tests/test1.rs

extern crate example;
extern crate utils;

use example::Number;

#[test]
fn test() {
    assert_eq!(example::double(Number(1)), Number(utils::double(1)));
}

tests/utils/Cargo.toml

[package]
name = "utils"
version = "0.1.0"

[lib]
name = "utils"
path = "lib.rs"

tests/utils/lib.rs

pub fn double(n: u32) -> u32 {
    2 * n
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test() {
        assert_eq!(double(0), 0);
        assert_eq!(double(1), 2);
        assert_eq!(double(2), 4);
    }
}

This allows usage of utils in both integration tests (tests/test{0,1}.rs) as well as in inline unit-tests (lib.rs). Unit-tests for utils itself can be run with a separate cargo test in tests/utils, they are not executed during example build.

Though, there is still one caveat if I try to define utilities that use types from example (e.g., fn equal(lhs: Number, rhs: Number) -> bool). In this case unit-tests in example cannot use these utilities while other tests can. The reason is that these types are considered different for some reason:

error[E0308]: mismatched types
  --> lib.rs:18:23
   |
18 |         assert!(equal(Number(4), double(Number(2))));
   |                       ^^^^^^^^^ expected struct `example::Number`, found struct `Number`
   |
   = note: expected type `example::Number`
   = note:    found type `Number`

Well, circular dependencies are bad, but I guess I’ll go ask about that in rustc issues. I believe this is because example-for-itself and example-as-dependency are actually different crates and this is expected behavior.


#4

Well, okay. It seems I have understood why that fails while I’ve been spelling the issue out as a to-be-posted issue in Cargo.

Consider the following project layout (source code):

                      libexample
[dev-dependencies]    β”œβ”€ Cargo.toml      <----------+
        |             β”œβ”€ lib.rs                     |
        |             └─ tests                      |
        |                β”œβ”€ test.rs                 |
        |                └─ utils                   |
        +------------->     β”œβ”€ Cargo.toml    [dependencies]
                            └─ lib.rs

There are multiple tests there:

  • example has #[cfg(test)] unit tests using utils
  • tests/test.rs is an integration test using both example and utils
  • utils have their own #[cfg(test)] unit tests

Now, libexample builds fine:

$ cargo build

Unit tests for tests/utils build and run fine:

$ cd tests/utils
$ cargo test

The integration test tests/test.rs builds and runs fine:

$ cd -
$ cargo test --test test

But unit tests for example fail to build:

$ cargo test --lib
   Compiling example v0.1.0 (file:///home/ilammy/dev/libexample)
   Compiling utils v0.1.0 (file:///home/ilammy/dev/libexample/tests/utils)
error[E0308]: mismatched types
  --> lib.rs:18:23
   |
18 |         assert!(equal(Number(4), double(Number(2))));
   |                       ^^^^^^^^^ expected struct `example::Number`, found struct `Number`
   |
   = note: expected type `example::Number`
   = note:    found type `Number`

error[E0308]: mismatched types
  --> lib.rs:18:34
   |
18 |         assert!(equal(Number(4), double(Number(2))));
   |                                  ^^^^^^^^^^^^^^^^^ expected struct `example::Number`, found     struct `Number`
   |
   = note: expected type `example::Number`
   = note:    found type `Number`

error: aborting due to 2 previous errors

error: Could not compile `example`.

To learn more, run the command again with --verbose.

It seems that in this case utils use their own example-as-utils-dependency crate, and example-as-itself is a different one and cannot be reused.

I expect The dependency graph to look something like this:

       +-------> (1) example.rlib <---- (3) utils.rlib (with #[cfg(test)])
       |          ^                      ^
tests (5)         |                      |
       |          |                      |
       +-------> (2) utils.rlib         (4) utils
                  ^
                  |
                  |
                 (6) example.rlib (with #[cfg(test)])
                  ^
                  |
                  |
                 (7) example

(1) and (6) are different crates and their Numbers are in fact different and cannot be used interchangeably. Unit tests in (7) create Numbers from (6) while its dependency (2) works only with Numbers from (1).

--verbose output seems to confirm this:

Cargo output
$ cargo test --lib --verbose
   Compiling example v0.1.0 (file:///home/ilammy/dev/libexample)
     Running `rustc lib.rs --crate-name example --crate-type lib -g
        -C metadata=b5e747e7654ba59a --out-dir /home/ilammy/dev/libexample/target/debug/deps
        --emit=dep-info,link -L dependency=/home/ilammy/dev/libexample/target/debug/deps`
   Compiling utils v0.1.0 (file:///home/ilammy/dev/libexample/tests/utils)
     Running `rustc tests/utils/lib.rs --crate-name utils --crate-type lib -g
        -C metadata=39d3d7de3615e756 --out-dir /home/ilammy/dev/libexample/target/debug/deps
        --emit=dep-info,link -L dependency=/home/ilammy/dev/libexample/target/debug/deps
        --extern example=/home/ilammy/dev/libexample/target/debug/deps/libexample.rlib`
     Running `rustc lib.rs --crate-name example -g --test
        -C metadata=0a02a15d79a43617 -C extra-filename=-0a02a15d79a43617
        --out-dir /home/ilammy/dev/libexample/target/debug/deps
        --emit=dep-info,link -L dependency=/home/ilammy/dev/libexample/target/debug/deps
        --extern utils=/home/ilammy/dev/libexample/target/debug/deps/libutils.rlib`
error[E0308]: mismatched types
  --> lib.rs:18:23
   |
18 |         assert!(equal(Number(4), double(Number(2))));
   |                       ^^^^^^^^^ expected struct `example::Number`, found struct `Number`
   |
   = note: expected type `example::Number`
   = note:    found type `Number`

error[E0308]: mismatched types
  --> lib.rs:18:34
   |
18 |         assert!(equal(Number(4), double(Number(2))));
   |                                  ^^^^^^^^^^^^^^^^^ expected struct `example::Number`, found struct `Number`
   |
   = note: expected type `example::Number`
   = note:    found type `Number`

error: aborting due to 2 previous errors

error: Could not compile `example`.

Caused by:
  process didn't exit successfully: `rustc lib.rs --crate-name example -g --test
        -C metadata=0a02a15d79a43617 -C extra-filename=-0a02a15d79a43617
        --out-dir /home/ilammy/dev/libexample/target/debug/deps
        --emit=dep-info,link -L dependency=/home/ilammy/dev/libexample/target/debug/deps
        --extern utils=/home/ilammy/dev/libexample/target/debug/deps/libutils.rlib`
    (exit code: 101)

$ cargo test --test test --verbose
       Fresh utils v0.1.0 (file:///home/ilammy/dev/libexample/tests/utils)
   Compiling example v0.1.0 (file:///home/ilammy/dev/libexample)
     Running `rustc tests/test.rs --crate-name test -g --test
        -C metadata=1dfc87e5a6315fee -C extra-filename=-1dfc87e5a6315fee
        --out-dir /home/ilammy/dev/libexample/target/debug
        --emit=dep-info,link -L dependency=/home/ilammy/dev/libexample/target/debug/deps
        --extern utils=/home/ilammy/dev/libexample/target/debug/deps/libutils.rlib
        --extern example=/home/ilammy/dev/libexample/target/debug/deps/libexample.rlib`
    Finished debug [unoptimized + debuginfo] target(s) in 0.30 secs
     Running `/home/ilammy/dev/libexample/target/debug/test-1dfc87e5a6315fee`

running 1 test
test test ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured