Thoughts on "broken build" (unit tests failing) --- its not what you think :D

help
1

I'm new to rust, and the best way for me to learn is create a project that has some value to me. My project is to take cargo test output and reformat it into html. In order to test all of the test cases, I have to have some failing tests. But that means my CI fails. You can see it here

How should I handle this:

  1. Have a separate project that's not in the CI for generating test data
  2. Use the #[ignore] attribute on the test methods unless I need them to fail
  3. Is there a better option?

I don't like #2 because it means I'm likely to forget the ignore attribute from time to time. Seems ugly.

I was wondering if anyone had some thoughts about what I could do.

Thnx
Matt

2

Tag them with #[should_panic].

https://doc.rust-lang.org/book/second-edition/ch11-01-writing-tests.html#checking-for-panics-with-should_panic

3

Wait, I think you're asking for something more complicated...

If you want this to be conditional, you can use #[cfg_attr(not(feature = "fail"), ignore)] with a "fail" feature in Cargo.toml. Is that what you mean?

4

I think so. I would like my CI build to pass, and during CI unit tests are actual unit tests to run. I would like to be able to get failed and ignored unit test results when I am testing my app (locally). Can you explain a little bit more how that might work?

Matt

5

In your Cargo.toml, you'd have something like:

[features]
fail = []

Then with that cfg_attr I wrote applied to a test, cargo test would effectively apply #[ignore], and cargo test --features fail would run it like a normal test.

The test runner also has an --ignored option, so you could just enable all #[ignore] tests at once using cargo test -- --ignored. (The middle "--" separates cargo options before and test runner options after.)

6

That makes sense. I must be missing something. There is no difference in the number of tests run between the command "cargo test" and "cargo test --feature gen_data" (instead of calling it fail I called it gen_data).

The tests have been decorated like this:

#[cfg_attr(not(feature = "gen_data"), ignore)]
mod tests {
    #[test]
    fn fail_integration_test_1() {
        assert!(false)
    }

    #[test]
    fn fail_integration_test_2() {
        assert!(false)
    }
}

and the toml looks like this:

[package]
name = "copper"
version = "0.1.0"
authors = ["matt raffel <matt.raffel@evernym.com>"]
categories = ["development-tools", "development-tools::build-utils", "command-line-utilities"]

[features]
gen_data=[]

Matt

7

I think #[ignore] will only work directly on each #[test] fn, not on a whole module. But you could just mask the module out of existence with #[cfg(feature = "gen_data")].

(rustc is not always good at warning about unused attributes.)

8

Unfortunately I am not able to get this to work. I'm sure its something on my end, but I'm going to resolve this but putting ignore on those tests for now. Someday....:smiley:

Thank you for your help, regardless. I learned something new, that in it self is a win for me.
Matt