Crate-level tests

[tczajka]

The Rust book recommends splitting your tests into integration tests (that only use pub API) and unit tests (that can use everything). What is missing for me is tests for pub(crate) items. I rarely want to test module-private items, more often I want to test pub(crate) items. Hence, here is how I have started to organize tests:

|
+- src
|  +- lib.rs
|  +- a.rs    // private tests, if any, go here
|  +- b.rs
|  +- tests   // pub(crate) API tests go here
|     +- mod.rs
|     +- a.rs
|     +- b.rs
+- tests  // integration tests go here
   +- integration_test1.rs
   +- integration_test2.rs

Thoughts?

[kpreid]

This is a perfectly good structure if you prefer it, but for myself, I don't move my tests to the top level like that; I just write the tests in or near the module they're testing.

The value of putting integration test code into tests/ is that you can be certain that it does not use any private items that you forgot to make public, so it accurately tests the usability of your library's public interface. However, for items which are private to the crate (or any narrower scope), there is no similar advantage, because if you make such a mistake you can always change it when you notice, whereas users of your published library can't just change it on the spot.

[tczajka]

@kpreid

However, for items which are private to the crate (or any narrower scope), there is no similar advantage, because if you make such a mistake you can always change it when you notice

But you might never notice that the unit test is testing using some private details rather than the same API that the other modules are using, and then you might never fix it. Putting the test outside of the module forces you to use the same API in the test that is available to the other modules.

[jumpnbrownweasel]

You have a point, if the goal is to be certain you're testing with pub(crate) visibility.

I have wished I could do use pub ... or use pub(crate) ... to restrict the visibility of the imports, so I could put all three types of tests in the module being tested. But we already have pub use ... for a different purpose and that would add confusion.

[Hyeonu]

@tczajka

But you might never notice that the unit test is testing using some private details rather than the same API that the other modules are using, and then you might never fix it.

It's true but the worst thing that can happen due to this bug is making your own crate and its unit tests fail to build. You can prevent it by running CI before every release, which is highly recommended out of context.

[jjpe]

@tczajka

But you might never notice that the unit test is testing using some private details rather than the same API that the other modules are using, and then you might never fix it.

Of course I can't speak for others, but I my 8-ish years of writing Rust code, I've never accidentally mislabeled a fn/method in terms of visibility. Never even came close really.

Requirements do change from time to time and so I'll make corresponding changes, e.g. a method that starts as private being made pub. But that doesn't mean that what came before was an accident.

[tczajka]

@Hyeonu

It's true but the worst thing that can happen due to this bug is making your own crate and its unit tests fail to build.

@jjpe

Of course I can't speak for others, but I my 8-ish years of writing Rust code, I've never accidentally mislabeled a fn/method in terms of visibility. Never even came close really.

Here is an example of how a unit test can build and pass, and yet a bug slips through. And it's not about mislabeling visibility.

The mistake would be caught if the test was written outside of the module and thus was only allowed to use the pub(crate) API.

mod vector {
    pub(crate) struct Vector {
        len: usize,
        capacity: usize,
    }

    impl Vector {
        pub(crate) fn new() -> Self {
            Vector {
                len: 0,
                capacity: 1,
            }
        }

        pub(crate) fn capacity(&self) -> usize {
            self.capacity
        }

        pub(crate) fn len(&self) -> usize {
            self.capacity // BUG
        }

        pub(crate) fn push(&mut self, elem: i32) {
            self.capacity *= 2;
            self.len += 1;
        }
    }

    #[cfg(test)]
    mod tests {
        use super::*;

        #[test]
        fn test_push_increases_len_by_1() {
            let mut v = Vector::new();
            v.push(42);
            assert_eq!(v.len, 1);
        }
    }
}

Rust Playground