Why does Rust test framework lack fixtures and mocking?

I’m sorry, @Zephilim, I should’ve been clearer. Let me try to answer some of your questions:

Sure, that’s totally reasonable! Using a complex testing framework is a bit like running, though, right? Especially in Rust, where it will probably involve macros, or even (unstable) compiler plugins. That shouldn’t stop you from learning, though!

I haven’t seen any crate that automates this, aside from stainless (unstable compiler plugin…). So, you’ll have to write a bit more boilerplate.

A test runner? No, cargo test will do that for you (in the threaded way I tried to describe above). I was talking about:

#[test]
fn your_test() {
    setup();

    assert!(1 < 2);

    teardown();
}

fn setup() { /* ... */ }
fn teardown() { /* ... */ }

I mentioned cargo specifically because they include the test! macro that you saw to do setup and teardown stuff. You can find the sources here (I searched for macro_rules! test).

1 Like

Can the test attribute be set on member functions on a struct? In the setup and teardown, I would need to init and destroy state, for use in the body of a test. Nothing I’ve read or seen in any repo suggests this is possible (but I suppose I could try it out, but I can’t believe that I am the only person that would need this, afterall, this is the prevailing methodology used in xUnit test frameworks). As I type this, a whole host of issues have cropped up in my mind; ie the setup and teardown would need to be on an instance; so how would you create that instance, arghhh. (Sorry I’m just thinking out loud …, but I’m sure you get the gist)

Many of the fundamental libraries don’t require very sophisticated test frameworks, so I’m not surprised that there isn’t a multitude of such frameworks yet. But I don’t think it will stay that way.

If you want to get an idea how more complex tests could look, maybe have a look at Diesel’s tests? It at least has to set up a database schema and connection for many of them.

One thing you might have to adjust is the expectation of pervasive object-oriented style. For example, instead of having tests as a method on a TestSuite type struct you typically have test functions, which create the required state themselves or call a helper function to create the state.

Ah, and don’t be afraid to use macros, this is exactly the scaffolding that they are a good match for, in order to reduce boilerplate. A non-nightly test framework written today would probably make extensive use of macros.

argh, so that is where the test macro is. I was working my way down the files in the test dir, and its just typical that it the last file in that long list of test files.

Ok, I’m going to deep delve into cargo and diesel, and see what else I can do to start writing production quality real world unit tests.

Might be worth that Rust itself has quite a bunch of tests so browsing the Rust repro might give some ideas. Here is a bunch of tests for fs for example https://github.com/rust-lang/rust/blob/master/src/libstd/fs.rs#L1460

The Rust test crate was designed to be very minimal, but to be extended in the hypothetical future by more advanced test frameworks.

Here’s a recent discussion on extensible test frameworks that at least hints at the direction we want to go in, but there’s nobody working on it right now.

In the meantime, if you don’t want to pull in a crate to add test fixtures, there’s a simple pattern for setup and teardown: create a function called setup that does the setup and accepts a closure to run the test, and passes it any common state. Then your test looks like

#[test]
fn make_sure_foo_works() {
    setup(|&fixture_state| { ... });
}
2 Likes

Hi brson, thanks for that, very useful. [newbie mode] I just tried your prosposed work around, and can’t quite get it right. This is a mockup:

#[cfg(test)]
mod fixt {

    struct TestFixture { name : String }

    fn Setup( o: fn (tf : TestFixture) ) {
         o.name = "Initialised";
     }

    #[test]
    fn make_sure_foo_works() {
        Setup(| &x | {
            assert_eq!(x.name, "Initialised");
        } );
    }
}

results in the following errors:

c:\dev\sandpit\rust\expressions>cargo test
Compiling expressions v0.1.0 (file:///C:/dev/sandpit/rust/expressions)
src\tests\fixt.rs:8:10: 8:16 error: attempted access of field name on type fn(tests::fixt::fixt::TestFixture), but no field with that name was found
src\tests\fixt.rs:8 o.name = “Initialised”;
^~~~~~
src\tests\fixt.rs:14:24: 14:30 error: the type of this value must be known in this context
src\tests\fixt.rs:14 assert_eq!(x.name, “Initialised”);
^~~~~~
src\tests\fixt.rs:14:13: 14:47 note: in this expansion of assert_eq! (defined in )
:5:8: 5:18 error: the type of this value must be known in this context
:5 if ! ( * left_val == * right_val ) {
^~~~~~~~~~
src\tests\fixt.rs:14:13: 14:47 note: in this expansion of assert_eq! (defined in )
:5:22: 5:33 error: the type of this value must be known in this context
:5 if ! ( * left_val == * right_val ) {
^~~~~~~~~~~
src\tests\fixt.rs:14:13: 14:47 note: in this expansion of assert_eq! (defined in )
src\tests\fixt.rs:13:15: 15:10 error: mismatched types:
expected fn(tests::fixt::fixt::TestFixture),
found [closure@src\tests\fixt.rs:13:15: 15:10]
(expected fn pointer,
found closure) [E0308]
src\tests\fixt.rs:13 Setup(| &x | {
src\tests\fixt.rs:14 assert_eq!(x.name, “Initialised”);
src\tests\fixt.rs:15 } );
src\tests\fixt.rs:13:15: 15:10 help: run rustc --explain E0308 to see a detailed explanation
error: aborting due to 5 previous errors
Could not compile expressions.

So, I’m not quite there, can you assist? Thanks.

Try making the name field public. So struct TestFixture { pub name: String }. That’ll fix the complaint about there not being a field called name.

The closure issue is different though, and I’m on my phone so can’t mess with that one…

I realised in the Setup, I also need to call into the lambda. I changed name to be public so it now looks like this:

#[cfg(test)]
mod fixt {

    struct TestFixture {
        pub name : String
    }

    fn Setup( o: fn (tf : TestFixture) ) {
         o.name = "Initialised";
         tf(o);
     }

    #[test]
    fn make_sure_foo_works() {
        Setup(| &x | {
            assert_eq!(x.name, "Initialised");
        } );

        // Do a similar thing here with a teardown
    }
}

but now I get:

c:\dev\sandpit\rust\expressions>cargo test
Compiling expressions v0.1.0 (file:///C:/dev/sandpit/rust/expressions)
src\tests\fixt.rs:11:10: 11:12 error: unresolved name tf [E0425]
src\tests\fixt.rs:11 tf(o);
^~
src\tests\fixt.rs:11:10: 11:12 help: run rustc --explain E0425 to see a detailed explanation
src\tests\fixt.rs:10:10: 10:16 error: attempted access of field name on type fn(tests::fixt::fixt::TestFixture), but no field with that name was found
src\tests\fixt.rs:10 o.name = “Initialised”;
^~~~~~
src\tests\fixt.rs:17:24: 17:30 error: the type of this value must be known in this context
src\tests\fixt.rs:17 assert_eq!(x.name, “Initialised”);
^~~~~~
src\tests\fixt.rs:17:13: 17:47 note: in this expansion of assert_eq! (defined in )
:5:8: 5:18 error: the type of this value must be known in this context
:5 if ! ( * left_val == * right_val ) {
^~~~~~~~~~
src\tests\fixt.rs:17:13: 17:47 note: in this expansion of assert_eq! (defined in )
:5:22: 5:33 error: the type of this value must be known in this context
:5 if ! ( * left_val == * right_val ) {
^~~~~~~~~~~
src\tests\fixt.rs:17:13: 17:47 note: in this expansion of assert_eq! (defined in )
src\tests\fixt.rs:16:15: 18:10 error: mismatched types:
expected fn(tests::fixt::fixt::TestFixture),
found [closure@src\tests\fixt.rs:16:15: 18:10]
(expected fn pointer,
found closure) [E0308]
src\tests\fixt.rs:16 Setup(| &x | {
src\tests\fixt.rs:17 assert_eq!(x.name, “Initialised”);
src\tests\fixt.rs:18 } );
src\tests\fixt.rs:16:15: 18:10 help: run rustc --explain E0308 to see a detailed explanation
error: aborting due to 6 previous errors
Could not compile expressions.

Take a look at the book on taking closures as arguments.

I think what you’re after is something like this:

fn Setup<F>( tf: F ) where F: Fn (&TestFixture) {
    let o = TestFixture { name: String::from ("Initialized") };
    tf (&o);
}

So you can pass in a borrowed fixture, do you standard setup, and then pass it to your closer to execute.

That should hopefully work :smile:

1 Like

KodrAus, thanks for that. This is my solution based on your fix:

#[cfg(test)]
mod fixt {

    struct TestFixture {
        pub name : String
    }

     fn setup<F>( tf: F ) where F: Fn (&TestFixture) {
         let o = TestFixture { name: String::from ("Initialised") };
         tf (&o);
     }

    #[test]
    fn make_sure_foo_works() {
        setup(| x | {
            assert_eq!(x.name, "Initialised");
        } );

        // Do a similar thing here with a teardown although need
        // to figure out how to get hold of the orginal TestFixture
        // to perform cleanup on it.
    }
}

This now works. I’ve not done that much rust yet, which is why I’ve stumbled on many of the issues. I’ve read most of the rust book, but it hasn’t all sunk in yet. I just needed something like this so I can start to write some meaningful tests. Thanks for the leg up :slight_smile:

4 Likes

No problem! You’re actually already using quite a lot of Rust syntax in your example test fixture, so there aren’t too many more surprises in syntax. The rest is borrow checking :slight_smile:

To bring closure to this question, I am posting a complete solution to the problem of not having fixtures. Also, the naming of the variables in my previous post was not particluarly helpful, in actual fact, I can see how it could be confusing to the reader, so I decided the re-write it for clarity. The solution I propose requires the use of a macro as you will see (inspired by the test! macro as defined in cargo). To paraphrase, what I wanted was the ability to define a test fixture with both setup and teardown methods (a la xUnit), which I could use to enforce principle of DRY (don’t repeat yourself).

struct FooTestFixture {
    pub name : String
}

impl FooTestFixture {
    fn setup() -> FooTestFixture {
        FooTestFixture { name: String::from("Initialised") }
    }
}

fn teardown(fixture : &mut FooTestFixture) {
    fixture.name = "".to_string();
}

#[test]
fn heap_foo_fixture_should_be_initialised() {
    let mut fixture : FooTestFixture = FooTestFixture::setup();

    assert_eq!(fixture.name, "Initialised");

    teardown(&mut fixture);
}

macro_rules! unit_test {
    ($name:ident $fixt:ident $expr:expr) => (
        #[test]
        fn $name() {
            let mut $fixt : FooTestFixture = FooTestFixture::setup();
            $expr;

            teardown(&mut $fixt);
        }
    )
}

unit_test! (heap_foo_fixture_should_be_initialised_using_macro f {
    assert_eq!(f.name, "Initialised");
});

The unit_test! macro $name is the name of the test, $fixt refers the binding that represents the test fixture and $expr is the body of the test. As it stands, this macro would need to be defined for every test module with the same test fixture, and some might say this limits it’s use, but it does serve the purpose. Logically, to get round this the macro would need to be generic (but I don’t think there is such a feature in Rust!). I just discovered that I maybe able to abstract out the name of the fixture type (in this case FooTestFixture) by introducing a ty parameter to the macro …

Ok, so this is the end game; I’ve changed the macro to include an additional identifier representing the type of the fixture, so it can be used accross different test fixtures:

macro_rules! unit_test {
    ($name:ident $fixt:ident $ftype:ident $expr:expr) => (
        #[test]
        fn $name() {
            let mut $fixt = $ftype::setup();
            $expr;

            teardown(&mut $fixt);
        }
    )
}

unit_test! (foo_fixture_should_be_initialised_using_generic_macro f FooTestFixture {
    assert_eq!(f.name, "Initialised");
});

Done!

For fun (and practice) I have modified your solution at bit, changing the syntax to be more readable and making actually work with different fixture types. Your solution is not quite enough since teardown only takes a reference of type FooTestFixture.

macro_rules! unit_tests {
    ($( fn $name:ident($fixt:ident : $ftype:ty) $body:block )*) => (
        $(
            #[test]
            fn $name() {
                let mut $fixt = <$ftype as Fixture>::setup();
                $body
                $fixt.teardown();
            }
        )*
    )
}

trait Fixture {
    fn setup() -> Self;
    fn teardown(self);
}

struct FooTestFixture<T> {
    name: String,
    num: T,
}
impl FooTestFixture<i32> {
    fn add_some(&mut self) {
        self.num += 10;
    }
}
impl<T: Default> Fixture for FooTestFixture<T> {
    fn setup() -> FooTestFixture<T> {
        FooTestFixture {
            name: String::from("Initialised"),
            num: Default::default(),
        }
    }
    fn teardown(mut self) {
        self.name = "".to_string();
    }
}

unit_tests!{
    fn some_test_name(f: FooTestFixture<()>) {
        assert_eq!(f.name, "Initialised");
        f.name = "Foo".to_owned();
        assert_eq!(f.name, "Foo");
    }

    fn another_test(g: FooTestFixture<i32>) {
        assert_eq!(g.num, 0);
        g.add_some();
        assert_eq!(g.num, 10);
    }
}
6 Likes

Ah yes you’re right. I was so focused on the setup, that I forgot about teardown. Thanks for taking the time to notice this. I will have a look at your solution, and probabaly look to use that as a model for writing unit tests, rather than mine. Many thanks again.

Look at my mocking library mockers, you need mocks and I need feedback :slight_smile:

5 Likes

A post was split to a new topic: Mock API difficulties