Galvanic: a testing trifecta for assertions, mocks, and test setup


#1

This post will be updated to reflect the current state of galvanic. The subsequent posts are used for discussion (feel free to comment) and announcement of new features.


Galvanic is a testing trifecta consisting of

  • galvanic-assert — a library for fluent assertions with a comprehensive list of matchers.
    Further the library supports the assertion of panics and has support for structural matching, i.e., assertion-matchers integrated with pattern matching syntax.

  • galvanic-mock — a library for behaviour-driven mocking of (multiple) generic traits.

  • galvanic-test — a library targeted for writing test suites: setup and tear-down of test environments, writing/injecting test fixtures, and parameterised tests.

The three libraries are intendend to work independently of each other. So you can use only the parts you need. Or if you like parts of other testing libraries, mix them up as you wish. Nevertheless, as they are developed in concert you can expect that they are designed to integrate well with each other, e.g., use galvanic-assert matchers as predefined argument matchers in galvanic-mock.

Examples for galvanic-test

Test case setup with galvanic-test:

#[macro_use] extern crate galvanic_test;

test_suite! {
    use std::fs::{File, remove_file};
    use std::io::prelude::*;

    fixture input_file(file_name: String, content: String) -> File {
        members {
            file_path: Option<String>
        }
        setup(&mut self) {
            let file_path = format!("/tmp/{}.txt", self.file_name);
            self.file_path = Some(file_path.clone());
            {
                let mut file = File::create(&file_path).expect("Could not create file.");
                file.write_all(self.content.as_bytes()).expect("Could not write input.");
            }
            File::open(&file_path).expect("Could not open file.")
        }
        // tear_down is optional
        tear_down(&self) {
            remove_file(self.file_path.as_ref().unwrap()).expect("Could not delete file.")
        }
    }

    // fixtures with arguments must receive the required values
    test another_test_using_fixtures(input_file(String::from("my_file"), String::from("The stored number is: 42"))) {
        let mut read_content = String::new();
        input_file.val.read_to_string(&mut read_content).expect("Couldn't read 'my_file'");

        assert_eq!(&read_content, input_file.params.content);
    }
}

A simple parameterised test case which reuses the same test code for different values:

test_suite! {
    fixture product(x: u32, y: u32) -> u32 {
        params {
            vec![(2,3), (2,4), (1,6), (1,5), (0,100)].into_iter()
        }
        setup(&mut self) {
            self.x * self.y
        }
    }

    test a_parameterised_test_case(product) {
        let wrong_product = (0 .. *product.params.y)
                               .fold(0, |p,_| p + product.params.x)
                            - product.params.y%2;
        // fails for (2,3) & (1,5)
        assert_eq!(wrong_product, product.val)
    }
}
---- __test::a_parameterised_test_case stdout ----
        thread '__test::a_parameterised_test_case' panicked at 'assertion failed: `(left == right)`
  left: `5`,
 right: `6`', src/main.rs:16:8
note: Run with `RUST_BACKTRACE=1` for a backtrace.
The above error occured with the following parameterisation of the test case:
    product { x: 2, y: 3 }

thread '__test::a_parameterised_test_case' panicked at 'assertion failed: `(left == right)`

  left: `4`,
 right: `5`', src/main.rs:16:8
The above error occured with the following parameterisation of the test case:
    product { x: 1, y: 5 }

thread '__test::a_parameterised_test_case' panicked at 'Some parameterised test cases failed'

Examples for galvanic-assert

A simple example for fluent assertions in galvanic-assert:

assert_that!(&1+2, all_of!(greater_than(0), less_than(5)));

A complex assert on the properties of a struct:

let var1 = Baz::Variant1 { xs: vec![1,2,3,4], y: 23.4 };
assert_that!(&var1, has_structure!(Baz::Variant1 {
    xs: sorted_ascending(),
    y: lt(25.0)
}));

Examples for galvanic-mock

Mocking with galvanic-mock:

#[use_mocks]
fn simple_use_of_mocks() {
    let mock = new_mock!(MyTrait, MyOtherTrait<String>);

    given! {
        <mock as MyTrait>::foo(|&x| x == 15, |&y| true) then_return_from |&(x,y)| x+y times 2;
        <mock as MyTrait>::foo |&(x,y)| x <= y then_return 2 always;
    }

    expect_interactions! {
        <mock as MyTrait>::foo |&(x,y)| x <= y times 1;
        <mock as MyTrait>::foo |&(x,y)| x > y between 2,5;
    }

    assert_eq!(mock.foo(15, 1), 16);
    assert_eq!(mock.foo(15, 2), 17);
    assert_eq!(mock.foo(15, 15), 2);
    mock.verify();
}

The main motivation to develop those libraries is that I consider testing as an important (and often unliked) part of software development, regardless of scale. Good tests verify that (most) things work as intended, but also act as documentation for the specification, and give yourself a feel for the usage of a library.

On the other hand I think that writing tests feels often cumbersome and time consuming. The availability of good testing frameworks play an essential role in writing useful/comprehensible tests and shape the community’s expectation on the quality of tests (e.g., if I think about high-quality Java libraries, I immediately expect to find JUnit, Hamcrest, or Mockito tests).

For simple tests the facilities provided by Rust’s standard libray are usually enough, but once things get more complex they often tend to get convoluted, and patterns repeat themselves. The announced libraries intend to close that gap. They are not perfect (nor probably bug free), but may serve as a basis for improvement.


#2

This is the roadmap for the next few versions


galvanic-assert (v0.9.0)

  • structural matchers for tuples
  • improve assert messages

galvanic-mock

  • improved mocking of generic methods <mock as MyTrait>::func::<i32,f64> ...
  • mocking of static methods
  • expect_interactions_in_order! a variant of expect_interactions! which imposes an order
  • refactoring and API documentation: the code has undergone 3 complete design overhauls of the code generation to find a design which supports all of: behaviours, methods with references, generic traits, static methods, and generic methods—so it’s quite fractured an carries some old design choices
  • Rust stable support (currently proc_macro_attribute is required)—if someone has an idea how to do this without syntex (discontinued) hints are appreciated
  • spying on objects implementing traits
  • mocking/spying on structs

galvanic-test

  • improving galvanic-mock currently has priority

#3

galvanic-mock (v0.1.1) now optionally supports matchers from galvanic-assert.

In many scenarios this makes writing behaviours shorter and more descriptive. Compare

<mock as MyTrait>::foo(eq(15), any_value()) then_return 1 always;

with

<mock as MyTrait>::foo(|&x| x == 15, |&y| true) then_return 1 always;

The galvanic_assert_integration feature must be enabled for this to work (have a look at the documentation).


#4

galvanic-test (v0.1.0) has been released.

We’ve now got a fixture-based testing framework (similar to pytest) with dependency injection. Check out the examples and the documenation on github!

Feedback on the crates is appreciated!