Table-driven (aka data-driven) testing


#1

The assert! macro exits immediately which means it cannot usefully be used in a loop to iterate over a table of input and expected values to test a function by example.

Python (at least in PyTest) has decorators for taking a set of functions and a table of input and expected values and generating individual test function for each case.

Go takes a different attitude and say instead of using an assertion that fails have a function call that adds a failure to the fail list. This means you can write standard loops since all cases will always be tested.

I haven’t seen anything like the Go approach in the Rust testing framework, have I just missed it?

I have seen BurntSuchi’s quickcheck (a port of Haskell QuickCheck) which is great for property-based testing, but doesn’t help with data-driven example-based testing.

Is anyone working on Rust support for data-driven (aka table-driven) example-based testing?


#2

I usually just write a macro that creates a test for each input I want to test, then let Rust’s unit testing do the rest. (e.g., https://github.com/BurntSushi/fst/blob/master/src/raw/tests.rs#L111-L158) I don’t think I’ve ever personally needed more than that, but it is true that a macro probably can’t easily satisfy all use cases. For example, if the set of inputs isn’t known a priori, then it might be tricky to use the macro approach. In that case, it seems easy enough to write a higher order function that takes a sequence of tests, runs them all and records all failures. I don’t see a need for a framework for that specific use case.


#3

Thanks for the pointer to that code, and for your reflections on what you have done to date. I’ll have a play and see if I can come up with something that works along the lines of pytest.mark.parametrize. Fall back will be to do something akin to Errorf in Go.