I wanted to highlight the above suggestion on how we might write tests against traits rather than just specific impls. Throwing out old school OO, Rust throws its full weight behind interfaces. Given that the standard library has a suite of tests around each trait what a gift it would be if we were able to leverage those tests on our own custom impls.
I'm arguing now is far better to talk about /implement this than in several years time as by then the ecosystem will be in place and ideally if we can do this for std traits, then we'd get even more benefit for the next levels up offering tests for free on their traits.
Let's not reinvent the wheel when writing tests, let's re-use existing tests where we can so we have more time to add specific interesting test cases.
I think a really interesting case is if we add a std test case on an existing trait, and suddenly downstream crates test suites break, are we breaking backward compatibility? I'd argue we're just adding an additional lint that the impl is doing something non-standard, and as an implementer I'd be keen to find that out.
Is there prior art? I've not seen many attempt this which I find surprising.
This is a really interesting idea! I think we can already do this, although probably not in the way you originally intended.
It sounds like what you want to test is that each implementation of a certain trait will satisfy some contract/property for all inputs you throw at it. @BurntSushi's quickcheck crate does this really well!
You're probably going to find that this is hard to do because traits of themselves don't have any innate behaviour/contract (besides their type signature). It's up to us as humans to ensure that, for example, the ToString trait gives you a realistic string representation of some type (instead of say garbage or the works of Shakespeare). Likewise even something as integral to Rust as the Send type doesn't really have any meaning to the compiler or type system, it's how we use and the assumptions we make when building on top of it that gives Send meaning.
On a more practical note, you also run into problems when trying to generate the construction of the objects being used. Most "interesting" implementors of a trait will have non-trivial (e.g. not Default) constructors, meaning it's not really possible for the compiler to generate code to create the tests in the first place.
The little good bit to start; Since implementations are lazily created by the compiler it is theoretically possible it can have the full list of structures.
For a test to be useful it has to check the output. So for any return that is generic (including self); these checks won't generally exist or will only be partial.
It must be able to create the input. Most traits don't supply any way to create self.
I made a rod for my own back as there isn't a Set trait that BTreeSet and HashSet implement. I got hung up on what to return for intersection (Very keen to understand what type is possible to return).
Anyway the key takeaway is without modification one could use HashSet tests against the BTreeSet, and also write a new set impl and run the trait tests against it. I consed that not every test will fit in nicely - the constructor tests obviously aren't particularly portable, but there exists a subset of tests that can describe the behavior that we would expect from implementations of this interface.
Having looked at macro_rules! and proc macros it seems neither is sufficient to implement trait_tests, so I've had to use a compiler plugin for now. I think proc macros when extended will do just fine, but for now they only cover derive.
So here's a working version of trait_tests using a compiler plugin. Feedback welcome before I publish and be damned on crates.io.
Rust's support for static methods on traits really helps. The more trait tests I've written, the more convinced I am that this approach has some merit.