Test crate against multiple dependency versions

#1

Say I maintained a really, really important crate. (I don’t, but I am prone to paranoia.) So important that if any of my dependencies happened to break in a new patch version, it would be highly disruptive, or expensive, or life threatening, etc.

I already know that I can specify ranges of versions in my Cargo.toml, but is there a tool I can use to automatically compile and run my tests against every version in that range, to ensure that each version is compatible?

Bonus points if I can also automatically test it against multiple versions of Rust itself.

#2

The responsibility is on the dependency to test it across all versions.

#3

There’s a feature request for something related: https://github.com/rust-lang/cargo/issues/4100

You could maintain multiple copies of Cargo.lock and swap them before testing.

As for multiple Rust versions, rustup default <rust version> before a test does the job.

#4

While I agree in principle, the responsibility is always on me to make sure my library functions using the dependencies that I tell people are compatible.

If I put rayon = "1.*" in my Cargo.toml and rayon version 1.1 (accidentally or not) breaks my library, that’s on me.

#5

If you worry that rayon 1.1 could break you, then so could 1.0.2 or any other update. There’s a level of trust required that your dependencies will act responsibly, otherwise you should probably reconsider whether you want to have that dependency at all.

#6

Yes that’s true, any update could break you. That’s why I brought up version ranges - to only allow versions you have tested against.

But to determine that range, you have to test against all those versions. I was wondering whether there was a tool to automate that process.

#7

It depends on the crate. If they promise to follow semver, then you can blame the crate (though it will do nothing good :stuck_out_tongue: )

Another thing to consider is that you are considering only level of dependency (direct or primary dependencies). What about indirect dependencies? How do you ensure your dependencies work on each version of their dependencies on every version of rust?

#8

I hope I’m not being too slack in my own personal paranoia, but if you check in your Cargo.lock and have CI matrix that looks something like this:

Then (by removing the lock file for not “WITH_LOCK” builds) you at least able to test some known base-line version and the latest released version in the range specified by your project’s Cargo.toml formal dependencies. If it works at the beginning (last lock) and latest end of the range, there is a good chance™ it works for all versions in between, right?