Cargo.lock, libraries and CI


#1

In one of my projects, I test with Rust releases back to 1.4 on CI. Recently the build broke for Rust 1.4 through 1.6 without any changes to my code. My library transitively (but not directly) depends on the gcc crate, of which the current version (0.3.37) fails to compile with Rust 1.4 through 1.6. Because the project is a library, I did not put cargo.lock under source control, so the most recent version is used on CI.

The gcc crate works fine with a recent version of Rust, and I don’t really want to lock it to an old version. On the other hand, an older version of the crate worked fine with Rust 1.4 through 1.6, so I see no reason to drop support for those; I’m not using any of the newer features myself.

What would be the best course of action here? I suppose RFC #1709 would help in the future. Is there anything that can be done currently?


#2

I think some might say (and I think I agree with them) that the gcc crate should have done a minor version bump (read: a semver bump) when it increased its minimum Rust version for precisely this reason. That is, requiring a new version of Rust should be considered a breaking change.


#3

I don’t agree that a Rust version upgrade is a breaking change. In that world it’s impossible for a library to settle into a stable 1.* version series, but would have to release version 2.0, 3.0 etc just to use newer Rust features in its implementation, even without any API breaks.


#4

This point has been brought up before, and the typical response is that you can selectively pick new Rust features to use with cfg! flags. There are many downsides to this though:

  • It makes the code more complex (potentially significantly so).
  • This doesn’t work for everything. For example, new Rust language features or if you want to export something in the public API which depends on a new addition to std. e.g., How could you selectively use specialization? You’d have to put it into a separate crate and put it behind a --feature flag, which sucks. (And will that even work in every case?)

Right. (Modulo the point above.) On the other hand, this is a trade off because a lot of people aren’t going to call your library stable if it has a semver-compatible version bump and no longer compiles with their rustc that they got from their distro. (We aren’t quite there yet, but I think everyone hopes we will be.)

A much longer discussion has been had, and we couldn’t quite reach a consensus: https://github.com/rust-lang/rfcs/pull/1619

(The way I try to head this issue off is by putting the minimum Rust version in CI. If CI breaks, then I either back out the change or do a semver bump. But I’m not perfect about this. It’s pretty tricky.)


#5

It was not my intention to discuss whether or not gcc should have done a major version bump in this topic. Given the current situation in which the major version was not bumped, is there anything that can be done?

Yes, this is exactly what motivated me to keep testing older Rust versions on CI. However, not putting Cargo.lock under source control then implies that the build can start to fail without changes to the code, as I found out the hard way.


#6

I see, sorry. The issues are closely related. This is the only thing I could find: http://doc.crates.io/specifying-dependencies.html#overriding-dependencies

@DanielKeep I think this situation has happened to you as well. Did you work around it?

I didn’t mean it as a solution to your problem, but as a solution for, say, authors of crates that don’t want to permit an increase in the minimum Rust version without some explicit action. (For example, the gcc crate only tests on Rust stable/beta/nightly, so there’s no reliable way to know when an older version breaks.)


#7

It’s worth noting that regardless of putting Cargo.lock under source control, Cargo will ignore the lockfile when doing resolution, and this is very much intended behavior. So doing that alone won’t solve the problem.

(I agree with everyone above that historically, I’ve put the “oldest minimum version” in my CI config, generally. I don’t always do it now, but if and when LTS becomes a thing, will for sure.)


#8

The only way I’ve found to “solve” this is to forcibly limit the maximum compatible version of the problematic dependency in your manifest. And yes, this does mean that people on newer compilers are also stuck with old, possibly buggy versions of dependencies. I know of no way to fix this.

Here’s part of cargo-script's Cargo.toml:

[dev-dependencies]
# ...

# gcc 0.3.29 is incompatible with Rust 1.4.0.
gcc = "0.3,<0.3.29"

Sadly, this doesn’t always work all that well. If a downstream crate A uses a sufficiently loose version spec for an even further downstream crate B, you may be forced to just drop the dependency on A entirely, or retroactively give up on backward compatibility.

To be honest, I’m about at the point of just giving up on the idea of backwards-compatible Rust code entirely. Pretty much everything about the Rust tooling and ecosystem makes it a Sisyphean task.


#9

Thanks for the input everybody! It turns out that I only depended on gcc as a transitive dependency of one of my dev-dependencies, so I added a gcc = "= 0.3.5" to my dev-dependencies too and that resolved the issue for me. Because it is only a dev-dependency I don’t mind locking the version like this. It is pure luck that this works though. For a regular dependency there is indeed the issue of forcing upper bounds onto dependent crates.


#10

This happened again, but this time due to time-0.1.36 introducing a breaking change on Rust 1.4 through 1.7. Only this time the issue is more subtle, because time is not a direct dependency of my crate. A dependency has a dependency on time ^0.1.

At this point I am strongly considering putting Cargo.lock under source control for my library. Apart from CI not testing with newer dependency versions (which is actually exactly what I want), are there any good reasons for not putting Cargo.lock under source control? And will it actually prevent breaking changes in the future? How does resolution come into play?


#11

Cargo’s resolution will ignore the lock regardless of you having it in your source control or not.

The reason is mostly to let people know that that’s a true thing; it will work the same regardless of what you do.


#12

I am confused. What is the lock file used for then? What triggers resolution?


#13

The lockfile is used when directly compiling that crate. It is ignored if the crate is a dependency.


#14

Ah, right. This is only about how my library is compiled on CI, such that whitespace changes or rebuilding at a later time will not cause a different result. Cargo.lock should be irrelevant to consumers of the library indeed.