How to stop Rust code from rotting all the time?

[parasyte]

@ZiCog

What else do I have to lock down? The compiler? My OS?....

In short, yes ^[1]. Everything always changes all the time.

As much as everyone wishes breakage didn't happen, and as hard as they try, it still does. You'll no doubt "fondly" remember all that happened with the transition from 32-bit x86 to 64-bit ^[2], or 16-bit to 32-bit for that matter. Over in Apple world, we've gone through several architecture changes. 68K to PowerPC to x86 to Arm. And that's just the Mac family.

There have been some incremental technologies that smooth the transition (software emulation, multi-architecture binaries). But pragmatically, breakage is inevitable.

@cod10129

They specified a dependency on the rand crate as >=0.8.

Does anyone know if any thought has been put into gradually phasing out this cargo misfeature? Hopefully one day it will reject the crate as early as parsing its Config.toml. Ranges can be reasonable, but unbounded > is just a completely illegitimate construct. No one can predict the future.

It's not a new issue: Unexpected dependency resolution when using ">" · Issue #3292 · rust-lang/cargo but I haven't seen any indication that cargo wants to fix it. Any RFCs or open issues tracking this? Should a discussion for deprecating unbounded > be opened on IRLO or Zulip?

---

1. "Old man yells at cloud." ↩︎

2. Unix-likes and Microsoft disagree on the size of C's long type. Oh well. ↩︎

[mroth]

@ekuber

I am to blame for the diagnostic telling you to do cargo update instead of cargo update time. That was an oversight of mine (and the reviewers) in the scramble to get something in place quickly.

I’d like to take this opportunity to thank you, your colleagues, and everyone contributing to the Rust compiler, standard library, and the Rust project as a whole.

A huge thank you for all the excitement and great work!

@ekuber

Seeing this thread made the hairs in the back of my head stand up. We have a "time bomb" assertion in the codebase to remind us to remove the special note from 1.89

In my experience, in the long run, breaking things and getting rid of legacy baggage is often cheaper than accumulating an ever-growing burden of inherited complexity.

[cod10129]

@steffahn

This problem could only be fixed either by nuid maintainers releasing point releases for 0.3.* (and probably also 0.4.*) that fix the incorrect rand dependency specifications

There’s an open issue against the nuid repo describing how 0.3.x versions are currently broken that was filed the same day rand 0.9 was released (Jan 27). But the maintainer hasn’t responded (and the repo’s last commit was in August 2023).

@steffahn

or by nats updating their dependency to nuid version 0.5

nats did actually do this, and just two days after rand 0.9: Bump legacy nats client nuid dependency & remove old client from CI runs by Jarema · Pull Request #1367 · nats-io/nats.rs · GitHub. The problem should be fixed on main, but the changes aren’t released to crates.io. (And even if the fix was published, it would be for 0.25.x versions of nats and ZiCog is using 0.24.)

@steffahn

Maybe something like cargo update time@0.3 works, too, but ensures even more minimal changes? [I’m not 100% sure if cutting off the version here is supported or if you need the exact time@0.3.…something… here]

Based on the syntax for “package id specifications”, it should be (although I didn’t test it). There’s even an explicit example^[1] that says regex@1.4 means the regex crate at version 1.4.*.

That’s probably a good improvement for that message, for it to recommend cargo update time@0.3 instead of updating all dependencies.

---

1. in the table that won’t cleanly format into a quote ↩︎

[2e71828]

@parasyte

Does anyone know if any thought has been put into gradually phasing out this cargo misfeature? Hopefully one day it will reject the crate as early as parsing its Config.toml. Ranges can be reasonable, but unbounded > is just a completely illegitimate construct. No one can predict the future.

An interim solution here would be to ban crates with this style of dependency specification from being published to crates.io. That protects the commons from this issue while letting private enterprises shoot themselves in the foot if they really want to.

[drmason13]

Is this case fixable by cargo patch?

To do something like replace the rand>=0.8 dependency specification with rand=0.8 instead in the nuid crate.

Basically to fix locally what would ideally have been patched and released upstream (but in this case can't been).

Suggesting such patches as part of error diagnostics could be helpful.

[kpreid]

@2e71828

An interim solution here would be to ban crates with this style of dependency specification from being published to crates.io. That protects the commons from this issue while letting private enterprises shoot themselves in the foot if they really want to.

I thought this was a great idea, so I went to propose it, and then found the original RFC, 1241, for prohibiting wildcard dependencies. It turns out that that requiring upper bounds was originally part of the RFC, but there were arguments against that, so it was narrowed to only prohibiting wildcards. Might be interesting reading.

[2e71828]

@kpreid

It turns out that that requiring upper bounds was originally part of the RFC, but there were arguments against that

If I'm understanding that thread correctly, the argument in favor of allowing half-unbounded version ranges is that a semver-major bump is too coarse a signal for many real-world use cases:

Often, the breaking change only affects a subset of the API. In such cases, there's no problem building against multiple different major versions, as long as you don't use the changed subset. Some crates even provide side channel (non-semver) guarantees of this by, for example, having explicit deprecation policies.

If you rely on a library that's managed this way, the work required to officially approve each new version is almost pure busywork. Furthermore, it can interfere with the adoption of the dependent's new major version as downstream users have to wait until their intermediate dependencies have vetted the new major version.

Reducing the scope of the RFC was due to a lack of consensus over the upper-bound issue, rather than a determination that it was a bad idea-- That portion of the debate was still quite active when the change was made, and the libs team were favorably disposed towards the initial proposal.

---

So, it does seem reasonable to re-open the discussion given that no real decision was reached on this issue. Based on the prior discussion, there are a few other alternatives to consider:

• Add a convenient way for version upper bounds to be retroactively added, once an actual conflict has been identified
• Let crates with stronger-than-semver update policies opt-in to being used with the looser dependency specifications

[simonbuchan]

Honestly I think it's perfectly reasonable to have a deprecation policy like "breaking API changes will be made after at least a full major version with the to be removed functionality being deprecated", which means you should have a bounded range like ">= 2.8, < 4" as you know any deprecations added have to remain for all of 3.x; and doing so lets the ecosystem migrate over a period. But it doesn't make sense to have a completely unbounded range, that's effectively a guarantee that you're never making a breaking change.

Strawman: crates can be published with an eg. "stability_range = 2.0" that means the default range added by cargo add should be two major versions like above. If you make breaking changes on minor versions, it could be 0.1, if you use CalVer and have breaking changes after 3 months, 0.4, etc...

[2e71828]

In your example, when the dependency publishes version 3.0, all of the dependents that are using <4 upper bounds can safely update that to <5 if they don't gain any deprecation warnings on the new version.

In an ideal world, there would be a CI system available to the dependency's author which would automate this check. Any dependent crate that passes would then be offered an automatic point release that updates the version bound without changing the actual code (which would obviously need to be approved by that crate's owner).

[parasyte]

I've been thinking about this more recently, especially after reading that discussion ^[1]. Definitely feeling like my original suggestion was overzealous. Tightening the constraints on the publish side is a reasonable tradeoff. I think it's a tradeoff that needs to be made, though.

Reducing churn seems nice at first, but I find it hard to believe that developer convenience should be given priority over build failures. Stronger dependency versioning constraints do hold back upgrades, and requires more attention by maintainers to, well, maintain their packages. Build failures are an entirely different beast. Those problems deny users the ability to use the software at all. I feel more empathy for the latter group, those are the users that need protection.

The subset of users affected by build failures with unbounded dependencies will only grow. It can never decrease. To paraphrase the Fight Club Narrator, "over a long enough timeline, the survivability for unbounded dependency versions drops to zero."

---

1. FWIW, the project used as a motivational example for allowing unbounded version ranges was archived a few years ago. I don't know the details of its archival. Seems like the project just didn't work out, whatever the reasons. ↩︎

[ekuber]

At the very least, we should be linting about such unconstrained entries in Cargo.toml, even we don't start rejecting their publication in crates.io, but this is a retroactive issue that doesn't help existing crates. I'd be tempted at changing the resolver behavior to ignore unconstrained rules, unless an opt-in flag is passed.

[michaelCTS]

You could use devbox or devenv to lock you rust dependency. devbox is probably easier (see doc).

The easiest way to manage Rust with Devbox is to install rustup, and then configure the channel you wish to install via Devbox's init_hook. You can also use the init_hook to configure rustup to install the Rust toolchain locally.

There's also direnv, which you can use to set rustup environment variables like RUSTUP_HOME.

.envrc

export RUSTUP_HOME=$(pwd)/.rustup
export RUSTUP_TOOLCHAIN=1.79.0
rustup install $RUSTUP_TOOLCHAIN

Or if you rather do it manually, add a note in the project of the supported rust version and the commands above. Compilation should the happen with that rust version. It would be great if rustup had a configuration file one could use or used a field in Cargo.toml and a rustup.lock or something, but that's just me dreaming.

[simonbuchan]

I mentioned the toolchain lock features earlier here: How to stop Rust code from rotting all the time? - #19 by simonbuchan

[CAD97]

@2e71828

In your example, when the dependency publishes version 3.0, all of the dependents that are using <4 upper bounds can safely update that to <5 if they don't gain any deprecation warnings

At least with the current resolver that always chooses the maximum allowed version and doesn't try to minimize or otherwise unify major versions between separate dependency specifications, there's almost no benefit to maintaining a range open over multiple major versions. So, actually, the upgrade path would be from >=2, <4 to >=3, <4 as soon as a 3.* version is out, which would then be changed to >=3, <5 once any deprecation warnings are dealt with.

Note also that things deprecated in 3.0.0 but not the highest 2.* version can be removed from 4.* under that policy, so the intermediate update is required to safely upgrade the top bound.

[simonbuchan]

In sure there's lots of detail, but yeah, this is pretty much what I was expecting the logic to be. You might install foo at version 2.7, lock contains (equivalent to) >= 2.7, < 4, come back a year later and run cargo update, you get the latest 3 version 3.12, a bunch of deprecation messages and a notice that 4.2 is available, fix the deprecations and cargo update foo --breaking (or whatever)

Needs a bit more cooking about how multiple breaking changes would be handled, maybe?