Just now I was trying to test num, in which we still keep Rust 1.0 compatibility but have a feature for serde, so I wanted to see if the new serde 0.9.0 would work. But it seems the registry format is somehow incompatible with such old cargo.
The break appears to be around Rust 1.7 / Cargo 0.8, whereas Rust 1.8 / Cargo 0.9 looks fine.
$ cargo +1.7.0 generate-lockfile --verbose
Updating registry `https://github.com/rust-lang/crates.io-index`
failed to parse registry's information for: serde
Caused by:
the given version requirement is invalid
$ cargo +1.8.0 generate-lockfile --verbose
Updating registry `https://github.com/rust-lang/crates.io-index`
It doesn't help to generate a lockfile with a later version first, as the older cargo build still fails with the same error. Also note, I haven't even updated the num dependency yet, so we're asking for ">= 0.7.0, < 0.9.0", not the new version.
I see serde/Cargo.toml has [dev-dependencies] serde_derive = "0.9.0-rc4" -- could that -rc4 be the trouble?
It's totally fine that serde itself doesn't support old toolchains, but I'd hope as an optional dependency it shouldn't cause problems...
The rc4 dev-dependency was a bit weird - we have a cyclic dev-dependency between serde and serde_derive because I want to use #[derive] in Serde's rustdoc but serde_derive requires serde to run tests. Either serde or serde_derive needed to be published first, and that one could not depend on 0.9.0 of the other.
In any case I released serde 0.9.1 which is now able to dev-depend on serde_derive = "0.9" so all is good. Can you see whether it works now on your end?
It could also be a consequence of this change which would be unfortunate.
Unfortunately not. I don't know why it's parsing these versions at all given they are out of my version range. Perhaps if you yank 0.9.0 it will be ignored? I'd rather not push you into that without knowing if it will help, but it should be harmless with 0.9.1 out there.
Yeah, the addition of categories was my first suspect, since at first it just says "An unknown error occurred", but the verbose error explicitly mentions the version requirement.
Yes, that looks like the problem, and lines up to about the right time frame that the fix would have landed in Rust 1.8, cargo 0.9.0-nightly (8fc3fd8 2016-02-29).
So, does this mean old cargo is now irrevocably broken if it sees a reference to serde at all? Or do you think yanking serde-0.9.0 might sufficiently hide it?
I tried it with a test crate, but yanking appears to have no effect on the problem. Even with the latest version void of dependencies and all others yanked, any reference to that crate using an old cargo fails to parse it.
Two of my colleagues ran into this a few days back. From their vantage point, this is a Rust issue. It is not clear to them that cargo is really separate from the BC guarantees of the Rust language.
The worst part about this is that upgrading cargo is not an obvious solution. In this case, they were both using versions of rust and cargo provided by Ubuntu.
Could crates.io serve different data to different cargo versions to work around this? Perhaps pretending that the problematic releases do not exist at all?
I'm not sure of the architecture, but I got that list from the git registry that cargo clones, so it may not be something that crates.io could influence.
I find it a critical miss that we broke software that was shipped with an LTS of an operating system. cargos compatibility guarantees notwithstanding, it erodes trust in our efforts to be a good OS citizen.
To be fair, it's in the community-maintained Ubuntu universe, not the main repository to which LTS applies. But that still makes it fairly likely that people will use those packages. I don't know their update policy for universe, but maybe rustc+cargo could be rebased to newer versions?
Whether or not we can solve this current issue, what should we do going forward?
Maybe crates.io could do a server-side cargo generate-lockfile on new uploads using a few cargo versions, including the oldest we want to support. Run this as a sort of mini CI with any other sanity checks we can think of.
A full build/test is probably too much to do server-side, especially since many crates won't support older rust, but at least old cargo should be able to deal with their Cargo.toml.
It's unclear to me who is maintaining the package in person. Possibly one of MOTU: MOTU - Ubuntu Wiki
As far as I can see as a non-Ubuntu-user, Universe does have strict release rules, they are just enforced by someone else and the stable update process is documented here: StableReleaseUpdates - Ubuntu Wiki
Also, there are no bugs filed against either cargo or rustc about this, except one to properly backport a package to build firefox:
Rubygems just serves multiple repository formats for just that reason. Also, care could be taken to keep the format future-compatible.
We should definitely keep up to speed which versions of Rust are deployed and easily accessible on which platforms. The last thing I want to have is the situation that Ruby had for a long time where the first support question was "do you use the system packages? If yes, uninstall, ./configure && make && make install.
Surely the burden to be "a good OS citizen" should be on the LTS distro in this case. When a distro promises support for a longer period than the upstream, the distro takes on the responsibility to deal with the difference in support period. (Distros that don't want to deal with the difference in support period should be rolling.)
That is, it seems to me that it's reasonable to expect Ubuntu to ship an update to its cargo package.
Also, it's worth considering what use cases distro-shipped rustc/cargo address. There are three obvious use cases:
building the Rust-based software in the distro's archive
sparing people who want to develop Rust code from running/trusting rustup (to have just one package manager to worry about) and
sparing people who want install distro-unpackaged Rust software from source from having to run/trust rustup.
Use case #1 should be offline, so it shouldn't matter if crates.io has moved on.
For use cases #2 and #3, it's not clear that an LTS distro sticking to a notably old version of Rust has a net-positive impact on the ecosystem in the current phase of Rust's development when major advances like custom derive are happening.
It's a two-way street - the distro packager has a responsibility to keep up with the ecosystem, and upstream has a responsibility to consider distro needs, if that's a use-case they care about.
(And again, being in universe means this isn't actually an LTS package, as far as Canonical is concerned.)
And what is the support period for crates.io? Certainly it's not latest-cargo-only, because that's incompatible with most non-rolling distros, and the developers do care about distro needs. I don't think it's explicitly stated, but I think the intent is not to break old cargo, and this case just slipped through the cracks. That's why I suggest crates.io should do some compatibility test when accepting uploads, so this doesn't happen again on accident.
They'll probably need to at this point, yes. I think @skade was referring to Ubuntu's bug tracker when noting the lack of cargo bugs. I'm not a Ubuntu user, but someone who is directly impacted should probably file that bug.
I've heard people mention the possibility of making an LTS-designated release of Rust (and by extension cargo)... maybe it's time for an RFC proposing that we do that in the near future? I'm not necessarily volunteering to start that, but I'm not NOT volunteering either. I think this issue means it's a good time to start the discussion, at least. Anyone in this thread want to start that RFC?