Cargo with local crates.io mirror

help
1

We try to achieve CI builds without internet access for our Rust projects. crates.io was mirrored by artifactory and rustup by panamax.
The first job in our CI pipeline installs Rust and after that, we try to install some cargo tools with cargo install. This fails, as cargo tries to reach out for crates.io:

cargo install --locked --index=sparse+https://cargo.packages.mycompany.net/artifactory/api/cargo/crates-io/index/ cargo2junit
    Updating `sparse+https://cargo.packages.mycompany.net/artifactory/api/cargo/crates-io/index/` index
 Downloading crates ...
  Downloaded cargo2junit v0.1.14 (registry `sparse+https://cargo.packages.mycompany.net/artifactory/api/cargo/crates-io/index/`)
  Installing cargo2junit v0.1.14 (registry `sparse+https://cargo.packages.mycompany.net/artifactory/api/cargo/crates-io/index/`)
    Updating crates.io index
warning: spurious network error (1 tries remaining): [28] Timeout was reached (Connection timed out after 60004 milliseconds)
error: failed to get `junit-report` as a dependency of package `cargo2junit v0.1.14 (registry `sparse+https://cargo.packages.mycompany.net/artifactory/api/cargo/crates-io/index/`)`
Caused by:
  download of config.json failed
Caused by:
  failed to download from `https://index.crates.io/config.json`
Caused by:
  [28]

As can be seen from the trace, I set up a ~/.cargo/config.toml with the local crates.io mirror. I expected cargo to get the index from this mirror, not from crates.io. Is this a bug?

1 Like
2

Can you post the registry replacement part of your ~/.cargo/config.toml?

3 
[registry]
default = "artifactory-cargo"
 
[registries.artifactory-cargo]
index = "sparse+https://cargo.packages.mycompany.net/artifactory/api/cargo/crates-io/index/"
 
[source.artifactory-remote-cargo]
registry = "sparse+https://cargo.packages.mycompany.net/artifactory/api/cargo/crates-io/index/"
 
[source.crates-io]
replace-with = "artifactory-remote-cargo"
 
[http]
check-revoke = false
debug = true
4

And our local mirror responds to index requests. So, no excuses for cargo :wink: :

curl -X GET https://cargo.packages.mycompany.net/artifactory/api/cargo/crates-io/index/config.json
{"dl": "https://cargo.packages.mycompany.net/artifactory/api/cargo/crates-io/v1/crates", "api": "https://cargo.packages.mycompany.net/artifactory/api/cargo/crates-io"}
5

Why are you passing --index to cargo install by the way?

6

Sorry, that's a residue of one of my attempts.

7

Keep in mind that the "alternative registries" feature can't replace crates-io, so your artifactory-cargo config won't work (an alternative registry is uniquely identified by its URL, which gets baked into published packages' metadata, and a URL change makes packages different and incompatible).

Only "source replacement" config is able to replace crates-io, and your artifactory-remote-cargo should do that (source replacement keeps using the old URL for the purpose of identifying packages, but makes network requests elsewhere). The crates-io URL must remain untouched in lockfiles and other package IDs, even if you're not fetching from there.

These are two very different features, which unfortunately use confusingly similar config syntax.

You've defined two different sources with the same URL. At some point Cargo translates registry to a URL and maps it back, and maybe the replacement gets lost then.

Remove the alternative registry config or change its URL to something not shared with your crates-io mirror.

8

Thanks for the replies. I have to admit, that in my case, it simply didn't copy the cargo config to the correct location with the correct file name in our Windows build container (powershell syntax still feels weird). But I will check @kornel 's comments, thanks a lot for that.

9

Sidenote: For a reason I don't understand, cargo vendor won't use the cargo config by default. There's an extra option (--respect-source-config) for that.

10

That has been the case since

which fixed

Basically if the [source] table in .cargo/config.toml was respected, you did have to remove .cargo/config.toml before running cargo vendor again to actually fetch new crates from crates.io rather than reusing the already vendored sources (and thus likely failing the vendor operation)

2 Likes