Using sparse protocol for crates.io in alternate repository results in duplicate dependencies

Hi — having some issues with an alternative registry and duplicated dependencies when using the sparse protocol for crates.io.

A simplified view of the situation is that I'm installing crates from an alternate (sparse-protocol-only) registry, which have dependencies on crates on crates.io. Additionally, in my project, I'm using some of these same dependencies. Ideally, the sparse protocol would be used everywhere.

On my registry, I'm populating the index files, specifying to use the sparse protocol for the crates.io dependencies (and null for the dependencies on the alternate registry). e.g., if I curl one of the registry index pages, I see:

{
  "name": "local-package",
  "vers": "0.3.1",
  "deps": [
    {
      "name": "prost-types",
      "req": "0.12.4",
      "features": [],
      "optional": false,
      "default_features": false,
      "target": null,
      "kind": "normal",
      "registry": "sparse+https://index.crates.io/"
    },
    {
      "name": "local-dependency",
      "req": "0.3.1",
      "features": [],
      "optional": false,
      "default_features": false,
      "target": null,
      "kind": "normal",
      "registry": null
    },
  ],
  "cksum": "a2ec051fc23b9a2bca4807ce62f2b866eb549e141bb55ce71663fc1c7098a0f5",
  "features": {},
  "yanked": false,
  "rust_version": "1.62"
}

In my main binary, I want to also use the prost-types crate, so I do a cargo add prost-types@0.12.4.

If I then run cargo tree --duplicates, I can see that I'm somehow getting duplicate prost-types dependencies, even though they're seemingly the same version:

$ cargo tree --duplicates
# other duplicates ...
prost-types v0.12.4 (*)
prost-types v0.12.4 (*)
# other duplicates ...

And if I try to use the types, I'm greeted with a confusing error message:

$ cargo run
stderr:    Compiling proc-macro2 v1.0.81
  Compiling unicode-ident v1.0.12
  Compiling anyhow v1.0.82
  Compiling proc-macro2 v1.0.81
  Compiling unicode-ident v1.0.12
  Compiling either v1.11.0
  Compiling anyhow v1.0.82
  Compiling either v1.11.0
  Compiling bytes v1.6.0
  Compiling bytes v1.6.0
  Compiling itertools v0.12.1
  Compiling itertools v0.12.1
  Compiling quote v1.0.36
  Compiling quote v1.0.36
  Compiling syn v2.0.60
  Compiling syn v2.0.60
  Compiling prost-derive v0.12.4
  Compiling prost-derive v0.12.4
  Compiling prost v0.12.4
  Compiling prost v0.12.4
  Compiling prost-types v0.12.4
  Compiling prost-types v0.12.4
  Compiling local-dependency (registry `private`)
  Compiling local-package (registry `private`)
  Compiling test v0.0.1 (/private/var/folders/q9/562j68913xs71bhqdrqh8drh0000gn/T/TestClustercargo_registry3611135760/003)
error[E0308]: mismatched types
   --> src/main.rs:15:25
    |
15   |         last_seen: Some(Timestamp::from(SystemTime::now())),
    |                    ---- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `prost_types::protobuf::Timestamp`, found `Timestamp`
    |                    |
    |                    arguments to this enum variant are incorrect
    |
    = note: `Timestamp` and `prost_types::protobuf::Timestamp` have similar names, but are actually distinct types
note: `Timestamp` is defined in crate `prost_types`
   --> /Users/stefanvanburen/.cargo/registry/src/github.com-1ecc6299db9ec823/prost-types-0.12.4/src/protobuf.rs:2297:1

I've seen issues before relating this situation to being due to multiple versions of the same crate being included in the build.


If I change the registry in the index files to be the old, git-based registry:

-     "registry": "sparse+https://index.crates.io/"
+     "registry": "https://github.com/rust-lang/crates.io-index"

And run cargo with CARGO_REGISTRIES_CRATES_IO_PROTOCOL=git, the duplicate dependencies go away in cargo tree, and cargo run works fine.

Has anyone seen anything like this before? Is there something I'm holding the wrong way, or other things I need to set in order to get this to work? Happy to provide any more context requested — would love to avoid using the git crates.io protocol.

The registry URL in dependencies identifies the registry, but doesn't specify the protocol to use. You're not supposed to use the sparse+ prefix for dependences, even when using the sparse registry protocol.

The index meta file should always use "registry":"https://github.com/rust-lang/crates.io-index" to identify crates.io, for any protocol.

Ah, yep. No need for the environment variable. For context, I had missed this context from https://doc.rust-lang.org/cargo/reference/registry-index.html:

registrycargo metadata uses a value of null to indicate that the dependency comes from crates.io. The index uses a value of null to indicate that the dependency comes from the same registry as the index. When creating an index entry, a registry other than crates.io should translate a value of null to be https://github.com/rust-lang/crates.io-index and translate a URL that matches the current index to be null.

Thank you!

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.