Using relative Git paths in Cargo

Hi,

we use the integrated Gitlab CI which requires relative Git Paths.
If I understand correctly, this was a long time not possible with Cargo but is now available with Add support for relative git submodule paths by x-hgg-x · Pull Request #11106 · rust-lang/cargo · GitHub

I tried it with Rust Nightly but was not able to find the correct path syntax.

Default host: x86_64-unknown-linux-gnu
rustup home:  /home/sven/.rustup

installed toolchains
--------------------

stable-x86_64-unknown-linux-gnu (default)
nightly-x86_64-unknown-linux-gnu

active toolchain
----------------

nightly-x86_64-unknown-linux-gnu (directory override for '<Project-Directory>')
rustc 1.66.0-nightly (81f391930 2022-10-09)

Cargo reports:

cargo 1.66.0-nightly (3cdf1ab25 2022-10-07)

The repository path is parallel to the current used repo, so I tried the following paths:

> cargo add drivers --git=../drivers
error: relative URL without a base

> cargo add drivers --git=git:../drivers
error: invalid url `git:../drivers`: cannot-be-a-base-URLs are not supported

> cargo add drivers --git=file:../drivers
   Updating git repository `file:///drivers`
error: failed to load source for dependency `drivers`

Caused by:
  Unable to update file:///drivers

Caused by:
  failed to fetch into: /home/sven/.cargo/git/db/drivers-b506f56b71427f24

Caused by:
  process didn't exit successfully: `git fetch --force --update-head-ok 'file:///drivers' '+HEAD:refs/remotes/origin/HEAD'` (exit status: 128)
  --- stderr
  fatal: '/drivers' does not appear to be a git repository
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.

Adding a submodule without Cargo by using git submodule add ../drivers works normally.

Did I misunderstand the fix in Add support for relative git submodule paths by x-hgg-x · Pull Request #11106 · rust-lang/cargo · GitHub and it is still not possible to use relative Git-Paths with Cargo?

Thanks & Best regards
Sven

Maybe cargo add's checker wasn't update correctly. Did you try directly adding the dependency in the Cargo.toml?

Thanks for your reply. I'll tried it, tested it with cargo update and here are the results:

Cargo.toml: drivers = { git = "../drivers", version = "*" }
Result:
    error: failed to parse manifest at `<Project Dir>/Cargo.toml`

    Caused by:
      invalid url `../drivers`: relative URL without a base

 

Cargo.toml: drivers = { git = "git:../drivers", version = "*" }
    error: failed to parse manifest at `<Project Dir>/Cargo.toml`

    Caused by:
      invalid url `git:../drivers`: cannot-be-a-base-URLs are not supported

 

Cargo.toml: drivers = { git = "file:../drivers", version = "*" }
    error: failed to get `drivers` as a dependency of package `<Package> v0.1.0 (<Project Dir>)`

    Caused by:
      failed to load source for dependency `drivers`

    Caused by:
      Unable to update file:///drivers

    Caused by:
      failed to fetch into: /home/sven/.cargo/git/db/drivers-b506f56b71427f24

    Caused by:
      process didn't exit successfully: `git fetch --force --update-head-ok 'file:///drivers' '+HEAD:refs/remotes/origin/HEAD'` (exit status: 128)
      --- stderr
      fatal: '/drivers' does not appear to be a git repository
      fatal: Could not read from remote repository.

      Please make sure you have the correct access rights
      and the repository exists.

Also cargo run reports these errors. Looks like they are using the same parser.

Add support for relative git submodule paths by x-hgg-x · Pull Request #11106 · rust-lang/cargo · GitHub is for git submodules used by git dependencies. Not for git dependencies themself. So if the dependency you included using git = "..." in Cargo.toml uses a relative path in .gitmodules this now works. However the path in git = "..." still needs to be absolute. Cargo doesn't know what it would be relative too. Cargo is not concerned with the cvs you store the project in.

1 Like

Thanks @bjorn3 for the clarification. I tried to workaround this by adding the submodules by hand and using the cargo add --path to integrate them. This worked until I had the following structure:

  • main project
    • submodule: common structures
    • submodule: drivers
      • submodule: common structures
    • submodule: data handling
      • submodule: common structures

Common structures need to be in multiple components as data handling doesn't belong to drivers and can be used standalone in other projects and vice versa.

When I try to cargo build this it says:

error: package collision in the lockfile:
packages common_data v0.1.0 (<path>/main/drivers/common_data) and common_data v0.1.0
(<path>/main/common_data) are different, but only one can be written to lockfile unambiguously

Cargo GitHub says this is a feature not a bug. So I guess I must imagine another workaround if I do not want to add SSH keys to my Gitlab CI docker.

Is there some reason you can't use GIT_SUBMODULE_STRATEGY=recursive to let gitlab do the checkout of the submodules for you, and cargo just uses paths? I've not had reason to do this myself, but the docs imply that's basically the intended usage, and I think it's what another project uses at work.

Sorry for the misunderstanding. That's exactly what I did and tried to explain (in the post before your answer, not as the original question). But this approach fails when the main repo and one of the submodules both use the same dependency as Cargo doesn't allow using the same dependency in both projects with different absolute paths.

Ah, that makes more sense to me now! The dreaded diamond strikes again.

I think the simplest option would be for the main cargo.toml to patch the submodules to use the main common crate (I think that should work?)

1 Like

Thanks for your suggestion. Can you please give me a hint how to achieve this? As I'm using Gitlab and the submodules must able to build standalone for tests, I guess I'm forced to use relative Git paths and enter the cloned paths via path in Cargo.toml. I tried to override the paths in the main Cargo.toml without success:

[patch.'file://common_data']
common_data = { path = "../common_data" }

I'm even unsure how to use the same path override for the main project and the included sub-project as the path can only either be relative to the main project or to the sub-path from the sub-project. If I understand correctly. the patch will always be applied to the main project and the sub-projects.

I think I found a solution I can live with:

main/Cargo.toml

[dependencies]
common_data = "*"
...
[patch.crates-io]
common_data = { path = "sub/common" }

Same goes for the submodules:

main/sub/Cargo.toml

[dependencies]
common_data = "*"
...
[patch.crates-io]
common_data = { path = "ext/common" }