Best practice for partial vendoring and patching crates

help
1

Hi,

In our rust project, we came up with the need of modifying external crates.

Our project is a workspace with a library mylib and the following Cargo.toml:

[package]
name = "mylib"

[dev-dependencies]
rstest = 0.12.0

As we need to modfiy the source-code of rstest a bit, we use git subtree to clone
the source-code of rstest into external/rstest.

We do our modifications as expected, need to find a way to make mylib use our patched version of the rstest crate.

Solution 1: Set the path

We can modify our library Cargo.toml to point directly to the package we need by using the path:

[package]
name = "mylib"

[dev-dependencies]
rstest = { path = "external/rstest" }

And this exactly what we want but unfortunately there is one drawback.

The subtree in external/rstest is also a Cargo workspace. Therefore, we finally end up with a workspace in a workspace. This is currently not supported by Cargo.

Solution 2: Patch the crate

We modify our workspace by patching crates.io to use our version of rstest instead.

In our workspace Cargo.toml we write

[patch.crates-io]
rstest = { path = "external/rstest" }

This solution works fine until rstest gets a new version on crates.io. Then the patched version which is 0.12.0 will not be used anymore, but the newer version from crates.io is consumed.
Also, developers might oversee the need of using the patched crate and simply update the version dependency of rstest to a newer version.

Possible Solution 3: Use an own registry

It would be nice to handle our external patched crates in a kind of own registry. As of now, it's possible to define a registry in config.toml with

[source.my-vendor-source]
directory = "external"

and corresponding library Cargo.toml

rstest = { version = "0.12.0", registry = "my-vendor-source" }

But I was not able to make it work, as the registry will need an index first.
And I am also not sure if it makes sense to simulate a local registry for this use case.

So my question to the community :

How do you solve this ?

Thanks,
Peter

1 Like
2

Instead of using a path to override the dependency, what about making it a git dependency? I believe this gets around the workspace-in-a-workspace issue.

I don't think this is an issue in practice because you would need to update the original rstest = "0.12.0" line in your Cargo.toml for this to occur. If a human is required to make such a modification, you could either add a comment to your Cargo.toml or create a test that depends on your fork's functionality (e.g. it calls a function that only exists in your fork) so PRs that accidentally modify the line will be rejected by CI.

1 Like
3

Thanks for the proposal,

Instead of using a path to override the dependency, what about making it a git dependency? I believe this gets around the workspace-in-a-workspace issue.

But wouldn't this also mean, that I need to but my external dependency into an own repository ?

I don't think this is an issue in practice because you would need to update the original rstest = "0.12.0" line in your Cargo.toml for this to occur.

As our repository has quite a lot crates and dependencies, we rather use tools like cargo update and cargo upgrade to switch to newer versions. Therefore, from a practical side, a comment wouldn't help.

Having a functionality, that is called might be an option to avoid upgrade issue, but it's simply extra overhead.

4

My normal process when we need to make modifications to a dependency is to,

  1. Fork the repo to our GitHub organisation
  2. Make the modifications in some side-branch
  3. Switch our project over to the dependency. Often we just use the [patch] section, but you could directly change rstest = "0.12.0" to rstest = { git = "https://github.com/my-org/rstest", branch = "my-fix" } in Cargo.toml
  4. Make a PR to rstest so the fix can be merged and we can switch back to the crates.io version

Switching over to a forked repository only takes a minute or two, and I find it a lot less hassle than using custom registries or submodules/subtrees + path dependencies.

I guess it depends on your organisation and typical dev workflow.

At my current job we normally make sure someone else reviews changes. Typically this might just be a case of skimming through a "Run cargo-update" PR's diff to make sure it looks okay, and things like codeowners means that the correct people will be pinged.

For your workflow, it sounds like depending on the modified rstest directly (either as a git dependency if you are forking or path dependencies if you are using subtees) in the projects that need the modified code might be easier than trying to [patch] it. That way there's no chance an accidental cargo upgrade will break things.

2 Likes
5

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.