Can a large Cargo.toml be split into multiple files?

help
1

Alright,

I understand that this might be a bit of a corner case, but I have a fast growing mono-repo with over 100 crates. In the last 6 months, the number of crates have already doubled, and there is no sign this trend will end any time soon. Bazel does the primary build and all CI stuff for obvious reasons, but I also maintain a Cargo config to preserve the rich Cargo tooling i.e. linting, checking, formatting, security scans etc. that Bazel either cannot run or are fairly complicated to setup. If anything, Cargo has become a really good ecosystem for dev tooling, hence I have strong reason to keep Cargo for as long as it is practical.

However, my main problem, though, comes down to the workspace Cargo.toml growing so large that it is a bit of an issue now, but most likely become a bigger problem when the number of crates double again.

In the repo, each crate has its own Cargo.toml file, but for managing dependencies, internal and external, each crate references the workspace Cargo.toml file.

External deps refers to every crate from Crates.io whereas internal dependencies refers to an alias to an internal crate. The later means that every internal crate that references another internal crate only by alias instead of its path. Using an alias has the implication that refactoring a crate location boils down to just updating the path in the alias defined in the workspace Cargo.toml. The alternative would be that you would have to search and update the path across the entire mono-repo, therefore aliasing is a meaningful time saver.

The workspace Cargo.toml file contains three sections that all grow really fast:

  1. The list of workspace members
  2. Internal aliases.
  3. External crates from Cargo.io

Therefore, I wonder, is there a way I can split the workspace Cargo.toml into multiple smaller files, such as:

  1. Sub-config file for general settings i.e. metadata plus profiles
  2. Sub-config file of all workspace members
  3. Sub-config file of all dependencies?

I did some online search and could not find any meaningful answer.

Also, I don't think that just a 100 crates is exactly large compared to what's out there in the industry, but I wonder what else can be done to maintain Cargo.toml as the monorepo grows further?

2

You don’t necessarily need to list all members explicitly; workspace.members can contain globs, e.g. members = ["client/*", "server/*"].

The Cargo reference also says:

All path dependencies residing in the workspace directory automatically become members.

but I’d guess it means dependencies of the root package, not workspace.dependencies, so that probably doesn’t help — but if it does, then you may be able to have a nearly empty member list.

1 Like
3

Thank you @kpreid

This does help quite a bit. I applied a bunch of globs, and it has reduced the number of workspace members quite a lot. However, somehow these globs are non-recursive meaning, I can do "server/*" but not "server/*/*"

It really is a missed opportunity that Cargo does not allow for nested workspaces. Anyways, I think I can get a long way with globs.

thank you for your help.

4

Actually, I have to correct myself.

Turned out, workspace members can be declared with kinda recursive globs, but only when path depth is strictly uniform. I mean, this is not exactly great, but at least something I can workaround with a uniform folder structure.