Workspace workflow tips, gotchas, or recommendations?


#1

I’ve not used workspaces yet, mostly because they introduce a new set of problems to deal with and I’ve not taken the time to explore how best to work with them. I’d like to soon for several of my crates.

So what works well for you and what doesn’t?

Examples

  • How do you do tagging? <crate>-<X>.<Y>.<Z> for each sub-crate?
  • How do you orchestrate your CI test step? Is --all in enough tools and good enough?

#2

I use workspaces for private projects:

  • everything is in a git monorepo, except some shared crates that are submodules (ugh) or just separate git checkouts.
  • I don’t set [workspace] members, since using [dependencies] foo = { path = "../foo" } automatically implies this
  • [dependencies] foo = { path = "../foo" } is pretty cool and convenient. And it’s automagically changed to crates.io reference if you publish the crate.

cargo build --all and cargo test --all work fine.

Bumping of versions and publishing to crates.io each subcrate individually is a bit annoying, but as I’ve mentioned, I mostly use such setup for non-public projects, so I haven’t sought solutions for this one.


#3

I use workspaces for private projects:

Unfortunately for me, I’m looking at situations where this isn’t the case. I know at least @BurntSushi does this with ripgrep. Hopefully he has time to jump over here and share some of his lessons he has learned.

Though I imagine this can’t handle trying different combinations of features. I remember someone was making a tool to try different feature sets, maybe adding workspace support to that would be the way to go if you want every combination tested. I normally only go for default, no-default, and all.


#4

For CI, I do:

  • From workspace root: cargo build --all-targets, then cargo test --all

    The --all-targets is meant to make it also build the examples for each crate, because I have tests for those and I don’t want them to be compiled in the test. However I suspect it might actually not be pre-compiling the examples (just checked CI output for those tests, it says it’s been waiting for over 60 seconds).

  • cargo-make to collect workspace coverage:

    KCov needs to be installed (can’t remember if cargo-make does this for you), and then:

    cargo make --no-workspace workspace-coverage
    
  • Have a separate job that does “conformance” checks:

    • Clippy
    • Rustfmt
    • cargo-audit (vulnerability checks)
    • cargo license (make sure I don’t accidentally pull in a GPL licensed crate dependency transitively)

    I run this on nightly, for clippy’s sake (and Rustfmt has better rules). I stick with a version of nightly and update it periodically (clippy-nightly compatibility used to break often).

There are a few types of commands/effects that I tend to want to do in a workspace:

  • Run this command at the workspace root, but the side effect is in each crate (e.g. cargo fmt --all)
  • Run this command for each crate, but the side effect is in the workspace (e.g. get code coverage where the source files are in each crate, but put them in workspace_root/target/coverage)
  • Run this command at the crate level, but use a common value from the workspace root (e.g. some shared assets or rustfmt.toml lives in the root)

cargo-make kinda suffices for the latter two bits, but some things don’t work on Windows because it does delegate to scripts in some cases. I was experimenting on making a pure Rust tool (hull) but the need hasn’t been so great for me to properly do it. It was going to be my answer to building workspace member crates with features specified at the workspace level, but I didn’t go that far.


#5

One of the biggest issues with workspaces is that they interact badly with features. See https://github.com/rust-lang/cargo/issues/4463 for details, but TL;DR is that cargo build --package foo and cargo build --pacakge foo --package bar could enable different features for foo.

See http://aturon.github.io/2018/04/05/workflows/ for a potential new Cargo feature which might help here, and https://matklad.github.io/2018/01/03/make-your-own-make.html
for an awful hack use it with today’s Cargo


#6

I probably don’t have many interesting insights. The primary reason that I switched to workspaces (for basically any repo I’ve actively worked on that has multiple crates) is so that they all share the same target directory, and therefore, compile times are better if you’re like me and tend to hop around and test each crate individually.

Otherwise, for the most part, workspaces have just worked. But I don’t have any particularly complex setup. I’ve found that --all works just fine in CI.

One thing that is somewhat painful is doing releases. Typically, the “main” crate gets a x.y.z tag while any sub-crates get a {crate-name}-x.y.z tag. But, this was true before I used workspaces as well. What I mean to say is that workspaces haven’t really changed how I do releases at all.


#7

@BurntSushi I believe @epage is mostly interested in “multiple crates in a single repository” with a side question of “and how do workspaces affect that” since they will commonly be used (just from my own understanding of where this thread is coming from).

One place to look in the near future may be futures. This is a relatively large workspace based project (probably 9 crates once the 0.3 release happens) with (I believe) plans to allow independent breaking releases of these crates. As part of the 0.1 releases there were only 2 crates in the repo, which had independent version tags. Since 0.2 was very short lived and 0.3 isn’t out yet I don’t think the plan for how to manage tags is fleshed out yet, but maybe there’ll be some insights from there soon.

The other well-known project I know of with a multi-crate repo is serde. Scanning through the tag to tag diffs it appears they bump the versions of serde, serde_derive and serde_test in lockstep, even when the latter two have no changes. Maybe @dtolnay might have some insights to share?


#8

Yes, by workspace, I was really meaning monorepo development which seems to typically be done using workspaces.

Other points I’ve been thinking of.

It seems each crate would have a README and CHANGELOG next to the Cargo.toml file. This would work well for crates.io and github even seems to automatically render the README which is more than I expected.

For crates.io, ripgrep at least seems to link to the subfolder for the repository link. I would have assumed it would go to root. Good to know.

Any issues people have run into with tracking Issues?


#9

I think there have been a couple requests from folks to split out crates, e.g., termcolor. My guess is that it will happen eventually. They are only in the ripgrep repo because it was just easier to develop that way. But as they mature and see fewer changes, it probably makes sense to split them out to their own repo, primarily because there isn’t any tight coupling. If there were tight coupling, then I’d probably keep them in the same repo.

e.g., the grep crate is going to soon become multiple crates, centered on grep as a public dependency, so there is some coupling there. I haven’t decided quite yet how I’m going to handle which repos they go in, but I’ll probably start simple.


#10

One tip I do have is to add symlinks up to the license files for each sub-crate, cargo publish will then copy the real license files into the package for you, and at least for the commonly used Apache-2.0 license you technically do need to include the license text in the package. This appears to be a pretty common mistake, I have so far opened PRs to 7 projects to fix it.