Shared build cache?


#1

I’ve been experimenting with ways to share a build cache between unrelated projects. I’m cranking out templated projects that start with the same deps. And I’m trying to aggressively cut compile times; first compile of a templated project would ideally be under 10 seconds (assuming warm build cache). Secondarily, I’d like to minimize compile after adding deps (since I anticipate overlap in many cases) and when upgrading rustc (after cleaning and reseeding the build cache).

I think the unrelated nature of these projects rules out the current iteration of Cargo workspaces. I’ve stumbled onto some early disccusions of distributed build cache per cargo #1997, but I’m really quite content for the near future with a single build server solution. I’ve had some experimental success with symlinks/bindmounts to target/release/{deps,build,.fingerprint}, though I don’t know how fragile it is to rely so heavily on this fingerprinting - this comment sounds promising but I’m not certain if it’s sufficiently complete. In my case, the environment and build command are constant, but user-editable Cargo.toml suggests that build.rs scripts could completely undermine that and possibly cause bigger problems like malicious replacement of shared build files.

So at this point, I’m chewing on 2 possible immediate solutions:

  1. Limit build cache to a per-user cache (user’s can only shoot themselves in the foot) and copy the template’s precompiled deps into the target directory when the project is initially created (rustc upgrades will be a bit painful)
  2. Strip package.build key(s) from any Cargo.toml before building, and go ahead and use a global cache (assuming fingerprinting is sufficiently robust)

Am I overlooking any major caveats in either case, or better solutions? I’m open to both short and long-term thoughts on how to accomplish some of these ideas - and possibly even contributing to a build cache solution (if this trojan-horse-of-a-project to support rust pans out).


#2

Both those possible solutions just hit a wall. Symlinking deps,build,.fingerprint did work, but mounting docker volumes to each of those does not as cargo tries to rename some things in a way that can’t cross filesystems. So I’m back to square one.