Hello! I’d like to tell you about a tool I’ve been working on called sccache.
sccache is a compiler cache like ccache, but it supports some non-local-disk storage options and also MSVC. You can read more about it in my original announcement blog post. We’ve been using it to cache C++ compilation in Firefox CI and Rust’s CI (since Rust builds LLVM) for a while now. In the past few months I added support for caching Rust compilation (with a lot of help from @alexcrichton!) and I think it’s ready for some wider testing.
sccache cannot cache every type of Rust compilation. Currently it will only cache compilations that output an
staticlib. This does mean that for most Rust projects you’ll wind up with a bunch of things that can’t be cached like build scripts and procedural macro crates. There’s still a significant build time win to be had even with these restrictions (see below). sccache’s Rust support has been written primarily with the goal of caching rustc invocations as produced by cargo. It does not attempt to support the full range of valid rustc commandline options. It should always do the right thing, it may just refuse to cache.
Currently you’ll need to pull and build sccache from GitHub. There is an old version on crates.io, I plan to get that updated shortly so that
cargo install sccache will work. I’d also like to get binaries published with GitHub releases to make things simpler, I will update this post with details when I get those sorted out.
The cargo that will ship with Rust 1.18 supports a
RUSTC_WRAPPER environment variable that makes using sccache with cargo simple: you just set
RUSTC_WRAPPER=sccache if it’s already in your
PATH). By default sccache uses a 10GB local disk cache in a reasonable per-OS location.
I’ve been testing sccache with servo, which I suspect is the largest stress test possible, as well as with webrender, since one of my colleagues had reported issues with that crate. The servo timings here include an as-yet-unsubmitted fix to update the
gl_generator crate to get a fix I submitted there. I plan to submit a servo PR for that ASAP. These timings are from my Linux desktop machine, a several-year-old Core i7 (
Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz) with 32GB of RAM and an SSD.
webrender, without sccache:
luser@eye7:/build/webrender/webrender$ cargo clean; time cargo build <...> Finished dev [unoptimized + debuginfo] target(s) in 70.78 secs real 1m10.989s user 2m55.532s sys 0m4.560s
webrender, with sccache:
luser@eye7:/build/webrender/webrender$ cargo clean; time RUSTC_WRAPPER=/build/sccache2/target/release/sccache cargo build <...> Finished dev [unoptimized + debuginfo] target(s) in 17.3 secs real 0m17.823s user 0m52.312s sys 0m1.892s
For servo builds, I used
ccache = "sccache" in my
.servobuild for both tests because otherwise building servo spends forever compiling all the C code in -sys crates.
servo, without sccache:
luser@eye7:/build/servo$ ./mach clean; time ./mach build <...> Build Completed in 0:07:15 real 7m15.579s user 36m21.104s sys 1m42.128s
servo, with sccache:
luser@eye7:/build/servo$ ./mach clean; time RUSTC_WRAPPER=/build/sccache2/target/release/sccache ./mach build <...> Finished dev [unoptimized + debuginfo] target(s) in 233.20 secs [Warning] Could not generate notification! Optional Python module 'dbus' is not installed. Build Completed in 0:03:56 real 3m56.748s user 17m51.400s sys 1m11.572s
selected sccache stats from building servo with a warm cache:
$ sccache --show-stats Compile requests 930 Compile requests executed 734 Cache hits 734 Non-cacheable calls 164 Non-compilation calls 32
Feedback, next steps
If you try sccache out I’d love to hear from you! It is primarily designed as a tool for CI, where we build the same things over and over on different machines, but I’ve been working on making it more usable for local development as well. I plan to enable sccache caching of the Rust compilation we do during Firefox builds in CI, and also investigate using sccache during Servo CI builds.