Why doesn't Rust incremental compiler store compiled libraries in central repository?

So I compiled a very simple project based on Warp yesterday, they recompiled it again today. Both times took about 15 minutes for initial compile. Which I suspect means that Rust's incremental compiler doesn't share compiled versions of libraries between projects.. how hard would this be to do? Is there a reason not to?

To be clear, you mean you compiled a different project today, right?

In that case you can use sccache, but you have to set it up.

2 Likes

I'm a bit surprised at your experience.

Typically I find my Rust projects can take some minutes to compile from clean as it compiles all the dependencies. But subsequently when I make changes to a file or two it only takes seconds to recompile. It only compiles the files I changed.

Starting from clean and then doing debug build it will again take a long time to do a release build.

Normally I only do "cargo check" when riffing on code so it all goes quickly enough.

Exactly, same project in the sense that it was the exact same code snippet. BUT it was compiled starting from a fresh project in a new location. sccache does seem a little like what I'm looking for, seems like it should be the default, not sure why not.

1 Like

I think that the way that cargo is setup would require putting all of you compiled assets into the same place. You can override CARGO_TARGET_DIR ( IIRC ) environment variable to do that, but then you can only cargo clean all your projects at the same time, and all of your built binaries, libraries etc. will all go to the same dir.

sccache is a very useful tool that probably does more closely what you want to.

1 Like

Isn't that down to the fact that every program I have may depend on different versions of the some crate?

It would still work with different versions, if they happen to use the same version.. great! no need to recompile. If not, then go ahead and store both versions in the same central repo.

I have had problems when taking some new version of a crate into use, finding my program no longer builds. Not a disaster for me but requiring I put time into tweaking my program to work with the new version. If I had a number of programs using that crate I would have had to modify them all if they depended on the same compiled blob. Probably not what an organization with many huge programs wants to happen.

I was wondering if sccache does that. Does it and/or cargo know what version of anything is once it has been compiled, such that it can select the right one?

The pressing need for sccache for me is not saving compile time but the fact that having all target directories around full of build artifacts eats many gigabytes of by little SSD,

Unfortunately it's not that simple. It's not just that you are using the same version, you also need to have exactly the same

  • Feature flags
  • Environment variables monitored by cargo (see cargo:rerun-if-env-changed=VAR)
  • Files monitored by cargo (see cargo:rerun-if-changed=PATH)
  • rustc and cargo versions
  • Host and target architectures/OS
  • and probably a bunch of other things I'm forgetting

Feature flags are used to conditionally add functions/modules/dependencies/whatever to the compilation. Build scripts may compile a native dependency or generate different Rust code based on an environment variable (e.g. a value to be injected or directory containing the native library's source code) or file (e.g. C source code that will be compiled).

Rust doesn't have a stable ABI so there's no guarantee artifacts generated by two different versions of the compiler will be able to work together (different calling conventions, missing metadata etc.).

The host and target bit should be fairly self-explanatory. There's no point caching some x86 machine code for use on an ARM processor.

Given all of this, you can get an idea for why the compiler will sometimes decide to recompile something than use a cached version. Tools like sccache and bazel try to help, but cache invalidation is hard so they might be a bit conservative when deciding when to reuse a cached artifact.

4 Likes

Well, I don't think sccache will actually save you space, though. All of the artifacts still end up in your target/ dir and you end up with a ( configurable ) 10GB extra in the sccache dir.

sccache wraps rustc and tries to detect when something Cargo calls rustc with exactly the same parameters as it did at an earlier time so that it can output the same exact artifact from its cache instead of re-computing the artifact. That means that the generated artifact still gets copied to your target/ dir for each project and only ends up adding 10GB of cache to your disk.

I think sccache is only meant to speed up builds by caching and/or distributing builds across multiple machines ( an intriguing fefature that I haven't tried before ).