How to prevent rebuilding whole crate

Hey!

I have a project with code-generated source files with size of around 10k lines. Of course I don't touch them really often but still my compile times are very big (20-50 sec) even if I make changes only to a small file 20 lines long.

With C++ experience I expect compiler to make object files only for those sources that have changed and just reuse object files in subsequent builds. This implies that I don't see warnings for those sources that haven't changed. But in Rust I see those warnings coming from 10k files. And I don't want compiler to even touch them.

Is there any way how I can prevent compiler rebuilding all files on each cargo build?

The compiler will always emit warnings whenever it compiles something. If you don't care about these warnings, add #![allow(...)] annotations at the top of your lib.rs to ignore the warnings.

This is odd. Usually incremental compilation will cache modules which haven't been touched. Often you can update just a single file and the next build will take less than a second.

Is there anything in your crate that's out of the ordinary? Like a cargo:rerun-if-changed=$PATH in a build script, or heavy use of procedural macros or generics?

2 Likes

If the crate is a heavy macro user, these compile times can make more sense.
I have 2 of those myself and if I ever touch one of the macro's I'll notice it in compile times. Otherwise, depending on what exactly changes, the compile times are more around the 10 seconds.

In C++, the unit of compilation is the source file. In Rust, the compilation unit is the crate. If you just want to make sure that compilation is reused, then splitting the generated code into its own crate is the way to get that.

However, I suspect that incremental can get you most of the way there. The problem will be that if your build script does not emit any rerun-if-changed annotations, cargo assumes that it needs to rerun the buildscript any time any file in your project (source? not exactly sure) directory changes. And then because the generated files are regenerated, they need to be recompiled because they're newer than before.

If the files are pregenerated rather than generated from a build script, you'll probably have to split the crates short term. (Long term, incremental might get better.)

The super simplified TL;DR why it's like this: C++ has headers that make you do the hard work of ordering the compilation graph. Rust doesn't have headers, so the unit is the crate, and compilation of the crate also creates an .rlib "header" that consumers compile against.

2 Likes

A bit more clarifications.
This large files are pregenerated and even committed into the repo so they don't change at all during build process. Those files derive a lot of proc-macros but I don't change anything they depend on (neither macros nor modules). And I don't use rerun-if-changed (actually never heard about it before :slight_smile: ).

So if Rust's unit of compilation is a crate then it makes much sense. I need to think how to move all this generated stuff into a separate crate under the same workspace. I hope I'll be able to re-export it not to break public API.

BTW, with incremental compilation it's not really comparable to C++'s concept of translation units. Incremental compilation is capable of recompiling only individual functions that changed.

Try looking at timings to see what is slowing down: Exploring Crate Graph Build Times with `cargo build -Ztimings` - #2 by dtolnay - cargo - Rust Internals

1 Like

Can you clarify, do you use a build script at all? The default is that the build script reruns whenever any file changed, so you might want to use rerun-if-changed to change that.

1 Like

No, I don't use build script. Only Cargo.toml.
I say "generated files" just to emphasize that they are really big but I don't regenerate them in the build process.

That's cool. I tried it and I have only one unit in output that is actually my whole crate taking 15 seconds to build (2.5s codegen). So it does not give granular output to see separate modules

Alright, I tried to move all the generated stuff into a separate crate (but in the same workspace) and my build times have dropped from 15s to ~1s.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.