Compilation Time vs cpu cores


#1

The code base i’m working on has grown big in the last year and a half since we started in rust, and as with big code bases comes the plague of compile time. I think cargo builds automatically take care of invoking parallel compilation of various rlibs as i usually see all 4 my cpu cores peak out during compilation (esp. after cargo clean which will recompile all libs again). Release builds take much longer than it used to and i remember same thing with C++ projects previously. The difference is that now i am planning to build a new system as mine is old. This i think is one area where multicore could make a serious difference. So just out of curiosity what are your experiences with say 8 core or more cpu’s on compilations in case you have been working on huge projects as well. Does it make noticeable difference or does it stops mattering after 4 or so cores ? Also what config do would you go for if say you needed to put up a desktop that your team of 4 or so people could use as a build server ?

Very few people actually write about configurations needed for software developers and compilations etc. as most of the review out there is about gamers and graphics card (most of the time i spend is in my (Linux) terminal so i hardly care in that dept) and all. So i just thought I would ask the take of hard-core developers on what they think would be a good config for a desktop/laptop (ram, cores/specific-processors, ssd’s).


#2

Have you tried the incremental recompilation that landed in nightly recently? Does it help, and if so, by how much?

ustulation (https://users.rust-lang.org/users/ustulation)
August 13

The code base i’m working on has grown big in the last year and a half since we started in rust, and as with big code bases comes the plague of compile time. I think cargo builds automatically take care of invoking parallel compilation of various rlibs as i usually see all 4 my cpu cores peak out during compilation (esp. after cargo clean which will recompile all libs again). Release builds take much longer than it used to and i remember same thing with C++ projects previously. The difference is that now i am planning to build a new system as mine is old. This i think is one area where multicore could make a serious difference. So just out of curiosity what are your experiences with say 8 core or more cpu’s on compilations in case you have been working on huge projects as well. Does it make noticeable difference or does it stops mattering after 4 or so cores ? Also what config do would you go for if say you needed to put up a desktop that your team of 4 or so people could use as a
build server ?


#3

Is incremental recompilation on by default on nightly or how is it triggered?


#4

You can enable incremental recompilation (highly unstable) by using rustc -Zincremental=directory, or if using Cargo, cargo rustc -- -Zincremental=directory. Note that it may not work.


#5

So is this supposed to decrease compilation time on a single crate if I touch a single file? Because in my test it doesn’t. rustc -Z help does not mention other options you can pass to -Zincremental=... are there options other than directory ?


#6

In my experience, it stops mattering after 4 or so cores, at least for Rust, for now.

I’d love to hear others’ experience, especially those who work on Servo.


#7

For release builds, using -C codegen-units=N should help with that, but the biggest bottleneck comes from linking anyway.

Wasn’t there a Rust attempt at parallel linking?