`cargo run --bin <some_bin>` causes build of unused dependencies

help
1
Overview 
+-src
  +-bin
    +-heavy.rs
    +-hello_world.rs
  1. I have a binary that uses a heavy dependency that takes a significant amount of time to compile:
// heavy.rs
use <heavy_crate>::<some_function>;

fn main() {
  // using <some_function> from <heavy_crate>
}
  1. I create another file in the src/bin directory that looks like this:
// hello_world.rs
fn main() {
    println!("Hello World!");
}

When I run cargo run --bin hello_world I see in the output a compilation of heavy dependencies used in the first binary heavy.rs.

Is this expected behavior?

It seems to me that quite recently, this did not happen, and I broke something in my project. Previously, compiling individual binaries that didn't have heavy dependencies (which could be used by neighboring binaries) was lightning fast. In the current situation I have to wait 20+ seconds for hello_world to compile on my machine, while with clean project I spend less than second on it.

2

Yes, it's expected (for now). Every binary in the package has access to all of the package's dependencies.

You can split the small binary into its own crate within a workspace. That way the small binary doesn't have the same dependencies as the other package. This is how we solved similar problems in pixels, which has many examples (binaries) that each depend on very different libraries.

There's a loooooong thread with plenty of details at Allow dependencies that only apply to specific cargo targets (bin, example, etc.) · Issue #1982 · rust-lang/cargo · GitHub

2 Likes
3

Thank you, now I understand that this is expected and am more calm. :relieved:

This doesn't quite relate to the title of my question, but could you please clarify that if I make changes to hello_world.rs, is it expected that all my crate's dependencies will be re-compiled, or is there some form of caching and the dependencies will not be re-built? For example:

fn main() {
    println!("Hello again!")
}
4

Well, there are many configurations that affect compile time. Incremental compilation is enabled by default on the dev profile, and it can be enabled on the release profile if needed. Linking time can be pretty bad with large dependencies, and using rust-lld might improve that aspect. Disabling LTO might improve build times slightly at the cost of some optimizations getting missed. And so on.

If you can avoid large dependencies, that’s usually the most thorough solution. Splitting the bin artifact into its own crate is a good example of that strategy.

5

Something that helps a bit, is: sccache - Github

Install it via:

cargo install sccache --locked

Put these lines into $HOME/.cargo/config.toml

[build]
rustc-wrapper = "/path_to_your_home/.cargo/bin/sccache"

sccache use environment variables for configuration. So put somewhere in in your dotfiles something like this:

export SCCACHE_CACHE_SIZE=100G
1 Like
6

I'm already using mold linker via .cargo/config.toml with:

[build]
rustflags = [ "-C",  "link-arg=-fuse-ld=mold" ]

Previously I can't install lld from my os distribution repo because of dependencies conflict. I hear that mold is quite unstable. I will try to test lld.

The default profile for run subcommand is dev, so it has opt-level = 0 and lto=false:

No LTO is performed if codegen units is 1 or opt-level is 0.

I will try sccache mentioned by @mroth, but I understand that splitting project using workspaces is a good and and in fact I prefer this strategy as you advise.

Thanks again for the clarification about incremental compilation. :wink:

7

No problems here. I'm using mold without any issues, compiled from stable branch:

[target.x86_64-unknown-linux-gnu]
linker = "clang"
rustflags = ["-C", "link-arg=-fuse-ld=/usr/local/bin/mold", "-C", "target-cpu=native"]
8

FWIW rust-lld ships with Rust on most supported platforms. You probably already have it! My ~/.cargo/config.toml looks like this:

[target.aarch64-apple-darwin]
linker = "rust-lld"
rustflags = ["-C", "target-cpu=native"]

[target.x86_64-apple-darwin]
linker = "rust-lld"
rustflags = ["-C", "target-cpu=native"]

[target.x86_64-pc-windows-msvc]
linker = "rust-lld.exe"
rustflags = ["-C", "target-cpu=native"]

Nightly compilers enable it by default on Linux: Faster linking times on nightly on Linux using `rust-lld` | Rust Blog