Setting global `CARGO_TARGET_DIR` still results in the crate being built multiple times

help
1

I have multiple workspaces, one for the crate and others for examples which use that crate.

I tried speeding up the CI by setting CARGO_TARGET_DIR to a common folder, so that the main crate is not rebuilt multiple times. However, while it is now build only once for examples, it is still being build the second time when the crate itself is being built and I don't understand why.

Essentially I have this structure:

+---dependency
|   |   Cargo.toml
|   \---src
|           lib.rs
|
\---user
    |   Cargo.toml # `dependency = { path = "../depencency" }` 
    \---src
            main.rs

Then I set CARGO_TARGET_DIR to a common folder like C:/temp and then run cargo build in both crates, which results in dependency being built twice. Why?

As a note, I know that putting the library and binary inside a single workspace will solve the issue, but I cannot do this in my case.

2

cargo checks several things to determine if it needs to rebuild a crate: in particular the set of enabled features is a common tripping hazard, but various environment variables also count. Here's all the horrible detail:

1 Like
3

I tried verifying that this is not related to feature flags (at least directly) by creating multiple new crates and I can clearly see that dependency crate is built twice there as well.

4

But at the same time I clearly see two .fingerprint files. One is created when I build the dependency crate and another one I build one of many user crates.

I can see the difference is in lib-dependency.json:

  -"path":17777289886553719987,
  +"path":13466389459039175021,

I also see the difference in dep-lib-dependency where modules are specified in a different order.

Everything else is the same.

And this is the output of CARGO_LOG:

dependency on  master [?] is 📦 v0.1.0 via 🦀 v1.83.0
❯ CARGO_TARGET_DIR="C:/test/dependency/target" CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build --release
   0.059695900s DEBUG prepare_target{force=false package_id=dependency v0.1.0 (C:\test\dependency) target="dependency"}: cargo::core::compiler::fingerprint: fingerprint at: C:/test/dependency/target\release\.fingerprint\dependency-f700e09833da5ba7\lib-dependency
   0.060017900s DEBUG prepare_target{force=false package_id=dependency v0.1.0 (C:\test\dependency) target="dependency"}: cargo::core::compiler::fingerprint: failed to get mtime of "C:/test/dependency/target\\release\\deps\\libdependency-f700e09833da5ba7.rlib": failed to load metadata for path `C:/test/dependency/target\release\deps\libdependency-f700e09833da5ba7.rlib`
   0.060175100s  INFO prepare_target{force=false package_id=dependency v0.1.0 (C:\test\dependency) target="dependency"}: cargo::core::compiler::fingerprint: fingerprint error for dependency v0.1.0 (C:\test\dependency)/Build/TargetInner { name_inferred: true, ..: lib_target("dependency", ["lib"], "C:\\test\\dependency\\src\\lib.rs", Edition2021) }
   0.060662600s  INFO prepare_target{force=false package_id=dependency v0.1.0 (C:\test\dependency) target="dependency"}: cargo::core::compiler::fingerprint:     err: failed to read `C:/test/dependency/target\release\.fingerprint\dependency-f700e09833da5ba7\lib-dependency`

Caused by:
    The system cannot find the file specified. (os error 2)
   Compiling dependency v0.1.0 (C:\test\dependency)
   5.608662900s DEBUG cargo::core::compiler::fingerprint: write fingerprint (1d1f3fe4d278a15c) : C:/test/dependency/target\release\.fingerprint\dependency-f700e09833da5ba7\lib-dependency
    Finished `release` profile [optimized] target(s) in 5.61s

dependency on  master [?] is 📦 v0.1.0 via 🦀 v1.83.0
❯ cd ../user

user on  master [?] is 📦 v0.1.0 via 🦀 v1.83.0
❯ CARGO_TARGET_DIR="C:/test/dependency/target" CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build --release
   0.008692400s DEBUG prepare_target{force=false package_id=user v0.1.0 (C:\test\user) target="user"}: cargo::core::compiler::fingerprint: fingerprint at: C:/test/dependency/target\release\.fingerprint\user-050454d770d83887\bin-user
   0.008976600s DEBUG prepare_target{force=false package_id=user v0.1.0 (C:\test\user) target="user"}: cargo::core::compiler::fingerprint: failed to get mtime of "C:/test/dependency/target\\release\\deps\\libdependency-72d5426137986e35.rlib": failed to load metadata for path `C:/test/dependency/target\release\deps\libdependency-72d5426137986e35.rlib`
   0.009289000s DEBUG prepare_target{force=false package_id=user v0.1.0 (C:\test\user) target="user"}: cargo::core::compiler::fingerprint: failed to get mtime of "C:/test/dependency/target\\release\\deps\\user.exe": failed to load metadata for path `C:/test/dependency/target\release\deps\user.exe`
   0.009451800s  INFO prepare_target{force=false package_id=user v0.1.0 (C:\test\user) target="user"}: cargo::core::compiler::fingerprint: fingerprint error for user v0.1.0 (C:\test\user)/Build/TargetInner { name: "user", doc: true, ..: with_path("C:\\test\\user\\src/main.rs", Edition2021) }
   0.009503600s  INFO prepare_target{force=false package_id=user v0.1.0 (C:\test\user) target="user"}: cargo::core::compiler::fingerprint:     err: failed to read `C:/test/dependency/target\release\.fingerprint\user-050454d770d83887\bin-user`

Caused by:
    The system cannot find the file specified. (os error 2)
   0.019171700s DEBUG prepare_target{force=false package_id=dependency v0.1.0 (C:\test\dependency) target="dependency"}: cargo::core::compiler::fingerprint: fingerprint at: C:/test/dependency/target\release\.fingerprint\dependency-72d5426137986e35\lib-dependency
   0.019415300s  INFO prepare_target{force=false package_id=dependency v0.1.0 (C:\test\dependency) target="dependency"}: cargo::core::compiler::fingerprint: fingerprint error for dependency v0.1.0 (C:\test\dependency)/Build/TargetInner { name_inferred: true, ..: lib_target("dependency", ["lib"], "C:\\test\\dependency\\src\\lib.rs", Edition2021) }
   0.019568100s  INFO prepare_target{force=false package_id=dependency v0.1.0 (C:\test\dependency) target="dependency"}: cargo::core::compiler::fingerprint:     err: failed to read `C:/test/dependency/target\release\.fingerprint\dependency-72d5426137986e35\lib-dependency`

Caused by:
    The system cannot find the file specified. (os error 2)
   Compiling dependency v0.1.0 (C:\test\dependency)
   5.375924400s DEBUG cargo::core::compiler::fingerprint: write fingerprint (409ea8e61a72d45) : C:/test/dependency/target\release\.fingerprint\dependency-72d5426137986e35\lib-dependency
   Compiling user v0.1.0 (C:\test\user)
   5.639403400s DEBUG cargo::core::compiler::fingerprint: write fingerprint (9ad8082a1839b5da) : C:/test/dependency/target\release\.fingerprint\user-050454d770d83887\bin-user
    Finished `release` profile [optimized] target(s) in 5.64s

user on  master [?] is 📦 v0.1.0 via 🦀 v1.83.0 took 5s
❯ cd ../user1

user1 on  master [?] is 📦 v0.1.0 via 🦀 v1.83.0
❯ CARGO_TARGET_DIR="C:/test/dependency/target" CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build --release
   0.008180600s DEBUG prepare_target{force=false package_id=user1 v0.1.0 (C:\test\user1) target="user1"}: cargo::core::compiler::fingerprint: fingerprint at: C:/test/dependency/target\release\.fingerprint\user1-2a370eb18f3bcc5a\bin-user1
   0.008550800s DEBUG prepare_target{force=false package_id=user1 v0.1.0 (C:\test\user1) target="user1"}: cargo::core::compiler::fingerprint: max output mtime for "C:\\test\\dependency" is "C:/test/dependency/target\\release\\deps\\libdependency-72d5426137986e35.rlib" 13402136473.760031300s
   0.008924600s DEBUG prepare_target{force=false package_id=user1 v0.1.0 (C:\test\user1) target="user1"}: cargo::core::compiler::fingerprint: all paths up-to-date relative to "C:/test/dependency/target\\release\\.fingerprint\\dependency-72d5426137986e35\\dep-lib-dependency" mtime=13402136468.426945800s
   0.009037300s DEBUG prepare_target{force=false package_id=user1 v0.1.0 (C:\test\user1) target="user1"}: cargo::core::compiler::fingerprint: filesystem up-to-date "C:\\test\\dependency"
   0.009143000s DEBUG prepare_target{force=false package_id=user1 v0.1.0 (C:\test\user1) target="user1"}: cargo::core::compiler::fingerprint: failed to get mtime of "C:/test/dependency/target\\release\\deps\\user1.exe": failed to load metadata for path `C:/test/dependency/target\release\deps\user1.exe`
   0.009211100s  INFO prepare_target{force=false package_id=user1 v0.1.0 (C:\test\user1) target="user1"}: cargo::core::compiler::fingerprint: fingerprint error for user1 v0.1.0 (C:\test\user1)/Build/TargetInner { name: "user1", doc: true, ..: with_path("C:\\test\\user1\\src/main.rs", Edition2021) }
   0.009258000s  INFO prepare_target{force=false package_id=user1 v0.1.0 (C:\test\user1) target="user1"}: cargo::core::compiler::fingerprint:     err: failed to read `C:/test/dependency/target\release\.fingerprint\user1-2a370eb18f3bcc5a\bin-user1`

Caused by:
    The system cannot find the file specified. (os error 2)
   0.009818500s DEBUG prepare_target{force=false package_id=dependency v0.1.0 (C:\test\dependency) target="dependency"}: cargo::core::compiler::fingerprint: fingerprint at: C:/test/dependency/target\release\.fingerprint\dependency-72d5426137986e35\lib-dependency
   Compiling user1 v0.1.0 (C:\test\user1)
   0.152295100s DEBUG cargo::core::compiler::fingerprint: write fingerprint (7b3e408a07816a55) : C:/test/dependency/target\release\.fingerprint\user1-2a370eb18f3bcc5a\bin-user1
    Finished `release` profile [optimized] target(s) in 0.15s
5

In my experience a global CARGO_TARGET_DIR does not play well when you have multiple crates with the same name (including multiple path dependencies to the same crate).

6

In the real example the names are different.
What I do not understand is why for building a dependency separately and as a part of another crate, the different paths are used?

dependency on  master [?] is 📦 v0.1.0 via 🦀 v1.83.0
> CARGO_TARGET_DIR="C:/test/dependency/target" CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build --release
    ... fingerprint at: ... .fingerprint\dependency-f700e09833da5ba7\lib-dependency
   
user on  master [?] is 📦 v0.1.0 via 🦀 v1.83.0
> CARGO_TARGET_DIR="C:/test/dependency/target" CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build --release
   ... fingerprint at: ... .fingerprint\dependency-72d5426137986e35\lib-dependency

user1 on  master [?] is 📦 v0.1.0 via 🦀 v1.83.0
> CARGO_TARGET_DIR="C:/test/dependency/target" CARGO_LOG=cargo::core::compiler::fingerprint=trace cargo build --release
   ... fingerprint at: ... .fingerprint\dependency-72d5426137986e35\lib-dependency
7

Cargo uses paths relative to the workspace root when building local (as in not from crates.io or a remote git repo) crates. When you don't explicitly use workspaces, each individual package is a workspace root. So if you build directly build dependency, cargo will cd into the dependency directory and do rustc src/lib.rs, while if you build user, then cargo will cd into the user directory and do rustc ../dependency/src/lib.rs for building dependency. This ensures diagnostics are always relative to the workspace root rather than the directory of the package that happened to be built.

1 Like
8

Does that mean if I specify the path to dependency in user1 differently, it will build the dependency the third time too?

9

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.