So many crates break when you set target-dir in ~/.cargo/config

Despite some significant performance-related (when using /tmp) and storage-related (when using /tmp or ~/.cache) benefits from setting target-dir in ~/.cargo/config, so many crates refuse to work with it and it is sad.

Personally, we use /tmp on raspberry pi's so as to not eat through USB sticks, and we use ~/.cache on the computer so we can rm -rf it when it gets up to a few hundred GB. We feel like these are important use-cases, and not supporting them is inconsiderate. We encourage others to support them too.

2 Likes

so many crates refuse to work with it and it is sad.

What causes the problem?

What should crate authors avoid to be compatible?

2 Likes

If anything in the build process hardcodes the target dir, things break.

This tends to be the case with projects that have a makefile to massage built artifacts after cargo. But anything that assumes the user is happy with the cargo default of polluting the current git repo's work dir with hundreds of gigabytes of cache junk by default, as opposed to putting it all in a central location that is easy to clean up as needed, is gonna have issues.

(Sorry, we're kinda salty about cargo rn.)

Do you have examples? I run with $CARGO_TARGET_DIR set to an absolute path (on windows) and have never run into a problem (beyond sometimes needing a cargo clean from switching to/from miri).

Even crates which hardcode artifacts into ./target shouldn't break, it's only crates which try to read from a hardcoded ./target path that would break.

As for best practice:

  • First off, avoid running cargo from build scripts if possible. This can result in deadlocks (recursively grabbing the build directory lock), especially if running with rust-analyzer active as well. (I recommend putting r-a's target dir distinct from your main target dir to avoid blocking manual cargo invocations, even though this does duplicate the cache somewhat.) However, while this consistently produced a deadlock a couple years ago, it seems to not hang now? (Perhaps because this attempt I used --locked? Needs more testing.)
  • If you output files in a build script, use $OUT_DIR.
  • If you want/need to access the target directory (e.g. you're running a child cargo invocation and want to copy the output into $OUT_DIR since --out-dir is still unstable), use cargo metadata --no-deps to grab the target directory rather than trying to produce it yourself.

Ah, that would be the difference: I'm happily still in the world where cargo is the orchestrator.

This is the default of most any build system. (Off the top of my head, npm, cargo, Visual Studio, and Maven/Gradle all put temporary build artifacts in cwd by default. And Make doesn't have a real default since it's just a bash script orchestrator.) Putting artifacts not in the current working directory is the (hopefully well supported) exception, not the norm.

3 Likes

Just to make sure I understand correctly... by "build scripts" here you mean cargo build scripts (i.e. build.rs) correct? Purely for my understanding as your best practice points seem very reasonable.

1 Like

Regarding the broader point... I do sometimes wish Cargo had a feature to centralize the storage of built dependencies(basically the deps directory in target/debug) because it always seems a little disconcerting to start a small sandbox project and see it use up 1 gig of hard drive space because you included Actix. The obvious counterpoint is that after the recent-ish kerfuffle over incremental builds I can easily appreciate how hard that would be to get right.

1 Like

Eh, for all we really care, cargo could use ~/.cache/cargo/target/$(pwd | sha256sum - | cut -d ' ' -f 1) by default and that'd be okay. Being able to easily nuke it is the useful part.

This isn't cargo's fault. Maybe you should be creating tickets against the offending crates?

If it's done in a Makefile, they can find the target directory using cargo metadata --format-version 1 | jq '.target_directory'.

5 Likes

I don't think they could rely on having jq installed, but that would be a solution.

1 Like

they can rely on cargo run (probably)

will do. tho cargo should really change defaults here so as to break everything accidentally relying on it. :‌p

Yes. It's the case of recursively calling cargo from a cargo invocation that has the potential of deadlocking. I should find my old issue about this...

1 Like

Yeah, you can easily put together the equivalent with sed or grep - I chose to use jq because it's obvious what you are trying to access, whereas it's easy to miss the important details when you're looking at a regex.