Because project using crates like bevy tend to have very big targetdirectories (especially because of the intermediate build artifacts I guess), it would be nice to be able to have all these build artifacts in another place (say an external storage). Currently, this is only possible by manually setting it via the build.build-dir option of the config.toml manifest, so it would be nice to have an option that redirects every build-dir to this location. Is this already an option I did not know of, are there any hacks to do it or is it currently impossible?
I think with target-dir = I can select a single target dir for all builds, and have the additional benefit that duplicated artifacts are avoided, e.g. when I have multiple local Bevy projects. Currently uncommented, because I am doing some build tests on /tmp.
stefan@hx90 ~ $ cat .cargo/config.toml
[target.x86_64-unknown-linux-gnu]
linker = "clang"
rustflags = ["-C", "link-arg=-fuse-ld=mold"]
#[build]
# Use a single absolute path for all build artifacts
#target-dir = "/home/stefan/.cargo/target_shared"
[profile.dev]
# One debuginfo file per dependency, to reduce file size of tests/examples.
# Note that this value is not supported on Windows.
# See https://doc.rust-lang.org/cargo/reference/profiles.html#split-debuginfo
#split-debuginfo="unpacked"
Target directories can become huge. Having them scattered around the drive can be a pain when you need to free up space. Using a shared location allows you to just wipe that directory away.
You can put the target dir in a ram disk for faster builds and less wear and tear on your solid state drives. (Though I haven't found a good solution for this on Windows.. The ram drive I use on Windows doesn't support some disk operation that the Rust standard library attempts to use).
Some cons with having a shared target directory:
Building two projects at the same time will cause one to block the other. This can be annoying if you have tools running in the background that may lock the target directory when you might not really expect it. (I don't use rust-analyzer myself, so it isn't really an issue for me).
As you say, cleaning will cause all targets to be wiped. However, one can always "opt out" and specify a different target directory when needed. (Our build-and-packaging pipeline builds release builds in separate target directories, just to be sure no other process will be touching the files).
I ran into an odd corner case where I wanted to build a crate from another crate's build script, but couldn't because the target directory was locked. This would have been possible of the target directories were separate. At the time it was a little annoying, but looking back on it I'm kind of glad it didn't work because it was not pretty.
One of the first things I do when setting up a new system is set up a shared target directory, so I definitely think the pros outweigh the cons. But remember that you can easily opt-out for specific cases if you need it.
Never heard of the principle of a ram disk, but that sounds pretty cool, definitely an option I will consider. I did not think of the file locks. Cargo does ensure that the same crate with different configs (like opt-level or lto) are never shared, right?
If you do some spelunking in the target directory you'll notice that it creates a bunch of directories like zerocopy-13198fb76fa3eaf4, zerocopy-1ab5c672fcfa4916, etc. I don't remember the exact details of what goes into those, but IIRC those numbers are hashes of various options/features, so it can keep multiple different builds separate.
But with regards to opt-level and lto and such -- if you're using build profiles to select different build types, then the build system will create a subdirectory for for the profile under which it builds everything. If you're switching by changing the opt-level and lto (or similar) settings within a build profile, then I think (someone please correct me if I'm wrong) the hashes will ensure that different builds don't interfere with each other.
ImDisk Virtual Disk Driver is designed to be a small, simple and yet powerful virtual disk driver. It runs on very old versions of Windows NT as well as modern Windows versions. However, because of this compatibility design and because it emulates disk volumes rather than complete disks, it is not always compatible with all applications and drivers. For instance, you cannot manage things like mount points, drive letters and similar for ImDisk drives using mountvol command line tool or in Disk Management in Windows. As another example, you cannot create or access shadow copies on ImDisk drives. So, applications that use similar Windows features as Disk Management dialog to enumerate disks and disk volumes to find disk properties like sector sizes and similar, might possibly not work as expected with ImDisk drives.
luckily, the author has also written a driver supporting whole disk emulation, it uses the SCSI miniport driver API and is well documented here:
a prebuilt and signed driver is included in the repository, under the DriverSetup directory. it is included in the "Arsenal-Image-Mounter" package from the website too.
you can write a small program (or windows service) to create and mount a ramdisk using the driver.
if you are using the "ImDisk Toolkit" package now, you can simply switch to AIM Toolkit, written by the same author, which has similar functionality but using the AIM kernel driver.
I don't recall which one I use -- IIRC it was some AMD branded one, but it might be a licensed ImDisk. But yes, the appears to be the exact same limitation (I tracked the problem down to a canonicalization call in Win32, and that's what's described in the issue as well). Back when I was originally looking for a ramdisk for Windows, I settled on the one I use because it supported slightly larger size than the others. At first I couldn't figure out why they all insisted on only supporting relatively small ramdrives, but then I noticed that multiple of them supported persistence over reboot (by storing to disk and restoring from disk on boot) .. so that may be a reason.
AIM Toolkit looks interesting -- if that can replace the one I'm using an allow me to point my target dir to a randrive on Windows as well, then I'll be a very happy camper!