we see that rust compiler always compile the crate when we want to compile my project i see that this can take a time and this is good if we want the binary to be one file without any shared data but what about if i want him to do shared data so i use pre built crate for fast compile + small size
i want is as feature if it is and if not i see that we need the pre built crates or can we build it sparely and use it in our project without build it another time
it is possible to have dynamically linked dependencies in rust for example bevy provides it as an option for ease of development
in a normal scenario tho your dependencies should be cached so it shouldn't be much of a problem you just compile everything once and then reuse the cache for most of the following compilations
I created this tool. Edit : created by AI and me in case for info to people that don't want to use thing related to AI
It precompile crates, saves the rlib in central folder outside the project dir, then can reuse the rlib in any project in any folder
It is far from perfect but the idea is clear, and perhabs can be adapted in Cargo, eg cargo add tokio --features=full --global
The effects are clear :
You don't need internet anyway once cargo has cached the crate sources and a valid lockfile exists. And cargo has --offline to allow it to pick a different semver compatible version that is already cached if the version in the lockfile is not available locally.
Ah I didn't say it more detail
What I mean by offline is
Cargo can't do that currently. Cargo caching is local not shared to multiple projects. It is equal to not deleting the target folder, the difference is Cargo only saves the code, so --offline flag after cargo clean still recompile from scratch. The auto semver switch works if the used crate minor version is not lower and higher than the cached. My tool currently does not has this, I will add it :]
The different with what Cargo has is that, Cargo is local scope while this one is global scope, meaning the saved rlib can be used in "cross project fully offline". The cross project is the primary idea that it is not target folder in compilation result or cargo source cache
To clarify, what you mean by "offline" is both not using the network and not compiling the dependency from source?
Not compiling from source is not related to network, I just show the dowside of the current Cargo is it recompiling again other than it is also local to the same project only
The network part it, reusing the rlib across any different project in any different folder without redownloading again, current Cargo can't do this
The main goals of the project is speeding up compilation speed by reusing compilation artifact globally like .so shared library but without needing any binding code. That then automatically get the no compilation artifact duplication. And automatically get the offline thing across any different new project in any directory not only in 1 project like the standart target folder and current cargo
I would have thought cargo would reuse the local cache of downloaded .crate files (for the same user, at least), just maybe recompile them for different target directories.
Cargo cache does not work for different target directory. A cache produced by project A can only be reused by project A, not project B
A better mechanism would be to try it directly, not guessing
cargo new a
nano Cargo.toml, add rand = "0.8.6"
cargo run
Then new project in any folder
cargo new b
Same lib, same version nano Cargo.toml, add rand = "0.8.6"
cargo run --offline
It will give error because the cache does not work across different project
Cargo also only cache the source file. Source file != rlib, rlib is like the Rust version of .a/.so shared library designed for direct Rust to Rust code. So there is no precompiled shared library in the cache, only source code, that makes after cargo clean, running --offline flag needs to recompile from scratch
The source cache absolutely does work across projects. It is in ~/.cargo for a reason rather than target/.
I can't understand what you’re trying to say because your words is vague
First, we all know files in the target/ folder are dedicated to each specific project and cargo clean will clean it
Second, saving files outside of target/ doesn't limit them from being used in project, in fact, it opens the path for them to be used by multiple projects
So what is the actual correlation between reconfirming the cache doesn't work cross project, and then saying "it is in ~/.cargo not target/ for a reason" ?
And what exactly are those reasons?
It would be much easier to understand if you type it clearly. For example, "The cache doesn't work across different projects because .... the reason. It's saved in ~/.cargo so cargo clean won't wipe them only clean target/ folder
For context, the tool also saves the cache in ~/.rlib which is just equal to ~/.cargo both is external folder outside the project folder
I forget if I've already asked this before, but what is the advantage over using a shared target directory? If you put the following into your ~/.cargo/config.toml, then each project you build locally uses the same target directory, thus making sure all intermediate artifacts are compiled only once between all projects. The only downside is that final artifacts are also in a shared directory, but I imagine that downside is roughly equal with the downside of having to use an external tool in addition to cargo.
[build]
target-dir = "~/some/dir/you/chose" # commonly `~/.cargo/target`
I think it's right approach for most people. All my projects use the same set of crates, so why to have a copy of them for every project?
If you use RUSTFLAGS interchangeably you might get hampered.
I'm saying the source cache is shared across all projects. As such builds work offline as soon as you have downloaded all dependencies you need for a given project, even if it was downloaded for another project. The build artifacts are not shared by default, but you don't need build artifact sharing for offline builds to work. It merely saves time.
I believe it would cause issues as compiled dependencies don't just output the same thing every time, and I believe it's because of things like dependency features, dependencies sometimes not using mainstream version, and so on.
Let's say Lib A and B uses Lib C
But Lib A uses feature "Foo" of Lib C
And Lib B uses feature "Bar" of Lib C
In those cases AFAIK, the compiler would only compile the dependencies based on what they actually use. And having shared cache might trigger a recompile which is counter-intuitive to what the person who made the post would've wanted.
FWIW it generally works ok to just set CARGO_TARGET_DIR to a common location like ~/.cargo/target
The biggest problem is that it can cause increased rebuilds if you're switching between projects because you're probably going to have different features enabled in some nested packages
I'd generally not recommend it regardless, just for "I don't want to have to figure out if this was why if something goes wrong"
It works good, combined with CLI it becomes easy to add and manage the cache. I will add this Cargo backend to my CLI
The problems are :
A shared target directory suffers from lock contention. If you have multiple projects open or multiple instances of rust-analyzer running in the background on your editor, they will constantly block each other with Waiting for file lock messages. This can delay diagnostic info, but it is fine if you work at 1 project at a time
It also will be cleaned by cargo clean, because running clean in one project will inadvertently wipe out the compiled dependencies of every other project because they use same target folder. But you can avoid this one just never run cargo clean, but delete the target folder manually
It is almost impossible to prune specific cache for specific library only for specific feature. But it is not big deal, because I found this one manage sharing files automatically by Cargo. I found this one can have smaller disk size
Without CLI to provide nice usage interface, managing multiple same libraries but different version and features set can be verbose
Eg
Project A uses Tokio version 1.4.8 features full
Project B uses Tokio version 1.5.3 features io-uring
We have to manually edit the Cargo.toml and lib.rs with name alias and feature flag to enable each project can choose only use certain library with specific version and features set which is prone to error. With CLI we just rerun the same process that is proven to work, eg cli add tokio features=full, then the CLI is the one that update Cargo.toml and lib.rs of the shared cache folder, using same logic that is proven can update the info without error, aka it becomes simpler and determistic
While the different with my project is :
I haven't studied RUSTFLAGS yet, but I trust you.
I think a lot of the pain described in this thread may come from the same underlying issue: a shared target directory behaves like a build directory, not like a real cache.
That would explain several of the symptoms people mentioned: lock contention, cargo clean wiping everything, RUSTFLAGS changes causing rebuilds, and feature-set changes rebuilding dependencies. They look like separate problems, but they are all different ways of running into the same artifact-collision problem.
Cargo fingerprints artifacts using inputs like the resolved feature set, RUSTFLAGS, the rustc version, and the build profile. It also bakes a hash of that into -Cmetadata. So if two projects share one target/ directory but differ in any of those inputs, they can produce different artifacts that all land in the same shared build area. That is where the relocking, rebuilding, and general confusion come from.
A content-addressed cache is a better fit for that problem, because each fingerprint becomes its own key instead of competing for the same build directory. sccache is the off-the-shelf option here. For example, project A using one Tokio version and feature set, and project B using another, just become two separate cache entries. cargo clean also will not wipe the cache store.
There are a couple of caveats: sccache generally wants CARGO_INCREMENTAL=0, and it will not cache absolutely everything. Build scripts and proc-macro codegen can still be tricky.
That said, I do not think this makes your CLI idea unnecessary. It sounds like you are solving more than just caching. Custom compiler flags, linker or allocator swapping, and the fcomptime reflection workflow are all good reasons for a dedicated tool.
The cleanest mental model might be two layers:
That way the cache layer stops being something you have to manually manage by version or feature set.
One separate trick, if the main pain is a single heavy dependency dominating cold builds inside one project: put it behind a small facade crate and make that facade its own workspace member. Then it tends to compile once and stay out of the way during normal incremental rebuilds, because your day-to-day edits stop touching it. I have seen that take warm rebuilds from minutes down to seconds.