In light of the arrayref incident, it's probably a good idea to try to think about dependency security differently.
It's often complained that this is due to missing feature in cargo, for example, crate namespaces. But we actually kind of already have everything we need: custom registry. And a design based on that already works mostly hassle free.
The core idea is: use custom registries to represent trust boundaries.
The default crates-io registry is disabled. And everything works on custom registries.
One curated stdlib custom registry: commonly used stuff like serde, tokio, etc.
Special purpose custom registries, or per-team registries.
This design already works and do not have ergonomics issue:
# <project>/.cargo/config.toml
# This gets loaded automatically by cargo.
# Disable crates-io
[source.crates-io]
replace-with = "crates-io-disabled"
# vendor/crates-io-disabled is an empty folder
[source.crates-io-disabled]
directory = "vendor/crates-io-disabled"
# Define all registries. This is the trust boundary.
[registries.stdlib]
index = "sparse+https://somewhere-stdlib.org/cargo/index/"
[registries.a-large-web-framework]
index = "sparse+https://somewhere-web-framework.org/cargo/index/"
[registries.other-downstream]
index = "..."
[registries.another-dev-user-trust]
index = "..."
In Cargo.toml dependency, it becomes:
# Cargo.toml file dependencies section.
tokio = { version = "1.0", registry = "stdlib" }
something-else = { version = "x.x", registry = "other-downstream" }
Cargo.lock file already properly resolves to the full registry url. So this actually should already work (at most small custom registry implementation changes that do not require RFCs).
The main advantages of this are:
Like go.mod, it forces users to consider the entire transitive trust graph -- all registries (as trust boundaries) must be defined. No surprise new trust boundary.
We get Rust's own stdlib -- a vetted, commonly used package set as a custom registry. Maintainable, as this can be kept intentionally small and well audited.
It's the namespace system many people wanted -- each team is its own custom registry, decentralized. No need for any unnecessary coordination.
The transition to custom registry is also doable:
A curated stdlib. And that only needs figuring out the core package set + republishing to custom registry.
The system then becomes immediately usable for those who want to practice minimal dependency philosophy.
Other teams then join by hosting their own custom registries.
I don't see how a custom registry per se prevents compromised 3rd party packages. Does it make a difference from where I pull the compromised library? If my team needs arrayref and puts it in the custom registry, Cargo will pull it, extract it and run the malicious build script on my computer, same as if I had pulled it from crates.io directly. I'm no expert, but to me trust boundaries and manual curation don't sound like very effective[1] measurements against the growing issue of software supply chain attacks.
Because it would not have been possible to put in the malicious arrayref in the first place. Think about publishing to stdlib as submitting a new PR, etc.
Multiple custom registries are what allow each registry to be curated. And then you can define your own trust boundaries on top.
Of course, the stdlib's team credential can also be stolen, but both npm and Rust showed that this is not what commonly happens. What commonly happens in supply chain attack is an individual who is usually independent/outside but maintains an important yet small crate, and that gets compromised.
And that's why security boundary is important and how the stdlib design reduces the number of security boudnaries needed -- arrayref update would have been a normal PR before its publishing.
Previously there's another security boundary just with the independent developer. Now that is removed. One less.
Common Rust projects have 300+ 400+ deps and a large number of them are cases like this. And a curated core set reduces the trust boundary by a lot.
I have idea to put fast security AI in between the crate upload, like Google's Magika that protect Gmail and Drive from mallicious file, but in this case is code instead of file
So the idea is
First, extract code different of the old and new version, kind like how Github can differ new and old code, but only take the new code
Send it to the AI to be analyzed
If the analysis result is it is mallicious code, reject the upload
If there is safe code that is rejected (false positive), send review request, show the list of review request in crate.io website in different page url, the whole community can contribute to the review request, to be approved requiring at least 5 acceptance of review + 1 final acceptance from the security team (so the team will look up fewer lists, because it is filtered out by the community first). If it is accepted by 1 security team it doesn't need the 5 community acceptance
I'm not convinced. Are you saying you want to personally vet every single version of all your 300+ 400+ (transitive) dependencies, to put them in your "trusted" custom registry? I can't see how this would prevent vulnerabilities in a meaningful way (I personally would probably miss 100% of any potential vulnerabilities). The merits and shortcomings of a "web-of-trust" approach to ecosystem curation are also discussed in the topic I linked above (in the footnote; posting it again for better visibility):
Are you using "trust boundaries" and "security boundaries" interchangeably or are these two different concepts?
The elegance of the custom registry design is that you do not need to wait for acceptance of your proposal. We can have multiple stdlib registries, "competing":
One is nixpkgs style manual PR and merge.
Another auto AI review like you envisioned.
Some other types with more rigid, smaller core.
Then people get to choose their trust boundary. At least in my mental model, the security risks would be much more understandable -- instead of tap into a global crates-io where there may or may not be malicious packages, I choose a registry that I can place higher trusts.
Yes I'm using those two concepts interchangably. Non-native speaker, but to put it concretely:
You check one registry, give it a number -- risk/percentage this registry may get hacked.
Each registry has one number.
The reason this is a lot more manageable is because there are simply a lot less numbers. Think single digit. Right now, an ordinary Rust project has 300+ 400+ dependencies all tangled together. So each time when you add a dep, you must analyze it separately and completely, 300+ 400+ times, every single user. Or we just blindly trust all crates on crates.io and do nothing, which is what majority of users do.
Yes. This is what right now everyone has to do anyway if they ever want to gain any notion of security. Or we wait a negative from security researchers, but that's fragile.
This is entirely doable -- we're not checking whether the code is absolutely secure. We're only checking whether the code is malicious. Nixpkgs manage 100k+ packages this way. Julia package manager is actually also designed this way (PR based, but last time I used it was 5+ years ago, so take this with a grain of salt). And they manage 10k+ packages.
You can think of it as a more organized cargo vet, with proper granularity -- place the trust boundary on a curation / team, instead of individual crates. Instead of 300+ 400+, each user then only needs to check 5-10. The magnitude reduced number is why this works.