I'm looking at getting a private crates.io registry up and running for my organisation[1]. I've so far spun up the crates.io application and a git repository to act as an index, and that seems to be functioning well.
Here's the issue: I'd like to be able to mirror crates from crates.io as well. Our existing work makes use of Artifactory[2], which proxies and caches artefacts from remote repositories (e.g. Maven Central).
Mirroring the git index for crate names and versions is fairly easy, but I'm looking for some guidance on how to handle getting the actual crates. Is the community likely to accept something into the crates.io code that fetches crates not found locally from another registry and caches them? Or can anyone suggest a better approach?
I've been meaning to make a caching mirror plugin for Artifactory for a while but haven't gotten around to it
There is some feature work on Cargo that would also be helpful for these kinds of situations to support the use of multiple registries. I think @alexcrichton has some ideas in his head about how this would work?
Currently there's no out-of-the-box support for private registries, but we have definitely designed room for it in Cargo!
The plan in Cargo is to support adding custom registries to a Cargo.toml, with the default being the current crates.io registry. Dependencies are then tagged which registry they come from so you can have crates from a private registry. Note that this feature is intended to be orthogonal to mirrors.
The main sticking point I haven't had time to think through is cross-registry dependencies. Presumably crates in a private registry need to depend on those from crates.io, yet I'm not entirely certain how to do this.
In the Java world, you just define multiple upstream repositories and they are simply checked in order. I don't think there's a way to specify that a dependency comes from a specific upstream - you just name things in a way that (hopefully) avoids conflicts.
Cargo coupled with crates.io, to me, works as package management systems. It seems new languages these days all decide to create their own, for some reason. This problem, as described here, has been resolved in many other package management systems already. Look at either apt or yum, for example. APT allows priorities between repositories, as well as the ability to say "package X from repo Y (my-custom-repo) should be pinned at prio Z which is higher prio then repo A (delivered by system)." Don't just look at how other fairly "new" languages have solved these types of problems, but how systems that have been around for decades have solved them. This is not a new problem. You may find bits in apt or yum which you can add to Cargo/crates.io and enhance them further. Just a thought (from a long time sysadmin/engineer).
