Rust behind a corporate firewall

I don't yet use Rust at my day-job, partially because I haven't found a really good use-case for it yet, but also because I've seen how our Python developers have had to configure their stuff and I'm sure I'd be asked to configure Cargo equivalently.

As I understand it, these are the requirements:

  • Building a project must never depend on external Internet access, so all required packages must be retrieved from a mirror within the corporate firewall.
  • If a project uses version X of package Y, the mirror should return an identical copy to every subsequent request, even if the "version X" tag is adjusted in the upstream repository.
  • Transparent mirroring: the mirror should act like all the upstream packages and versions are present locally, downloading them from the Internet on first use and caching them forever.
  • Non-transparent mirroring: Silently pretend that certain upstream packages do not exist (for example, hiding GPL-licensed libraries, or libraries not on a whitelist).
  • Internal developers should be able to publish internal-only packages to the internal mirror, without making them publically available.
  • If an internal package named "X" exists, any upstream package named "X" should be completely ignored - i.e. even if it's got a bigger version number, it must never be offered as an update.
  • A system-wide config file to point Cargo at the internal repository, so it can be done as part of the standard provisioning system, instead of every developer having to set it up manually.
  • Non-GitHub-based authentication (if it's purely append-only, it may need no authentication at all)

The section Replacing sources in the Cargo docs mentions some features that go a little way toward meeting some of these requirements, but not all of them.

I wanted to provide this list because up until now I've only seen discussion of simpler use-cases like automated offline builds, not the full internal-ecosystem use-case, so I figure a more detailed list might be helpful. Also, if there's any plans toward these kind of features, I'd love to know where I can find out more and maybe even volunteer some time, or at least be a sounding-board for ideas.

10 Likes

Virtually all of this is "needs someone to drive it forward and do the work", basically.

This bug report of mine is the best place I know of to see progress towards this. I'm definitely interested in working on this, and I've pushed on this a bit.

If anyone's interested in paying for work done along these lines, we'd be interested :slight_smile:

2 Likes

I cannot access your bug report so I'm answering here.

Apart from creating a private mirror, where can I find a list of all the necessary urls (plus backup?) we need to use rust in enterprise.

Something like github, crates.io (plus the redirected urls?), static.rust-lang.org for rustup etc ... ?

Does it even exist?

I'm interested in this topic as well (I posted a similar question/topic a few months back, but didn't get any replies IIRC).

I suspect having a decent story here would help substantially in Rust adoption.

Mirroring has been done before. There's no turnkey solution for it, yet, but it does work.

Yes I know. What I am looking for, before going down that road, is simply a list of urls we can ask exceptions for in our firewall. This may probably be enough most of the time.