Margo — a simple Cargo registry using static files

announcements
1

Margo is a minimal alternate Cargo registry comprised of static files. This allows the user to have full control over how the registry is hosted. You can place the registry behind a well-known web server such as nginx or host it on something easy to set up like GitHub Pages. We provide a GitHub action to add a crate to the registry as part of your CI/CD process.

Hosting your own registry is an ideal alternative to using Git dependencies as Cargo is able to unify semver-compatible versions. This is especially useful in corporate settings where you may have a large number of internal dependencies across a large number of consumers.

Hosting your own registry can also be useful if you are unsatisfied with how crates.io operates:

  • Is the crate name you really wanted taken? Host your own registry and pick any name you want — the registry is your namespace!

  • Want to store gigabytes of asset files inside your crate? Host your own registry and use as much disk space as you want!

  • Want to have a crate with a million features? Host your own registry and revel in the combinatorial explosion of choices!

26 Likes
2

great work.

I think an entry can be added to this list to help people discover it:

7 Likes
3

Is it possible to require authentication against the registry?

Not really related to margo, but: Client certificate requirements could be set up on the nginx server, but is there a way to tell cargo to use a certain key/cert when connecting to a specific server?

Is there any "registry authentication" user/password support in cargo/margo?

4

There is! We even have a test ensuring that HTTP basic authentication works. You should be able to use any authentication scheme supported by Cargo.

This aspect was a big driver in Margo's design. Instead of having to make Margo itself infinitely customizable with respect to authentication, we can rely on what existing tools support. If you have some crazily-complicated enterprise authentication going on, Margo should basically not need to care, so long as Cargo itself understands how to communicate with the HTTP server.

One downside is that there's no way to have a private GitHub Pages site, so if you wanted to have authentication, you'll need to investigate some alternate hosting provider.

5 Likes
5

Edit: Ah, I think I get it. The server must actually implement the Credential Provider Protocol. Right?

Edit 2: Strike that. I have no idea how this is supposed to work. :slight_smile:

Are the credential-provider's only supposed to store tokens?

I've been playing around with setting up a local registry, but I realize I don't understand how the pieces are meant to fit together.

On running margo init, it asks if the registry should require authentication. This seems to set auth_required (in margo-config.toml) and auth-required (in config.json) to true.

I set up a simple Rocket app, which uses a static share to share the registry directory. I can access index.html, which lists the packages I've added.

Trying to cargo add a repo from this registry fails, because it reports:

error: no token found for `my-reg`, please run `cargo login --registry my-reg`

As far as I can tell, I'm not supposed to be using tokens. But this leads me to wonder how exactly these things are supposed to work? My presumption was that my http server should be requiring basic authentication headers, and that it should be possible to get cargo to add them.

In my ~/.cargo/config.toml I have:

[registries.my-reg]
index = "sparse+http://127.0.0.1:8000/"
credential-provider = ["cargo:macos-keychain"]

But at this point I'm stumped. Have I understood the basic gist of it? Is cargo supposed to pass along username/password from the system's keychain via the http basic authentication header? Is there a way I can create this entry in the keychain using cargo? If not, how do I find out what it should be identified as?

6

I think "token" is overloaded. For example, in the test, we create the HTTP Basic credentials:

let creds = format!("{username}:{password}");
let mut token = String::from("Basic ");
base64::engine::general_purpose::STANDARD.encode_string(creds, &mut token);

and set them as a token (via an environment variable because it's easier for the tests):

if let Some((name, credentials)) = self.credentials {
    let name = format!("CARGO_REGISTRIES_{name}_TOKEN");
    cmd.env(name, credentials);
}

So in this case you take the HTTP Basic username and password, base64 them, add the "Basic" prefix and that's your token. I have no idea how you get that token into your credential provider and then back out to Cargo, but theoretically that's documented by the credential provider and/or Cargo.

7

It would appear that cargo will read the env variable

See also registries.<name>.token, and the cargo login cmd

8

I don't think I'm totally following you. @blonk is trying to do two things: get HTTP basic authentication working and provide that authentication via the macOS keychain credential provider. My post shows how to construct the token for HTTP basic authentication, and I state that I don't know how to store it in the credential provider.

The environment variable you link to is the one that I showed in my post, but Cargo reads that, skipping the credential provider completely.

registries.<name>.token is configuration for the cargo:token credential provider, not the macOS keychain, so I don't see the connection.

cargo --login might work. The docs say

If a registry has a credential-provider specified, it will be used.

But it also says

All the arguments following the two dashes (-- ) are passed to the credential provider.

So it's possible that you might have to pass arbitrary other arguments for the macOS keychain.

I tried this, but it doesn't work:

% cargo login --registry margo 'Basic: BASE64'
error: registry `margo` does not support API commands

This feels like an vestigial limitation on Cargo's side, as it doesn't matter that it doesn't support API commands because authentication is required to just download from the registry. I think someone should look for and/or file an issue about that.

The docs for cargo:macos-keychain do suggest using the Keychain Access app, which might be another way to store the token to start with, but I don't know what parameters to use.

1 Like