How would you create a proprietary library in Rust?

At work we've got an open-source library and would like to create proprietary functionality that builds on top of it. Ideally as another Rust crate which builds on to natural extension points provided by the open-source crate (new types implementing core traits, etc.).

Obviously this proprietary crate can't be published on crates.io because that'll give away intellectual property and I'd prefer not to provide a C API because you lose a lot of the niceties of Rust, so what other options are available to us?

Are there alternative registries which we can run that have an authentication component?

Cargo has some support for git authentication, so you could set up your own git server with one user account per customer / authorized user.

1 Like

See also: Pre-RFC: Cargo alternative registry authentication - cargo - Rust Internals

2 Likes

Yeah, that was my initial thought. We could probably wire something up using GitHub's API so whenever a customer pays for their subscription it gives their organisation read-only access to the repo.

That's an interesting path to go down. I haven't seen many people use alternative registries yet, but it seems like the way of the future for these sorts of things.

FWIW, you're in good (or at least notable) company there; this is how Epic handles source access to Unreal Engine. (Though there it's just signing a legal agreement with no payment involved.)

Alternatively, with a bit more effort, you could use abi_stable to provide the API ("headers") and distribute the implementation separately, like you might with a proprietary C library. I wouldn't recommend it if a different source distribution scheme satisfies your legal team, though.

1 Like

This may not be as common on crates.io, but it is important to check nonetheless: Do you know whether that open source crate has a copyleft license or a more permissive license? Any license with a copyleft (for example, GPL or MPL) will not allow your modifications to be private.

In this case we wrote the open-source crate and it is MIT/Apache-2.0 dual licensed so we should be fine on that front, but it's still a really important thing to check!

It's nice to know we're not the only ones to think of this approach!

Also, thanks for pointing Unreal Engine out because now I can check out their docs to see how they might have implemented it.

If you are happy to provide clients with source code, then providing access to a Git repo or private registry is a good option. An even simpler one is just to provide a source tarball download!

If you want to provide a compiled binary and no source, it becomes a lot more complicated.

I've only started doing some cursory research for something similar (providing a Rust API to a closed-source application), but from what I've found so far:

  1. If you can dictate the exact compilation environment (e.g "use this exact patch release of the Rust toolchain on this platform with version x of y, ..." or "must compile using this Docker container" etc), then you can use binaries straight from rustc. This is fairly common with C++ application SDKs (where you need to use very specific compilers to produce plugins), but probably wouldn't work so well for providing a library
  2. Providing a C shared library is the "safest" option - in that it's a very well known approach. Constricting the API through the lowest-common-denominator of C data types does lose a lot of the Rust niceties, but using things like cbindgen to create the shared library, and bindgen to generate the "client-side" bindings should help a bit (at the very least make it less tedious!). This has the advantage that you could also provide bindings in, say, C++ or Python with less effort
  3. The other "safe" approach might be to instead have the library be a executable and communicate with it via some IPC method (e.g communicate via stdio, or expose a gRPC interface, etc). Excellent for some particular cases, but somewhere between impractical and impossible for others
  4. It's possible that using Web Assembly as a platform-agnostic "binary" format might work for some cases, but you still have the same "funnel everything through restricted API" problem of the C bindings, and wasm is a lot more "experimental" than a .so/.dll. E.g something like providing a .wasm file and have the client side run it using wasmer is an interesting approach, but probably quite niche applicability

I found this article quite useful - it has a better explanation of most of the above points. The article is approaching things from the opposite direction as you, but the problem of providing a closed-source library is essentially identical.

1 Like

Yeah, binary-only distributions would be preferred, but they are somewhat of a second class citizen in the Rust world.

That's probably not going to any time soon (if ever) because generics require access to source code or you need to embed enough type data in the intermediate artifacts that you may as well just distribute the source :disappointed:

Ooh, that article references things I've written in not one, but two places :partying_face:

More people have tried writing Rust FFIs in the past, thankfully, so we can take a look at existing tutorials in order to see their experience:

  • The one and only Amos wrote an extremely detailed blog post on fasterthanlime here, specifically about live reloading Rust — a closely related topic.
  • Michael Bryan made a guided introduction to Plugins in Rust here, and also wrote a tutorial for his unofficial Rust FFI book here.
  • @zicklag, who had read Michael’s article, tried it by himself in order to add a plugin system to Amethyst, and posted this tutorial. When he shared the post on the official Rust forum, it was accompanied by this demotivating comment, after failing [14] to implement it for Amethyst: