How can I share my lib?

help
1

How can I share my the crate(lib.rs) but with not exposing my source code? Others can use the crate's pub functions or other things jut like use the normal crate.

2

Note that if you don't expose the source code, the created library will be platform dependent. That said, I'm also curious about how binary distribution with Rust works. I saw that cargo generates an *.rlib file, but there isn't such a thing like header files in Rust, is it?

1 Like
3

Related SO question:

I don't know your concrete use-case, but I use a private registry for my libs that I don't want to share on crates.io. Source code is still shipped as part of the package though, so this may not be an option for you.

1 Like
4

From that link:

While the metadata is binary, not JSON, using librustc to extract all exported function definitions from a compiled crate, and pretty-printing the ASTs is fairly easy.

What is librustc? I don't see such a crate on crates.io, and I also don't have such a binary on my system.

I see the cited post is 8 years old; maybe things have changed since then?

5

Oh my, you are right, the post is probably outdated, my bad. It even predates the Rust v1.0.0 release.

6

I guess it's a lib collection related to rustc. Like all crates in Index of crates with their names starting with rustc.

1 Like
7

Easy. That:

Is very explicitly not supported mode.

Not only that but there are also no guarantee that *.rlib files would be usable not just with other Rust compiler but with rust compiler used with different set of options.

IOW: the ability to share binary versions of crates is not supported in any capacity, shape or form.

Challenges are both technical and political: Rust developers don't want to see this (at least not at this time) and thus what topicstarter wants is very much impossible. And not in a sense “it's hard to do” but more in a sense of “don't try to do that, or you'll be sorry”.

Your best bet is to provide a shared library (.dll/.dylib/.so) with C interface with, optionally, Rust-like wrapper (in source form) for more ergonomic use.

6 Likes
8

I'm curious; is there some official documentation on binary libraries being "explicitly not supported"? (for the static and dynamic case) I remember reading something about dynamic linking being discouraged in general, but I don't remember where I read it.

So basically interoperability of the *.rlib files is very limited? So those files mostly serve "caching" purposes (to not recompile everything over and over)?

I think updates to the ABI or the library format aren't making it "impossible" to distribute binary libraries. Of course, if there are breaking changes, you would have to distribute a new version of your library. Same holds if you want to use the library on a different platform. There are a couple of reasons not to ship binary libraries, but I don't see a reason yet why it shouldn't be possible. Is the library format that unstable/volatile? What sort of options affect the *.rlib file?

Luckily I don't have any motivation or need at the moment to ship binary program code; I pretty much like open source (for a variety of reasons), but I'm curious nonetheless. It would mean that Rust can't do something which C can do (of course it's way easier for C to achieve this, not judging here, but I think it's noteworthy).

9

The primary problem with stabilizing any kind of ABI is you immediately run into problems with generics. IIRC rlibs currently just include the MIR generated by generic items, which is then compiled for each instantiation of the item in each crate that uses it.

This post about Swift's ABI stabilization goes into some of the details of why this is more complicated than it might seem at first glance.

2 Likes
10

Here's the exceptio probat regulam in casibus non exceptis proof. And people would introduce a way to break crates which depend on structs layout if they would have been believing in stable ABI.

It's not discouraged, but you have to use stable #reprs (mostly #[repr(c)]). They just want you to make binaries which expose Rust ABIs (and static libraries do that automatically which means you need dynamic libraries for now).

Yes. Usually cargo recompiles things automatically when needed. And you can use Rust types across the shared library border — as long as you use the same compiler with the same options for everything… but that's incompatible with goals of topicstarter either, because popular crates drop support for old versions of compiler pretty quickly.

It's not the library format, it's data structures format. Rust doesn't give any promises about compatibility of data structures between two different versions of Rust (even 1.67.1 may have different data structures from 1.67.0, e.g.) and they leak from static libraries thus you need to ensure that you are not using things like Option or Result in your API which you provide in your shared library.

And without Option or Result you, pretty much, couldn't provide anything resembling normal Rust API.

Note that C-on-POSIX very much an exception, very few languages actually give you the ability to ship and link separate static libraries produced by different versions of the compiler.

Not even libraries compiled for different versions of C compilers (on platforms where there are no one, single, C compiler, e.g. Windows) are compatible.

Indeed. And I really hope at some point Rust would do the same. But it's enormous undertaking, not even technical one, but more of a political one… not sure how many years may it take.

4 Likes
11

I'm not sure about documentation, but read the conversations at e.g.

The ABI is effectively that volatile; as per those conversations there isn't even a definition of what constitutes the ABI. But if you can guarantee the same version of the compiler, you could compile an rlib AFAIK. Here's what goes into them. rustc won't link rlibs from a different compiler version, so you'd have to ship a new version every 6 weeks or less.

Also, the rlib format is meant for consumption by rustc, not linkers more generally. It's considered an implementation detail, and can change without notice. I'm not sure how often that is in practice, but it doesn't matter for the OP really since you can only link rlibs compiled with the same compiler. It could change every rustc version and their situation would be the same. (There's a pre-RFC linked at the end for something external linkers can use, but that's about rlib format compatibility, not ABI compatibility, so you'd still need the same rustc version etc.)

And this all may be moot for the OP even if they were willing to ship that often, as the rlib could contain the MIR and so on.

Sure, but read about the ramifications from @khimru's link. Making this possible without crippling the language or stdlib in some other way will be a challenge (especially with Rust being so young, DSTs and generators and robust async and dyn still being iterated upon, etc). Consider this relatively recent change for example.

Also, C can do other things Rust has already decided to never support, like 9-bit bytes. I'm not saying this is or should be one of them, but pointing out that "well in C you can..." isn't an automatic :+1: for Rust to support it. Most langauges have no stable ABI; that's why we all drop down to "what C does on this platform".

IMO Rust should at least do it differently, if it does it.

3 Likes
12

Rust can already do exactly as well as C here, by exporting a C API -- just repr(C) all the types on the boundary, and extern "C" all the functions.

If you only have a black-and-white printer, it doesn't mean that a colour image editor "can't do something" that a black-and-white-only image editor can't do, just because it also supports colour (and defaults to colour).

5 Likes
13

I was going to make this same point, but really, they almost surely meant "same capabilities but in Rust", similar to how the OP said "[just like they would] use the normal crate".

Incidentally,[1] the interop PR I linked is also (probably) not what they're talking about, as that's like having a C API only covering more of Rust, but still not a stable Rust ABI. However, there is a lot of overlapping/conflicting interests [2] and like tmandry says, they feed into each other. If we get interop there will be pressure to stabilize more and more std types as repr(interop) I imagine.[3]

It would make "just ship it with an FFI interface" more palatable (but is still more FFI than native IMO).

  1. directed at anyone following along and not Scott in particular ↩︎

  2. and thus ABI conversation in the thread, which is why I linked it ↩︎

  3. e.g. the mutex example previously cited ↩︎

14

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.