External crate symbol lookup

Disclaimer: I am a C++ dev. exploring Rust.

I would like to understand how Rust finds symbols from other crates.

In C++ when, I have a dependency to another library, then the conan package manager downloads the compiled library with its include files, so I know what functions (including the parameter) are available in the library and from the type declarations in the header file I also know the memory layout of classes and structs. Conan supports downloading a library with source code and compile it locally.

At the moment this step is not clear to me in Rust: I have made the tutorial with the rand crate, but there it was fetched as source and was compiled, when I was building the binary crate. I wonder if this is the only way in Rust or you can fetch a crate with compiled library (with a .dll.so) and have some metadata file that describes the exported symbols (like an include file). Or does the .dll/.so a "reflection" section where this info is available?

If only the build from sources option is available, how can you protect a proprietary library to be exposed as a source code?

Rust only really supports distributing fully-featured Rust code in source form. If you think about it, it only makes sense: it's not really possible to do anything else in the presence of generics. You need the source of generics, or at least a high-level IR representation (which is what Rust actually stores in .rlib files) in order to instantiate them with concrete types.

Furthermore, Rust makes no guarantees about ABI stability, which might vary between compiler releases. Thus, in order to create a reliable build, the whole binary must be compiled with the exact same compiler.

The exception is C FFI, of course. Compiling generic-free Rust code into a .so or .dylib and accessing it as if it were C is fully supported, but you'll have to mark such functions with extern "C" and #[no_mangle].

You can't, in general, unless you are willing to prepare FFI bindings and not expose any generics. Cargo won't know about them, though, so you'll have to pretend you've got a C library and write a build.rs build script, in which you instruct the build system to link against your proprietary library.

5 Likes

Even if you ship an rlib you will still leak a lot of information in the crate metadata. This includes but is not limited to everything crate documentation would show (including private items, type definitions, function signatures, doc comments), for every function that is generic or marked as #[inline] the full function body after desugaring (technically just the MIR, but it isn't too hard to construct human readable source code from that), the name of every source file together with a hash of the contents as well as the location of every newline and the source location (filename, line, column) of everything previously mentioned.

2 Likes

If you want to ship a closed source library I would recommend compiling it to a dynamic library exposing a C api and then give your users an open source rust binding to this pre-compiled library. Make sure that you are not compiling it as static library. That will cause symbol conflicts if you try to use it in a Rust program again. See Don't leak non-exported symbols from staticlibs · Issue #104707 · rust-lang/rust · GitHub.

3 Likes