On the issue of a Rust program depending on a static library generated by another Rust program

1

A Rust program depends on a static library libfoo.a, but this libfoo.a is actually also written and generated in Rust. When I use nm libfoo.a, I see that it exports all symbols, including those from libstd (e.g., std::*) and symbols from its own dependencies (e.g., tokio::*). I want to know what will happen when the Rust program links against this libfoo.a again. Will the symbols from std and tokio conflict? Or could this lead to undefined behavior (UB)?

2

If that..... the Rust version used to compile the executable differs from the version used to generate libfoo.a, or when there are mismatched tokio versions.

3

they will conflict if they are built with the exact same version of rust toolchain.

in general, the crate-type = "staticlib" is intended to be linked by programs written in other languages such as C/C++. for a rust application, you should use the default crate-type = "lib" (or `crate-type = "rlib" if you want to be explicit).

if you are using the same version of rust toolchain, the linker conflicts can be workaround by dynamically linking the standard library, but if you mix different versions, you'll almost certainly get some kind of UB, for instance, it is common for a library to return some heap allocated data, which are dropped in the caller. if there are multiple instances of the standard libraries, you are deallocating memory of a different global allocator, which is UB.

you'll very likely get a runtime panic when trying to spawn async tasks in the library because the main application entered a different version of tokio runtime.

so the safe solution is, put the generated code in a crate and use it as a regular cargo dependency.

1 Like
4

Thank you very much for your response. Actually, in my scenario, it's not a direct Rust program dependency on libfoo.a, but rather a chain of Rust → C++ → Rust (libfoo.a). This is precisely why I can't simply use crate-type = "rlib".

I'm wondering: when generating libfoo.a, could I use a linker script to forcibly specify only the functions I need (marked with #[no_mangle] and extern "C"), while hiding all other symbols like std::* and tokio::*? Would this approach effectively confine the Rust and Tokio functions within libfoo.a, preventing them from leaking into the broader linkage scope?

5

so I assume the C++ code is built as a library?

if it's a static library, you don't need to add libfoo.a to it, you should be able to just build the C++ library with a header file for libfoo.a. you only need libfoo.a when linking the final program.

if it's built into a shared library, then libfoo.a must be linked into it, and it would contain code from the rust standard library. in this case, you need to compile both libfoo.a and your main rust app with -C prefer-dynamic, which will dynamically link rust's standard library. this would eliminate the conflicts, but you do need to install a copy of libstd from the exact same version of rust toolchain along with your program, if you need to install it on a machine different from the build environment.

it may be possible, but I'm not sure.

libfoo.a is a static library, it doesn't "export" symbols, it's just an archive containing many object files. even if you can control the visibility of the symbols, you can only do so for shared libraries, such as the C++ code as a whole.

the unsafety is not caused by code duplication, but incorrect passing data between incompatible functions. as long as you use ffi data strictly as opaque types, you should be fine.

as an counter example, suppose the C++ code has an API to allow you get some pointer:

class FooManager;
class Foo;
/// factory function or constructor
extern "C" Foo *new_foo(FooManager *mgr, int i);
/// destructor
extern "C" void release_foo(Foo *foo);
/// stub/proxy for some method to make it safe at ffi boundary
extern "C" void foo_method_xxx(Foo *foo);

and it so happens that the returned pointer is a Box<libfoo::Foo>, and you want to take advantage of this knowlege, so that in the main application, you can call methods on foo directly instead of going through the C++ ffi stub, something like:

unsafe extern "C" {
    fn new_foo(mgr: *mut bindings::FooManager, i: i32) -> *mut bindings::Foo; 
}
fn main() {
    // ...
    let foo = new_foo(42);
    if !foo.is_null() {
        // this could be UB!!!
        let foo: Box<libfoo::Foo> = unsafe { Box::from_raw(foo.cast()) };
        foo.method_yyy();
    }
}

this code is unsound, it could potentially run into UB, e.g. if you have multiple copies of libstd hence multiple instances of global allocator, etc.

instead, you should treat the returned pointer *mut Foo as opaque, and only invoke its methods through exposed ffi API.

but this may not be enough. libfoo must be written in a way that is "self contained", meaning it should not make assumptions about global states of the program, for example, you cannot just assume the current thread is within a tokio runtime context, instead, you must explicitly manage your async runtimes.

6

Thank you very much for your reply—it was very helpful to me. To simplify the build process and address potential security concerns, I've decided to rewrite libfoo.a in C++ instead.

7

Static libraries are not really libraries, they're just a bunch of unlinked .o files bundled together.

If you end up bundling multiple copies of the same .o files from libstd, tokio, etc., then you will get duplicate symbols. It will end up causing similar problems to the ODR violation in C++.

The solutions are either:

  1. Build the C++ → Rust part as .so/.dll dynamic library, so you get Rust → solib(C++ → Rust) (unlike .a, this is an actual binary with a public interface), and threat it as an external non-Rust dependency for the rest of the Rust project. This is awkward to work with. It will properly link its own copies of libstd, tokio, so you will end up with multiple copies of the Rust deps in the final program, but they won't clash at the linking stage. Overall I don't recommend such approach, unless you really really need the libfoo.so to be a library that hides its dependencies.

  2. Make it Rust → C++ only (or Rust → Rust → C++). The .a files are not real libraries, so you don't need to treat them as libraries. They don't do any linking, so they don't actually have dependencies. They just have unresolved symbols that will end up resolved sometime later when the final binary is linked. This means you can build the C++ code separately without its Rust dependencies, and then link it into a larger Rust project.

I assume that you use a build system for C++, and it forces you into this Rust → C++ → Rust sandwich by trying to make a complete C++ product for C/C++ consumers. This is unfortunately a battle you have to have with your C++ build system to build C++'s .o files only, and give them to Cargo without bundling the Rust part in it. If the C++ part is simple, then try building it with the cc crate, using only cargo as your build system.

8

I understand what you mean.
Given the dependency chain:
MyRustApp -> libcpp.a -> librust.a

In reality, libcpp.a only needs the header files from librust.a, but compiling libcpp.a doesn't require the actual function definitions.
Therefore, I can structure it as:
MyRustApp -> libcpp.a (where libcpp.a only contains symbol references to std::* and tokio::*)
MyRustApp -> librust.rlib (cargo will handle both tokio and std dependencies for both MyRustApp and librust.rlib)

Is my understanding correct?

1 Like
9

yes, you are correct.

and as I said above, this works when your C++ code is built as a static library (libcpp.a), but not when as a shared library (libcpp.so).

the reason is, as @kornel explained, "static" libraries are just an archive of objects, generating a static library don't need a linker, namely, the references to symbols from librust is not resolved at the time when you build libcpp.a.

you just need the type and function (forward-)declarations from librust, typically in the form of header files, in order to compile the C++ source code. for simple cases, you don't even need a separate header file, you can just manually write out the declarations before using them in the C++ code, similar to how you use extern blocks in rust to declare ffi functions.

10

Yes, generally something like that. There are some details that need to be clarified:

C++ can't use tokio directly from C++ code, so typically C++ code will have references not directly to tokio, but to some custom Rust wrapper library that exports extern "C" functions for C++. Like extern "C" fn run_tokio_for_my_cpp() {…}, and C++ will have references to the run_tokio_for_my_cpp symbol only. The Rust/Cargo side will need to compile that Rust wrapper library that exports fn run_tokio_for_my_cpp, and have the C++ .a or .o linked together with it.

The .rlib format is private to Rust, so you wouldn't use that yourself directly. Typically Cargo does the linking automatically. You only need to tell it Rust/Cargo packages to use, and use build.rs Cargo build script to tell Cargo where the additional C++ libfoo.a is.