How to export a C symbol from a native library?

Hi everyone!

I’m using cbindgen/SWIG and the cc crate to ultimately use a Rust library I’m building from C#. This requires including the contents of a single .o file into the resulting dynamic library, but I’m unable to make that happen: the required symbols never show up in the output generated by rustc.

For a static library I can use +whole-archive, for example like so

#[link(name = "swig_wrap", kind = "static", modifiers = "+whole-archive")]
extern "C" {}

and nm target/debug/libmycrate.a shows me all the desired symbols.

How can I achieve this for a dynamic library? Even mentioning symbols in the extern block does not get them included in target/debug/libmycrate.so. (I did check that using the wrong name yields a linker error, so the link attribute does something).

Thanks a lot,

Roland

Did you define the functions/symbols/items as public?

yes :slight_smile:

FWIW, the following proves that what I want should be possible:

gcc -shared -o x.so -Wl,--whole-archive target/debug/libmycrate.a -Wl,--no-whole-archive
nm x.so
# now shows all those nice symbols

I found another strange behaviour: I can make a symbol show up in the dynamic library by referencing it from a pub fn, but only if that function has attribute #[no_mangle].

I believe if you include all functions you want to export in the extern "C" {} block (and make them public I think) that they will be exported. Otherwise rustc doesn't know that those functions exist in the first place and as such doesn't add them to the symbol export list.

Edit: are you writing a cdylib? in that case my suggestion doesn't work I think. You need #[no_mangle] on a function calling those functions to be exported. Without #[no_mangle] rustc won't export public rust functions as they can't be used anyway for cdylibs due to the lack of crate metadata compared to rust dylibs.

1 Like

Thanks for the confirmation! Meanwhile I dug deeper and found the precise mechanism that thwarts all my efforts: rustc uses cc -Wl,--version-file=... to install a whitelist for exporting symbols, and it seems that no following cmdline options can change ld’s mind afterwards — I tried replacing the whole linker script with -C link-args=-Wl,-T,myscript or adding a new version script in the same way, no success.

This is somewhat annoying, I think it should be possible to modify the whitelist without having to declare an auxiliary symbol that is only needed to “use” the symbols I want to export — it would be much more robust to be able to say “just add this .o to the link and export all public symbols from it”. Do you think it makes sense to open a discussion on IRLO or a github ticket?

An extern block just declares that some functions are available somewhere. When you add the #[link(...)] attribute it tells the Rust compiler that your crate will need to statically link against the swig_wrap library, however the linker is still free to throw out any code that is never used (--gc-sections).

When defining your own cdylib, you should be able to pub use the foreign symbols from your crate root to make them available in the end binary.

For example, I made this experiment using a random static library I have on my computer (ZeroMQ).

$ rustc --version --verbose
rustc 1.64.0-nightly (d5e7f4782 2022-07-16)
binary: rustc
commit-hash: d5e7f4782e4b699728d0a08200ecd1a54d56a85d
commit-date: 2022-07-16
host: x86_64-unknown-linux-gnu
release: 1.64.0-nightly
LLVM version: 14.0.6
# Cargo.toml
[package]
name = "repro"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib", "staticlib"]
// build.rs
fn main() {
    // rustc didn't seem to add this by default for some reason
    println!("cargo:rustc-link-search=/lib");
    // other dependencies
    println!("cargo:rustc-link-lib=stdc++");
    println!("cargo:rustc-link-lib=sodium");
    println!("cargo:rustc-link-lib=pgm");
}
// src/lib.rs
use std::os::raw::{c_int, c_void};

#[link(name = "zmq", kind = "static", modifiers = "+whole-archive")]
extern "C" {
    pub fn zmq_socket(_: *mut c_void, ty: c_int) -> *mut c_void;
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn smoke_test() {
        unsafe {
            let socket = zmq_socket(std::ptr::null_mut(), 0);
            assert!(socket.is_null());
        }
    }
}

When I compile the library, the zmq_socket() function is included in the final librepro.a.

$ cargo build --release
$ objdump -d target/release/librepro.a
     ...
0000000000000430 <zmq_socket>:
     430:       41 54                   push   %r12
     432:       55                      push   %rbp
     433:       48 83 ec 08             sub    $0x8,%rsp
     437:       48 85 ff                test   %rdi,%rdi
     43a:       74 24                   je     460 <zmq_socket+0x30>
     ...
1 Like

Thanks for your help! As I said above everything works fine when building a static library — does the symbol also get exported when you look into librepro.so?

Oh oops, guess that's what happens when you don't read the original post properly :sweat_smile:

Neither the debug or release versions of my librepro.so contain the zmq_socket symbol so I'd say this is a bug in the way modifiers = "+whole-archive" is implemented.

I did manage to find some zmq symbols though, so maybe the functions are still present but not exported?

$ nm target/debug/librepro.so | grep zmq
0000000000001020 t _GLOBAL__sub_I__ZN3zmq5ctx_tC2Ev
0000000000004010 b _ZN3zmq5ctx_t13max_socket_idE

Further investigation shows that I can make the symbols show up in the dynamic library like so:

  • lib.rs: pub mod other_symbols;
  • generate other_symbols.rs to contain
    extern "C" {
        pub fn SomeSymbol();
        ...
    }
    #[no_mangle] pub unsafe fn __dummy() {
        SomeSymbol();
        ...
    }
    

But unfortunately all SomeSymbols are only private!

> nm target/debug/libmycrate.so
...
000000000012ba2f t SomeSymbol

In order to access this symbol from C# I need the t to be a T — how can I do that?

How many symbols do you need access to? If there are only a handful, it might be possible to create wrappers that export the symbols you need.

For example:

extern "C" {
    pub fn SomeSymbol(a: *const c_char) -> c_int;
    ...
}
#[no_mangle] pub unsafe fn mySomeSymbol(a: *const c_char) -> c_int {
    SomeSymbol()
}

The wrapper writing could probably be automated using bindgen and syn and kept up to date with a test, but you get the gist. This falls firmly under the "hacky workaround" umbrella, but could work until a proper solution is merged upstream.

Thanks for the suggestion! I just opened public extern "C" symbols are not public in cdylib target · Issue #99411 · rust-lang/rust · GitHub, in which I also explain that this wrapper approach doesn’t work, since the name exported from the .so needs to be the exact name that I get from the C library.

Meanwhile, I’ll look into solving this problem outside rust/cargo — but that’s a shame since I’ll have to reinvent quite a bit of cross-building infrastructure for that (we’re targeting about a dozen platforms, including Windows).

Is it feasible to distribute the original library as its own DLL alongside your Rust one?