PLT symbols with `cxx-build`

help
1

Hey folks, sorry if this is the wrong place to post this, but I can't really distinguish whether this problem is related to cxx, rustc, rust-lld, or gcc, static linking or something else entirely.

I'm working on a project where I'm writing a plugin for a C++ program in Rust. The program contains some header files which defines a special inline function (call it register), which is automatically called from a class constructor for a global instance instantiated in the plugin source code. For a usual C++ project, when one compiles their shared library, this function is generated (the definition is in the header file), but is marked a "relocation" and the symbol exists in the .got.plt table for the shared library, as can be seen through readelf -r. This symbol is then replaced with the actual one in the "host" program after the library is loaded.

Now, the program I'm writing is a Rust cdylib alongside a C++ shim that pretends to be a plugin as above, which essentially calls the functions I wrote in Rust, with a build.rs script that compiles the C++ through cxxbridge and builds the final shared library. The build in build.rs is more or less as follows:

    let mut build = cxx_build::bridge("src/lib.rs");

    let build = build
        .files(cpp_source_files)
        .link_lib_modifier("+whole-archive");

    build.compile("plugin-bridge");

The problem is that none of the symbols in this plugin-bridge are exported unless built with +whole-archive, and more relevant for this post, the aforementioned inline function in the header file is generated and placed in .text, and does not show up in .got.plt as is the case with C++ above.

I understand that the root cause for this probably is cc, which only builds static libraries, though I have no idea how I can go about fixing this. It is technically possible to compile the C++ shim separately as a shared library (which I haven't tried yet) and link with the Rust program, but I'd like to obtain a single shared library which also mentions register@plt if possible. I've already tried a few things, notably passing -fPIC to cc::Build and passing relro-level = "off" to .cargo/config.toml to no avail.

Is this even possible to do through Cargo? Any input would be appreciated.

2

I'm confused, how does an inline function get an entry in the .got.plt section, which is for dynamically linked functions?

can you show the signature of this "inline" function, and maybe some example output from the readelf command too?

3

The C++ function in question has signature

inline auto Factory::register(const Factory& factory) -> void

and the call to readelf -W -C -r produces

Relocation section '.rela.plt' at offset 0x50c0 contains 92 entries:
[...]
000000000000b230  0000003b00000007 R_X86_64_JUMP_SLOT     000000000000687e Factory::register(Factory const&)

As to how this happens, I don't really have a good guess. From what I understand, the C++ keyword inline generates weak symbols, and this might be related to the issue at hand, but I don't know.

The only reason I suspect this causes the issue is by A/B testing with a plugin compiled in the C++ way and inspecting symbols with gdb, where the exact same function symbol is plt in the status quo and not so in the Rust plugin I've built.

4

Does it work better if you turn the build around? That is, have cargo build a staticlib (not a cdylib), and have the C++ build system for the plugin link the static library that cargo creates into the final plugin, along with the C++ build artifacts.

I'm guessing that cc has built the C++ code as a static library (not a shared library), and hence you don't have the weak symbol you need to make this work. I would also guess that if you built the plugin registration code in Rust (and had cxx export the necessary symbols for this to work), that it would work, but I'm not sure how to do that - or indeed whether it's practical to do that.

5

yes, but that's should be an weak export symbol, intended for the "backup" body of the inline function, since the compiler is allowed to ignore the inline specifier.

nevertheless, even if the call to the function is not inlined but a regular function call is emitted, it should reference the local label, which needs no relocation fixups.

unless...

this function declaration (in the header file):

has only the prototype declaration without a function body definition.

but if that's the case, I think you should get a warning, like "inline function used but not defined", or something similar, and the compiler would generate a function call as if it was a "regular" (non-inline) function.

in this case though, in the rust side, you should be able to import the function as a normal (ffi) function, and just ignore the inline specifier.

on the other hand, if it does have a body, then the calling code (in C++, of course) should not reference an external symbol at all, whether the function gets inlined or not. that's what I'm confused about.

btw, what optimization level do you set when compiling the C++ code? I believe the optimization level can affect the compiler's decision whether to inline a function, so that's also a possible reason to explain the difference.

6

Thanks, these are both great suggestions. I don't think the first one particularly fits my use-case, though it's good as a short-term solution to see if it will work at all.

For your second suggestion: I think I would need to fake the function, as well as the derived class Factory, though I'm unsure how I can do so in Rust in a way that would satisfy the function signature emitted by C++. Does it suffice to get the memory layout of the type correct (with a struct Factory { ... } for example?)

7

It indeed has a function body provided in the header file.

As for the optimization level, I am unsure what the exact level is for the release builds, but I can say that both the C++ release and debug builds (which do not set any optimization flags) exhibit the same behavior of generating function symbols; indeed, this behavior of emitting a function definition in .text and using the definition obtained from .got.plt seems to be intended behavior.

8

As an update, I've managed to get the build system to emit the desired symbols for both debug and release builds by defining the function as extern and re-exporting it through cxxbridge:

#[link(name = "<library-where-the-function-is>", kind = "dylib")]
unsafe extern "C" {
    #[link_name = "<mangled-function-symbol>"]
    pub unsafe fn emit_register(reference: *const c_char);
}

#[cxx::bridge]
pub mod ffi {
    extern "Rust" {
        pub unsafe fn emit_register(reference: *const c_char);
    }
}

I also removed the link_lib_modifier("+whole_archive") directive in the hopes that doing so would lead the linker to use the dynamic library as a source for the symbol, though this did not work and the symbol is present only in .text.

Although very ironically, the throwaway symbol emit_register is now in the relocation section:

Relocation section '.rela.plt' at offset 0x42f470 contains 326 entries:
[...]
0000000000c58458  000012e300000007 R_X86_64_JUMP_SLOT     00000000005d7760 cxxbridge1$200$emit_register + 0
9

After a bit more tinkering, I'm more convinced that this is some specific behavior either induced by accident or through some deliberate tinkering with symbol visibility during compilation. In either case, I think the only realistic option here would be to implement the plugin registration mechanism in Rust, and just specify extern "C" { fn register(...); }. I won't mark this as solved just yet, at least while implementing it, though I doubt I'll come across anything that will change my approach.