Multiple c++ libs and one C binding : undefined references

Hi,

I have a lot of C++ static libraries that have to be called from Rust now.

Compiled recently using Clang 17, on Linux, for Linux only usage.

Luckily, there is only one entry point, for which I created a C binding.
It means that the other libraries are all invoked from this entry point. I don't call those other functions in Rust.

On a local test environment with a very simple test case (one c++ static lib with a c binding included), it worked.

Now, the issue is that I with all my static libs, I end up with many undefined references.

I also have the search path redefined before hand in the build.rs

    println!("cargo:rustc-link-search=/my/path/to/statics/libs/");
    println!("cargo:rustc-link-search=/usr/lib/");
    println!("cargo:rustc-link-search=/usr/lib/x86_64-linux-gnu/");
    println!("cargo:rustc-link-search=/usr/local/include/");
    println!("cargo:rustc-link-search=/usr/include/x86_64-linux-gnu/");
    ...
    println!("cargo:rustc-link-lib=static=some_lib");
    println!("cargo:rustc-link-lib=static=opencv_core");
    println!("cargo:rustc-link-lib=static=gmp");
    ...

All the libs belong to those paths.

The entry point in my main.rs, regarding the extern "C":

extern "C" {

    fn BINDING_call_my_entry_point(
        ptr: *const u8,
        size: u32,
        ptr_out: *mut u8,
        max_length_in_exact_length_out: *mut u32,
    );

}

in the main function:

    fn main() {
    ...
    unsafe {
        BINDING_call_my_entry_point(
            body.as_ptr(),
            body.len() as u32,
            v.as_mut_ptr(),
            &mut len_v,
        );
    };
    ...

I took the same order of compilation than my already existing CMake (needless to say it works in a pure c++ environment).

I tried to link cc to clang-17, just in case, same result.

The issue seems to be that the linking process is pure C and I am linking some libs which are C++ without extern inside, except for the one with my entry point. Name mangling playing its role here.
I can't write extern everywhere otherwise the c++ part cannot compile anymore.
I can't rewrite everything in Rust either.

The alternative I immediately but which suck performance wise is to create a binary and call it from rust.
I don't want that.

Cxx-rs deals with source code. Mine is already in a static lib.

So I need to link using a linker that accepts C++ for the generation and present my extern "C" entry point to rust as pure C code.

How to do that ?

what are the undefined references? you didn't show any of them, so it's hard to guess what's actually missing. maybe it's the C++ runtime. did you include libcxx ? (or maybe it's called libc++, I don't have experience with clang toolchain. for gcc, it's libstdc++)

Sure!

One example:

One from my custom library (modified the paths and the formatting, but the rest is the same)

/usr/bin/ld: my_path/structural.cpp:192:(.text+0x9c4): undefined reference to 
`P1::Generic::Common::Mapper::set_text(P1::F_A&, nlohmann::json_abi_v3_11_2::basic_json<std::map, std::vector, s
        td::__cxx11::basic_string<char, std::char_traits<char>, 
        std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::json_abi_v3_11_2::adl_serializer, 
        std::vector<unsigned char, std::allocator<unsigned char> > >&, nlohmann::json_abi_v3_11_2::basic_json<std::map, std::vector, 
        std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, 
        nlohmann::json_abi_v3_11_2::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >&, std::vector<unsigned int, 
        std::allocator<unsigned int> >&, nlohmann::json_abi_v3_11_2::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, 
        std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::json_abi_v3_11_2::adl_serializer, std::vector<unsigned char, 
        std::allocator<unsigned char> > >&, nlohmann::json_abi_v3_11_2::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, 
        std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::json_abi_v3_11_2::adl_serializer, std::vector<unsigned char, 
        std::allocator<unsigned char> > >&, std::map<P1::Accessory_type, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, 
        P1::Generic::P, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string
        <char, std::char_traits<char>, std::allocator<char> > const, P1::Generic::P> > >, std::less<P1::Accessory_type>, std::allocator<std::pair<P1::Accessory_type 
        const, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, P1::Generic::P, std::less<std::__cxx11::basic_string<char, 
        std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > 
        const, P1::Generic::P> > > > > >&)

In the meantime changed the system libraries strategy, asking to load the shared lib instead and they disappeared from the list.

Now it's only my libs:

  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

I will reduce the group to target the error better..

Also, for the stdc++ lib, it is the default on Linux, even using Clang,
called with
println!("cargo:rustc-link-lib=dylib=stdc++");

did you check the set_text function is present? is it a generic function or a member of template class which isn't instantiated?

Yes I double checked everything,

I am still investigating, pretty sure a template is at play here..

Ok..
So..
It end up being Boost and Eigen making some fuss..

Not sure how to get around that since they are header only libraries and template heavy..
Implementing a local socket to pass the value via the C binding seems to be the last hope.

Before I dive in, anyone has a better workaround ?
:sleepy:

header only libraries are over-rated IMO, even for pure C++ project (no ffi bindings), they increase compile time drastically.

if you find out which class or function template is causing the problem, maybe try to explicitly instantiate those templates?

Yes this is what I did for some of them but I will enter a hellish territory because the code base is large enough for this task to waste days. Boost is everywhere on a part of the code.

Instead.. and I don't like it, but I will use ZMQ ipc on both sides and be done with it.
Performance will suffer but I have no other choice, given my limited time frame.

Future development is Rust only now. Now way I waste so much time on this again in my life (I hope).
Pfeww..

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.