Dynamic linking inconsistency issue between cargo run and cargo test

Hi Rust community,

I'm facing a weird linking issue. I have the following code in my build.rs:

println!("cargo:rustc-link-lib=dylib=my_lib");

I have some code in my main.rs to invoke the function defined in my dynamic library (my_lib), when I do cargo run, it cannot find the function to be invoked. But I copied the same code to a test in my lib.rs and do cargo test, the function (defined in the dynamic library) can be found and invoked with no problem. Why is that?

Thanks for the help in advance!

The -l flag is only passed to the library target of the package, unless there is no library target, in which case it is passed to all targets. This is done because all other targets have an implicit dependency on the library target, and the given library to link should only be included once. This means that if a package has both a library and a binary target, the library has access to the symbols from the given lib, and the binary should access them through the library target’s public API.

Build Scripts - The Cargo Book

So you can do one of the following:

  • Move the library bindings to another crate (usually in the same workspace), and make that a dependency of this one, so you can use it in lib.rs and main.rs.
  • Reexport or wrap the relevant functions in lib.rs, and use those reexports in main.rs.
3 Likes

Thanks for your answer, that explains it!

I tried your Option1 by having the build.rs (which does the linking) in the dependency library crate, but it still does not work for my crate somehow(same error that the function in the linked library cannot be found). Am I missing anything?

A side question: I tried adding an example code to the dependency library crate to debug (run via cargo run --example), it cannot work. Is it due to the same reason?

Thanks in advance!

You also have to move your extern functions to the dependency and reexport or wrap them.

If it's not that, then you'll probably have to show some actual code and error messages.

Thanks for your suggestion! Let me explain my code a little bit (hopefully I can explain it clearly):

I have a system crate A written by others, it exposes a Function::get(function_name: &str) function to get a function from a shared library.

Then I wrote a crate B that depends on A, I wrote a function F1 which calls Function::get("f1") where f1 is a function in the shared library I linked in the build.rs of my crate B. Then I am facing the issue in the post: if it's a test that calls F1 in the lib.rs it works, if F1() is in main.rs it fails. I would love to hear your advice. :slight_smile:

It sounds like you're trying to recreate libloading, which is a library for dynamic loading, not dynamic linking. Dynamic linking is essentially the same as static linking, except the library isn't included in the binary. Dynamic loading lets you decide, at runtime, which library and functions to include. I don't know if there's anything in between, where you know the library name but not the functions.

If you know all the functions you may need at compile time, you could include them all with extern and have Function::get simply look them up in a big match or phf table. If you don't know which functions you need, use libloading and just hardcode the library name.

Does that sound like what you're looking for?

Thanks for your thoughts as always! I think showing the actual code would help explain the issue. :slight_smile:

Here is my crate (crate B in the previous threads), its build script, and here is where it interacts with crate A.

Function::get is implemented here in crate A, basically there is a registry system in the native library of crate A: tvm/src/runtime/registry.cc at 51bdaec6e35cddb8256f19a5cd2184b0bb7cb824 · apache/tvm · GitHub. The native library is written in C++ and it has an C API, so the crate A exposes Rust API such as the one I am using to get the global registered function which is in my linked library libmlc_llm_module.so (linked in my crate's build.rs).

If you know all the functions you may need at compile time, you could include them all with extern and have Function::get simply look them up in a big match or phf table.

Yes, I do know the function I need at compile time, which is the "mlc.llm_chat_create" in the dynamic library, but it has to be loaded by the Function::get.

Thank you for your patience! Would love to hear your thoughts on it!

Unfortunately, this is going over my head. I thought you just wanted an FFI function, but it looks like you have an FFI function that's supposed to find a pointer to another function. It's very different from regular rust FFI.

Do you have any suggestions in this case? I just don't want any libraries that depend on my crate to also include a build.rs to do the linking... :cry:

I have no idea. You'll have to talk to the person who designed this system.

Thanks for your help!

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.