How to force linking a customized lib without explicitly calling any function in it?

help
1

I'm defining a GlobalAlloc to collect certain heap allocation stats. I define the allocator wrapper in a separate lib and use #[ctor::dtor] to dump stats before the program terminates. However, it seems that the linker keeps optimizing away the functions in the lib, unless I explicitly call some lib function from the target crate, which I would like to avoid.

In cargo build --release -vv, I confirmed that this library is passed to the linker (-L lib_lath -lmy_lib), but the compiled binary does not include any symbol of my lib.

How to force linking this lib?

2

You can try doing extern crate lib or just use lib, but I think that just gets the library passed to the linker.

Does it still work if the call is in dead code?

// this function is never called
pub fn _force_linking_the_library() {
    if false {
        lib::foo()
    }
}

You can experiment with the if false and making the function actually get called[1].

  1. maybe put the call under an if rand::random::<u128>() == 42 or something else impossible that the compiler can't prove will never happen ↩︎

3

Did you mean adding use lib in the target crate's source code? Yes, adding use lib works, but I'd like to avoid modifying the source because I need to test many crates.

Calling from dead code is unnecessary. Simply adding use my_lib does the trick.

4

have you tried the #[used] attribute?

https://rust-lang.github.io/rfcs/2386-used.html

5

Rustc doesn't actually add dependencies unless they are referenced by the crate in some way. For example using use lib;, extern crate lib; or calling a function in this lib. The only way to force add a dependency without modifying the source code is by passing --extern force:my_lib as rustc argument.

1 Like
6

Yes, I have. It didn't work, even with Debug build.

1 Like
7

Thanks. This seems to cause a dependency chain issue. I tried it and got **can't find crate for ctor which my_lib depends on** (my_lib depends on ctor). I'm confused about this: my_lib.rlib should have included ctor, correct? A libctor-hash.rlib exists in my_lib's target/release/deps. I also tried adding this libctor-hash.rlib (though I believe we should avoid directly linking a lib with a hash as it's meant to be intermediate), and I got a cannot find ctor_proc_macro error.

8

Maybe use cargo rustc -- --extern force:my_lib to only pass --extern force:my_lib when compiling the main executable?

9

Yes, that was exactly what I tried (my_lib is the absolute path of libmy_lib.rlib in target/release), and it threw a can't find crate for ctor which my_lib depends on error.

10

Try adding -L /path/to/my_lib's/target/release/deps.

11

This doesn't work either, still the cannot find ctor error. Earlier I also tried "-L my_lib/target/release -lmy_lib" and ld complains cannot find libmy_lib (I confirmed that libmy_lib.rlib exists in the release dir.).

12

For -lmy_lib not working, that is expected. -l only works for C libraries with the extension .a or .so. And it wouldn't allow you to overwrite the global allocator anyway. And -L my_lib/target/release would not allow finding the dependencies of my_lib as those are in my_lib/target/release/deps. Can try --extern force:my_lib=/path/to/libmy_lib.rlib -L /path/to/my_lib's/target/release/deps and if that fails post the full error message?

13

Here is the full error message:

$ cargo rustc -- --extern force:heap_counter=/path/heap_allocator/target/release/libheap_counter.rlib -L /path/heap_allocator/target/release/deps
   Compiling libc v0.2.155
   Compiling bitflags v2.5.0
    .... More Compiling ...
error[E0463]: can't find crate for `ctor` which `heap_counter` depends on

For more information about this error, try `rustc --explain E0463`.
error: could not compile `cfg-if` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
error: could not compile `cfg_aliases` (lib) due to 1 previous error
error: could not compile `is_terminal_polyfill` (lib) due to 1 previous error
...More similar errors
14

Are you sure you don't have RUSTFLAGS set? It seems like some of your dependencies are getting compiled with --extern force:heap_counter=... too rather than just the executable.

15

No. echo $RUSTFLAGS shows nothing. And I just tried in a new terminal and it was the same error.

But I did use my customized compiler with some of my own LLVM passes. (but they were turned off for the compilation.) I specified the path of my own rustc in .cargo/config.toml. Could this be related?

16

Are you setting rustflags in any .cargo/config.toml in the current directory, any parent directory or your home directory?

17

Ah good catch. I set rustflags in .cargo/config.toml because it was more convenient than executing the long command directly in terminal. So earlier I have this in my .cargo/config.toml:

[build]
rustc = "/path/rust/build/aarch64-apple-darwin/stage1/bin/rustc"
rustflags = [
    "-Z", "unstable-options",
    "-L", "/path/heap_allocator/target/release",
    "--extern", "force:heap_counter=/path/heap_allocator/target/release/libheap_counter.rlib",
]

with which cargo build raised the same cannot find ctor error.

After commenting off the rustflags in config.toml, directly running the command in terminal finally worked!

Could you help me spot the problem in the rustflag setting in .cargo/config.toml?

18

"-L", "/path/heap_allocator/target/release" is missing /deps after the ../release. It needs to be "-L", "/path/heap_allocator/target/release/deps".

1 Like
19

@bjorn3 I have a follow-up question. Is the force option still under development? I didn't find it in the rustc book but only in this open issue . This option indeed works, as without it the compiled binary will not include anything from my lib. I'm curious about the status of this (seemingly unstable) option and its implications.

20

I'm not aware of any issues with it other aside from the question of what the exact cli should be (Tracking Issue `--extern` crate modifiers · Issue #98405 · rust-lang/rust · GitHub), but also not aware of anyone actively trying to push for stabilization of it.