Solving dependency issue for sys-crate

help
1

I am trying to build a library around a C library libbgpstream. There already exists a libbgpstream-sys crate. The library also requires libwandio, for which there exists wandio-sys crate.

I can compile both libraries fine. However, as soon as try to link to libbgpstream for any binary, the linker throws errors: There are undefined references to function names, both from libwandio and libbgpstream. In addition, it cannot link to functions from parsebgp, a library that is part of libbgpstream. Further, it cannot link to functions from rd_kafka, a dependency of libwandio.

The way I managed to fix those issues it to write a build.rs script that explicitly adds the linker flags to all dependencies of libbgpstream and libwandio:

fn main() {
    println!("cargo:rustc-link-lib=rdkafka");
    println!("cargo:rustc-link-lib=parsebgp");
    println!("cargo:rustc-link-lib=bgpstream");
    println!("cargo:rustc-link-lib=wandio");
    println!("cargo:rustc-link-lib=bz2");
    println!("cargo:rustc-link-lib=z");
    println!("cargo:rustc-link-lib=lzo2");
    println!("cargo:rustc-link-lib=lzma");
    println!("cargo:rustc-link-lib=zstd");
    println!("cargo:rustc-link-lib=lz4");
    println!("cargo:rustc-link-lib=curl");
}

I mannually checked which -l flags are generated by Cargo. Without the build script above, Cargo adds the following flags (while the -L flags are correct and point to the locations where the .a files are found):

... -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc ...

I have a few questions:

  • libbgpstream-sys print the line cargo:rustc-link-lib=static=bgpstream (and wandio-sys prints cargo:rustc-link-lib=static=wandio). Why do I have to explicitly say to link to both bgpstream and wandio explicitly in my crate? Is that related somehow to the rustc-link-lib=static=... part? When I request rustc to statically link a library, doesn't that mean that the library becomes part of the binary, and any subsequent caller libraries don't have to dynamically link to that library using the -l flag?
    Edit: Removing the static= from libbgpstream-sys makes sure that my links with the argument -lbgpstream. So what does that static= do?
  • The library parsebgp is compiled by libbgpstream-sys. Should parsebgp also be added as a link tag to libbgpstream-sys crate?
  • How can I avoid linking to all the other libraries (like libz, libzstd, libcurl, etc...) from my crate? Those are dependencies of wandio-sys. I could not find sys crates for all of those (except libcurl-sys, but the author of wandio explicitly says that they could not complie the thing with libcurl-sys.

Do you have any recommendations for solving the dependency mess here?

2

In your sys crate you should not print linking instructions for any other libraries. Depend on other sys crates that will find libz, bz2, rdkafka, and other dependencies for you. One sys crate == one C dependency.

If the sys crate is used in a Cargo build of a Rust binary or a Cargo build of a Rust cdylib, then all of them will be linked together.

If the sys crate is used to make a static library, to be used later by something else than Cargo, then there's no linking happening. Static libraries differ from dynamic libraries by the fact that they're an "unfinished" raw file that hasn't been linked yet. In that case the non-Cargo user of the static library will have to find and link all the dependencies again.

3

I see, thank you for your answer. If I understood correctly, I should make wandio use all the other sys crates. IIUC, when building wandio, I must tell it to use the things compiled by the other sys crates. How do I get the path to those within the build.rs for adding C flags?

I was trying the same thing as rdkafka-sys does and use DEP_ZSTD_ROOT (see here), but that environment variable is undefined.

4

Unfortunately, there isn't uniform standard way of getting C headers from C library dependencies. It varies between sys crates, so you need to check their docs or build.rs sources. Usually, they will set some DEP_ variable with location of include paths for their headers.

If some sys crates don't expose the headers, the best solution is to file a bug or make a PR adding that (println!("cargo:include={}", header_path)).

You can only use DEP_<name> for your immediate build dependencies. If your crate does not depend on zstd in build deps, then you won't see the DEP_ZSTD_* env var.

Check what vars your script has available:

panic!("{:#?}", std::env::vars().collect::<Vec<_>>());