Link C native library per target/crate

The question is about linking a Rust package to some C native libraries. The Rust package has one library target and several binary targets. All binary targets use the library target, and each binary target may have its own C native libraries to link with. The library target may link with some common C native libraries for use within the library itself and/or the binary targets.

(First of all, I'm against the idea that the library target links everything used by the binary targets. For example, if a specific C native library is only needed by one binary target, then it should be linked in that binary target, not the library target. This seems in contradiction with the current implementation, but pls read on.)

The first thing that comes to my mind is the #link attribute. I can add a #link attribute to every extern block in all targets, and the code does work. But the disadvantage is, users of my package do not easily see what native libraries are required, and I need to duplicate that information somewhere else.

Then I think it's better idea to put those C native libraries in Cargo manifest. There comes the links field. But it's a field under package section, which means it applies to the whole package not per target. And this field looks only informative having no effect during linking. What actually has an effect is the build script instruction rustc-link-lib. But this instruction only applies to the library target, and I cannot use it to configure linking options of binary targets.

Therefore I now have no means of configuring the linking of each binary target. What I'm looking for is a links/rustc-link-lib option in Cargo manifest that exists under each (lib/bin) target so that I could list C native libraries to link with, for each target. But this option obviously doesn't exist yet. Is there any chance that this linking scheme could be implemented?

I believe the intention is that you have one Cargo package per C library, not that you try to cram multiple libraries into a single package. The simple solution to your problem seems like it would be to just have multiple packages (one per C library), and have each binary package depend on the library packages it needs.

Given that (insofar as I'm aware) you can't even specify Cargo dependencies on a per-binary level, I wouldn't hold out for an option to specify per-binary C libraries.

1 Like

I think this reveals discrepancy between Rust dependencies ([dependencies]) which are available to all crates in a package and C native dependencies (rustc-link-lib) which are available only to the library crate.

This looks like wrapping each C native library in its own Rust package, as cargo and rustc are more clever in dealing with Rust dependencies, letting each crate "pulls" only the dependencies it needs. However this still hides information about which dependency is required by which crate. Let's say if our package depends on a large CLI toolkit that's only used in a binary crate, users won't easily figure out where it's needed by looking at the dependency list in Cargo.toml. We have dev-dependencies to list dependencies needed only in tests, but not something similar for each binary.

As an example, if we find a packge on crates.io that ships a library and some binaries, but we are only interested in using the library, then we may run cargo build --lib to build it, and this shouldn't pull the dependencies solely required by the binaries. This isn't what happens right now.

linker flags are transitive, if a package contains a library target and a binary target, cargo will automatically add the library crate as a dependency of the binary target, so the native libraries will link correctly.

this is possible, but it requires the package manifest be structured properly. the dependencies only required by binary crates should be marked as optional, and the binary targets then use required-feature to enable the dependency specifically.

example:

[lib]
name = "foo"
path = "src/lib.rs"

[[bin]]
name = "foo-cli"
path = "src/main.rs"
require-features = ["cli"]

[dependencies]
bar = "1.0.0"
baz = { version = "2.0.0", optional = true }

[features]
default = []
cli = ["dep:baz"]

by default, cargo build will not build the binary target, and will not pull in "baz" as dependency either. to build the binary, you must explicitly enable the feature, i.e.

cargo build --features cli --bin foo-cli
1 Like

That may force a library target to link dependencies itself doesn't need. For example, some CLI argument parsing library which is typically only required by binaries.

#[link(name = "...")] is recorded in the crate metadata and as such only used when the source code actually depends on the crate that uses it, rather than always when a cargo dependency is specified. You also need it for statically linked C libraries to correctly get their symbols exported from a rust dylib.

1 Like

Using features this way has a side effect: When the binary crate depends on the library crate, this command will rebuild the library crate with feature cli. This rebuild is unnecessary and may be harmful given that the library crate doesn't have to know feature cli.

cargo build --lib
cargo build --features cli --bin foo-cli //rebuilds lib

Maybe you can write a blog post with some examples about #[link(name = "...")]? I'm still not very clear where this would be useful and build script instructions wouldn't be. I'd be surprised if something must be inlined in code and can't be passed as a rustc cli option.

yes, and that's the best we can do if you put the binary crate into the same package.

if you don't want the unnecessary rebuild, you'll have to use separate packages. after all, package is the unit cargo manages, not crate. crate is the unit rustc compiles.

When compiling a dylib crate, rustc explicitly tells the linker which symbols need to be exported. For statically linked C libraries, rustc determine which symbols are necessary to be exported by looking for an extern "C" {} blocks with #[link(name = "...")] attached for static libraries. All symbols mentioned in the extern "C" {} block are then considered as required from the static library and thus re-exported from the rust dylib.

If I understand what you said correctly, it's about rustc re-exporting symbols in a static C library. And #link attribute is one way of specifying which symbols to re-export.

Is this documented anywhere? And why this cannot be done with a rustc cli option? Possibly because specifying every symbol to be re-exported in cli looks much more awkward than doing it inline?

The convention is to create a crate called nameofthelibrary-sys that precisely reexports the entire C library, and then either create a nameofthelibrary crate that wraps it with safe functions and nice Rust features, or just directly depend on nameofthelibrary-sys from your application.

nameofthelibrary-sys will never be shipping any binaries except a build.rs, if custom library install location search logic is needed.

2 Likes

Actually that specific paragraph applies to non-native libraries too. There is no way to specify binary-only dependencies and those have to be put in package dependencies. Then cargo build --lib builds them even though the library crate doesn't need them.

For native libraries, the -sys convention looks OK to me. That turns native library problems into non-native ones. Yet there are still problems with package-wide dependencies even with only non-native dependencies.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.