Shared library and apllication which uses it in one workspace?

newpavlov · May 12, 2023, 9:02am

I have shared (i.e. cdylib) library libfoo written in Rust, wrapper around it foo-sys, and application foo which uses foo-sys and thus depends on the shared library. Ideally, I would like to keep all 3 crates in the same workspace, but naive approach does not work. Cargo does not know that libfoo must be compiled before foo-sys and is not able to link freshly compiled libfoo from the target folder. Do you have any suggestions which I could use here?

alice · May 12, 2023, 9:56am

How are you building libfoo? You should do it in the build script for foo-sys, which is guaranteed to run before compiling foo-sys.

newpavlov · May 12, 2023, 10:13am

Right now I keep them in two separate workspaces. libfoo is built using nothing more than cargo build. I manually place the resulting so into a discoverable location (e.g. /usr/local/lib). Finally, foo-sys links to libfoo using #[link(name = "foo")] without any build scripts.

I would like keep foo-sys usable outside of the workspace (e.g. after publishing it to crates.io), so I don't think building libfoo in a build script would be a right solution.

H2CO3 · May 12, 2023, 10:18am

What makes you think that using a build script would prevent the resulting library from being used elsewhere or publishing to crates.io?

newpavlov · May 12, 2023, 10:21am

My understanding of the @alice suggestion is to build libfoo in foo-sys's build script. Meaning it needs access to libfoo's source code, which obviously would not work for foo-sys published to crates.io.

H2CO3 · May 12, 2023, 10:27am

Yet that's what -sys crates usually do. You have several options for wrapping a separately built library, e.g.:

Include the source code (usually gated under a feature, conventionally named bundled)
Require the library to already be present on the host system.

newpavlov · May 12, 2023, 10:35am

I intend to support only dynamic linking of libfoo, so the bundled approach is not applicable. I effectively use the second approach right now, but during development I would like for foo-sys to link to libfoo which was built from the same workspace. I could introduce a special development-only feature or flag, but it would mean I have a slight difference between development and published versions which may cause issues. Also it requires addition of a build script, while the current version of foo-sys does not have it.

H2CO3 · May 12, 2023, 10:38am

I don't get this. How can you not support static linking?

I understand, but you can't expect Cargo to be able to correctly manage dependencies completely outside of its scope.

jofas · May 12, 2023, 11:06am

To me this sounds like it could be solved with artifact-dependencies. Maybe you could add libfoo as an optional artifact-dependency to your foo-sys crate? I never experimented with artifact-dependencies before so I don't know if that would work, but maybe you'd consider looking into that possibility. Unfortunately artifact-dependencies are still unstable, so maybe they aren't even an option for you. Here the RFC:

github.com

rust-lang/rfcs/blob/master/text/3028-cargo-binary-dependencies.md

- Feature Name: (`bindeps`)
- Start Date: 2020-11-30
- RFC PR: [rust-lang/rfcs#3028](https://github.com/rust-lang/rfcs/pull/3028)
- Tracking Issue: [rust-lang/cargo#9096](https://github.com/rust-lang/cargo/issues/9096)

# Summary
[summary]: #summary

Allow Cargo packages to depend on `bin`, `cdylib`, and `staticlib` crates, and use the artifacts built by those crates.

# Motivation
[motivation]: #motivation

There are many different possible use cases.

- [Running a binary that depends on another](https://github.com/rust-lang/rustc-perf/tree/master/collector#how-to-benchmark-a-change-on-your-own-machine). Currently, this requires running `cargo build`, making it difficult to keep track of when the binary was rebuilt. The use case for `rustc-perf` is to have a main binary that acts as an 'executor', which executes `rustc` many times, and a smaller 'shim' which wraps `rustc` with additional environment variables and arguments. This RFC would allow splitting the shim into a separate crate, building that crate as an artifact dependency, and invoking it as part of the top-level crate.
- Building tools needed at build time. Currently, this requires either splitting the tool into a library crate (if written in Rust), or telling the user to install the tool on the host and detecting the availability of it. This feature would allow building the necessary tool from source and then invoking it from a `build.rs` script later in the build.
- Building tools needed for testing. A crate might build a binary or module designed to work in conjunction with some other tool. The test harness for the top-level crate could have an artifact dependency on the tool, and invoke that tool as part of the testsuite.
- Building and embedding binaries for another target, such as firmware, WebAssembly, or SPIR-V shaders. This feature would allow a versioned dependency on an appropriate crate providing the binary, and then embedding the binary (or a compressed or otherwise transformed version of it) into the final crate. For instance, a virtual machine could build its system firmware, or a WebAssembly runtime could build helper libraries.
- Building and embedding a shared library for use at runtime. For instance, a tool for profiling or debugging other programs could depend on a shared library that it loads into those programs using [`LD_PRELOAD`](https://man7.org/linux/man-pages/man8/ld.so.8.html#ENVIRONMENT). Or, an operating system kernel could build a userspace API library that it loads into userspace applications running on it, in the style of the Linux kernel's [VDSO](https://man7.org/linux/man-pages/man7/vdso.7.html).

This file has been truncated. show original