Linking with local C library

I need to link my rust application to an external C library that may be in a completely unpredictable place ... which is to say, I know where it is, but the code under git doesn't.

At the moment I have a build.rs file with the following content:

const LIB_PATH: &str = "/path/to/my/lib";

fn main()
{
    println!("cargo:rustc-link-search={:}", LIB_PATH);
}   

and then I have to run this with the following command (I have a main.rs for testing at the moment as well as a lib.rs for the library I'm writing):

LD_LIBRARY_PATH=/path/to/my/lib cargo run

So I think I have two issues here:

  1. First, how can I link the library path into my build? For a C build I'd use -Wl,-rpath=/path/to/my/lib, but I haven't seen a way to do that in build.rs.

  2. How should I best specify the path to my local library? It can't go into anything under source control, because it changes. In other projects I use a CONFIG file which can be read by make files and which needs to be edited by users first before trying to build.

I notice that this question is rather similar to an older question with the same title, but the problems were slightly different: Linking with custom C library. (Huh: I would have recycled the same title, but the forum says "no"!)

Normally you'll tell rustc "just link to libfoo" (cargo:rustc-link-lib=foo) and make sure the library is somewhere the linker normally looks (e.g. /usr/lib). Then when the program is started the dynamic loader will see that your library requires libfoo and search a pre-defined set of locations (of which $LD_LIBRARY_PATH is one of the first to be checked, if set) for libfoo.so.

If you're on Linux the dynamic linker/loader man page is really useful (man ld.so). Otherwise if you're on a Windows machine, Dynamic-Link Library Search Order may point you in the right direction. I don't use Mac, but hopefully I've provided enough keywords to help you find the right information.

2 Likes

Linux

It seems it is not possible to override the rpath from within the build.rs script, so you'll need to append -Clink-args=-Wl,-rpath=/absolute/path/to/library/folder to the rustc flags. You can do so:

  • through the RUSTFLAGS environment variable,

  • or through the .cargo/config file:

    [build]
    rustflags = ["-C", "link-args=-Wl,-rpath=..."]
    

I've tested locally and the following command + build.rs script (for the -L flag) works:

  • command

    (
        export LIB_NAME_PATH="$PWD/path/to/library/folder"
        export RUSTFLAGS="-Clink-args=-Wl,-rpath=$LIB_NAME_PATH"
        cargo r
    )
    
    • (I have used a subshell to scope the env vars)
  • build.rs

    fn main ()
    {
        let lib_path =
            ::std::env::var("LIB_NAME_PATH")
                .expect("Please provide the `LIB_NAME_PATH` env var")
        ;
        println!("cargo:rustc-link-search={}", lib_path); // -L $LIB_NAME_PATH
    }
    

Regarding the name of the library rather than its containing folder, you'll need either the build.rs cargo directive suggested by @Michael-F-Bryan, rustc-link-lib, or annotate the extern "C" { ... } block with the #[link(name = "name_of_the_lib")] attribute
(both append -l name_of_the_lib to the compilation command)


Aside

FWIW, you can have relative rpaths using the $ORIGIN "magic variable", which could solve your whole "unpredictable path" by using a relative path within the repository (e.g., using a submodule to include the library you link against); you just have to make sure the $ sigil is properly escaped so that shell substitution does not happen:

ln -sf ./target/debug/bin-name  # or .../release/...
(
    export LIB_NAME_PATH="path/to/library/folder"  # relative path to ./bin-name
    export RUSTFLAGS="-Clink-args=-Wl,-rpath=\$ORIGIN/$LIB_NAME_PATH"
    cargo build # --release
) && ./bin-name
  • now you could even hard-code these paths within the repository code (build.rs and .cargo/config)

You can use a linking wrapper script to insert the linker args you want:

Create link wrapper

First, create a linking wrapper script rust_link_wrapper.sh (make sure it's executable):

#!/bin/sh
set -eu
# Pass through arguments
# Use correct linker for your target
gcc "$@" -Wl,-rpath=/path/to/my/lib

Tell cargo to use link wrapper

Now we need to tell cargo that to use that as the linker. You can do that in several ways:

  1. Create a wrapper to rustc that sets the linker by adding the args -C /path/to/rust_link_wrapper.sh. When calling cargo, set the RUSTC_WRAPPER environment variable to this new script or add a .cargo/config file with the rustc-wrapper option.
  2. Set target.<triple>.linker, either through an environment variable or .cargo/config.

Config files

How to create a cargo config file:
https://doc.rust-lang.org/cargo/reference/config.html#hierarchical-structure

Environment variables

How to set environment variables to specify cargo config:
https://doc.rust-lang.org/cargo/reference/config.html#environment-variables

If you set options through an environment variable, then you can create a script or Makefile that will automatically set pass set the environment variables:

#!/bin/sh
set -eu
CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=rust_link_wrapper.sh \
    cargo "$@"

Run as: ./cargo.sh build

1 Like

Thank you all for your helpful comments. It's a bit of a shame that build.rs can't generate the extra linker argument, means I have to have the appropriate logic in two different places, and the wrapper script becomes a key part of the build.

So here is my build.rs script:

use std::env;  

fn main()
{   
    let lib_path = env::var("EPICS_LIB_PATH")
        .expect("Must define EPICS_LIB_PATH");
    println!("cargo:rustc-link-search={:}", lib_path);
}

and here is the wrapper shell script I have made:

#!/bin/sh

[ -e CONFIG ]  &&  source ./CONFIG
: ${EPICS_BASE:?Must define EPICS_BASE}
: ${EPICS_HOST_ARCH:=linux-x86_64}

export EPICS_LIB_PATH=$EPICS_BASE/lib/$EPICS_HOST_ARCH
export RUSTFLAGS="-Clink-args=-Wl,-rpath=$EPICS_LIB_PATH"

cargo run

Seems kind of klunky, but does the job.

RUSTFLAGS is used globally for the entire project, including linking compiler plugins, build scripts, and all other libraries, so it usually breaks things in non-trivial projects.

I suggest fixing rpath in the final executable as a post-processing step.

How would I go about that? I was a bit concerned about setting RUSTFLAGS for the reasons you give. Do I use the link wrapper process as described above (which I definitely don't understand at the moment), or is this another process.

Actually I like the suggestion of @tmfink a lot. This way you do not need the build.rs script at all! Just add a -L${EPICS_LIB_PATH} on top of the -Wl,-rpath=${EPICS_LIB_PATH} (the wrapper script can itself source CONFIG as you did) + the .cargo/config and you should be good to go: cargo run should just work.

Something along the lines of this post.

Ooh. Didn't know about readelf and patchelf. Will give that a go.

BTW, if you're doing this for your own project then an env var with a path may be fine. But if you're publishing this as a sys crate, then it may be hard for users to know they need to set up a path for a dependency-of-a-dependency.

Rust sys crates usually take a bit of extra effort to search for the C library location if the exact path is not specified (e.g. check usual system directories, try pkg_config, etc.)

1 Like

Yes, I understand this. For the time being this is an experiment in learning how to do rust. Unfortunately the conventions for installing EPICS are very unpredictable and vary from facility to facility; where I work we'll have a number of versions installed in paths like /dls_sw/epics/R3.14.12.7/base and the like. Typically also I don't have the power to install anything in system directories, or at least when I do this is highly frowned on, so automated searching for the CA libraries is never going to work.

I agree that I should do the necessary searches if the base directory has not been given. I'll try and figure out what I need to do reading your sys create link, looks helpful, but I've got lots of other stuff to figure out before I get there.

2 Likes