Platform dependent behaviour for changing dylib linking paths?


#1

I’m encountering some issues doing runtime linking on Linux. I’ve written a small test program to demonstrate the issue:

// dylibBar -requires-> dylibBaz
// binFoo runtime loads dylibBaz
//  works
// binFoo runtime loads dylibBar
//  works on Windows 7
//  crashes on Linux (Fedora, Debian), says it can't find dylibBaz
//   but it works if it's passed LD_LIBRARY_PATH with dylibBaz's path, at startup.
//   modifying LD_LIBRARY_PATH at runtime to include dylibBaz's path doesn't stop it from crashing.
//  why?
//  Can we have cross-platform behaviour?

#![feature(std_misc)]
use std::mem;
use std::process::Command;
use std::env;
use std::fs;
use std::fs::{File};
use std::io::Write;
use std::path::{PathBuf};
use std::dynamic_lib::DynamicLibrary;

fn main() {

    // Lets create some dylibs.

    // Dylib1 will be a library that Dylib2 relies on.
    let dylib1_src = "
    #[no_mangle]
    pub fn hello() {{
        println!(\"hello from dylib1!\")
    }}";
    let dylib1_bin = create_dylib("dylib1", &dylib1_src, None);


    // Dylib2 will be a library that's hotloaded by the current executable.
    let dylib2_src = "
    extern crate dylib1;
    #[no_mangle]
    pub fn run() {{
        println!(\"hello from dylib2!\");
        dylib1::hello();
    }}";

    let mut dylib1_target_dir = dylib1_bin.clone();
    dylib1_target_dir.pop();
    println!("{}", dylib1_target_dir.to_str().unwrap());

    let dylib2_bin = create_dylib("dylib2", &dylib2_src, Some(vec![dylib1_target_dir.to_str().unwrap()]));

    let mut dylib2_target_dir = dylib2_bin.clone();
    dylib2_target_dir.pop();

    // We can successfully open dylib1.
    match DynamicLibrary::open(Some(&dylib1_bin)) {
        Err(why) => {
            println!("Could not load dylib1: {}", why);
        }
        Ok(binary) => {
            println!("Opened dylib1");
            let hello_func = load_symbol(&binary, "hello");
            hello_func();
        }
    };

    // Changing the environmental variables for the current process works on Windows.
    // Doesn't seem work on linux, see below.
    let mut search_paths = DynamicLibrary::search_path();
    search_paths.push(dylib1_target_dir);
    search_paths.push(dylib2_target_dir);
    env::set_var(DynamicLibrary::envvar(), &DynamicLibrary::create_path(&search_paths));

    // open dylib2
    // it fails, claiming
    // "Could not load dylib2: libdylib1.so: cannot open shared object file: No such file or directory"
    match DynamicLibrary::open(Some(&dylib2_bin)) {
        Err(why) => {
            println!("Could not load dylib2: {}", why);
        }
        Ok(binary) => {
            println!("Opened dylib2");
            let run_func = load_symbol(&binary, "run");
            run_func();
        }
    };


    // if we run this executable with
    // LD_LIBRARY_PATH=$LD_LIBRARY_PATH:path/to/dylib1/target
    //  it runs

    // How can we have these linking paths be updated at runtime?

    // If I'm not mistaken, modifing std::env::set_var should do it,
    // but it doesn't.
}

fn load_symbol(dylib: &DynamicLibrary, name: &str) -> fn() -> () {
    println!("Loading {} symbol", name);
    unsafe {
        match dylib.symbol::<fn() -> ()>(name) {
            Err (why)   => { panic! ("Loading error: {}", why); }
            Ok  (func)  => { mem::transmute(func) }
        }
    }
}

fn create_dylib(name: &str, src: &str, link_paths: Option<Vec<&str>>) -> PathBuf {

    // create dir ./name
    let current_dir = env::current_dir().unwrap();

    let mut src_dir_path = current_dir;
    src_dir_path.push(name);

    // create dir ./name/target
    let mut target_dir_path = src_dir_path.clone();
    target_dir_path.push("target");

    fs::create_dir_all(&target_dir_path).unwrap_or_else(|e| {
        panic!("failed to create dir: {}", e)
    });

    // create file ./name/name.rs
    let mut src_path = src_dir_path.clone();
    src_path.push(format!("{}.rs", name));

    let mut f = File::create(&src_path).unwrap();

    // populate file contents with src
    f.write_all(src.as_bytes()).unwrap();

    // rustc --out-dir /name/target --crate-type=crate_type ./name/main.rs
    let mut command = Command::new("rustc");
    command.arg("--out-dir").arg(&target_dir_path);
    command.arg("--crate-type").arg("dylib");

    if link_paths.is_some() {
        for path in link_paths.unwrap().iter() {
            command.arg("-L").arg(path);
        }
    }

    command.arg("-C").arg("prefer-dynamic");

    command.arg(src_path);

    let output = command.output().unwrap_or_else(|e| {
            panic!("failed to execute rustc: {}", e)
        });

    if output.stdout.len() != 0 {
        println!("{} stdout: {}", name, String::from_utf8_lossy(&output.stdout));
    }
    if output.stdout.len() != 0 {
        println!("{} stderr: {}", name, String::from_utf8_lossy(&output.stderr));
    }

    // return ./name/target/name
    PathBuf::new().join(&target_dir_path).join(&format!("{prefix}{name}{suffix}",
        prefix = env::consts::DLL_PREFIX,
        name = name,
        suffix = env::consts::DLL_SUFFIX)
    )
}

I’ve been unable to tell if this is a Rust issue, or just expected behaviour under Linux. Any help would be greatly appreciated!


#2

This is expected behavior under Linux. From the man page on dlopen:

       The function dlopen() loads the dynamic library file named by the
       null-terminated string filename and returns an opaque "handle" for
       the dynamic library.  If filename is NULL, then the returned handle
       is for the main program.  If filename contains a slash ("/"), then it
       is interpreted as a (relative or absolute) pathname.  Otherwise, the
       dynamic linker searches for the library as follows (see ld.so(8) for
       further details):

       o   (ELF only) If the executable file for the calling program
           contains a DT_RPATH tag, and does not contain a DT_RUNPATH tag,
           then the directories listed in the DT_RPATH tag are searched.

       o   If, at the time that the program was started, the environment
           variable LD_LIBRARY_PATH was defined to contain a colon-separated
           list of directories, then these are searched.  (As a security
           measure this variable is ignored for set-user-ID and set-group-ID
           programs.)

       o   (ELF only) If the executable file for the calling program
           contains a DT_RUNPATH tag, then the directories listed in that
           tag are searched.

       o   The cache file /etc/ld.so.cache (maintained by ldconfig(8)) is
           checked to see whether it contains an entry for filename.

       o   The directories /lib and /usr/lib are searched (in that order).

       If the library has dependencies on other shared libraries, then these
       are also automatically loaded by the dynamic linker using the same
       rules.  (This process may occur recursively, if those libraries in
       turn have dependencies, and so on.)

Note specifically that it uses the value of LD_LIBRARY_PATH at the time the program was started, not at the time of loading the library; so changing it from within your program won’t work.

Your best choices are probably to install the libraries in the standard system search directories, or link your executable with an rpath that points to the directory to search for your libraries. I’ve never done this, but looking at the man page, you should be able to do this with the rustc -C rpath=/path/to/libs; it looks like you can’t pass rustc flags to cargo yet, but there are some workarounds in that ticket like having your rustc or ld be a shell script that passes in the appropriate options.