Libloading appears to be using a stale dylib

Hello,

I am attempting to understand how dylibs work in rust (with hopes of building an application that uses them as plugins) and I am a little confused.

main.rs
use libloading;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    loop {
        let render = libloading::Library::new("src/plugins/render/target/debug/librender.dylib")?;
        unsafe {
            let func: libloading::Symbol<extern "C" fn() -> u32> = render.get(b"it_works")?;
            println!("{}", func());
        }

        std::thread::sleep_ms(1000);
    }
}

Then in my "plugin" called render.rs:

#[no_mangle]
fn it_works() -> u32 {
    42
}

The code works great and I see 42 printed out every second. However, when I edit render.rs and change that number to 43, then run cargo build inside that project (not the main project), the number does not change.

Are dylibs cached in some way that I am not understanding? Is it possible to do a "hot reload" while the main application is running?

Thanks

I'm not completely sure why this isn't working but something I would like to point out is that your it_works function should be declared like so:

#[no_mangle]
pub extern "C" fn it_works() -> u32 {
    42 
}

What OS are you running? I'd have to assume macos because you're dealing with dylibs but if it's windows I might be able to help.

I changed the definition as you suggested, but the result is the same.

I am running on macOS.

If you have a chance, I am curious if something this basic works on your windows machine. I don't have access to one currently.

Thanks.

1 Like

This seems to be working with the following structure for me on windows:

project_dir/
            Cargo.toml
            src/
                main.rs
                plugin/
                       Cargo.toml
                       src/
                           lib.rs

And the following code:

  # Cargo.toml 1
[package]
  # ...

[dependencies]
libloading = "0.5.2"
//main.rs
use libloading as lib;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let library = lib::Library::new("./src/plugin/target/release/plugin.dll")?;
    unsafe {
        let func: lib::Symbol<extern "C" fn() -> u32> = library.get(b"it_works")?;
        println!("{}", func());    
    }
    Ok(())
}
  # Cargo.toml 2
[package]
  # ...
[lib]
name = "plugin"
crate-type = ["dylib"]
//lib.rs
#[no_mangle]
pub extern "C" fn it_works() -> u32 {
    43
}

If I change 43 to something else, go into ./src/plugin/ and cargo build --release, then go cargo run in the project dir, it works.

As to where you could've gone wrong, here are some possibilities (But then again, I have almost 0% experience with macos)

  • Try ./src/.../... instead of src/.../...
  • Try release mode on the plugin
  • Try running cargo clean in ./src/plugin and then in .
  • Try specifying the full path, which I think would look like ~/path/to/project_dir/src/plugin/target/release/plugin.dylib
  • Try changing dylib in your Cargo.toml to cdylib, which as far as I know makes everything exposed be an extern "C" function (Correct me if I'm wrong).

First off, thank you for spending your time looking into this.

If I change 43 to something else, go into ./src/plugin/ and cargo build --release , then go cargo run in the project dir, it works.

Yes, I get the same thing. The issue is when I tuck that code inside of the loop{} and then repeat the above steps while the application is running, I see no change.

Thanks.

Oh, I believe you've misunderstood what Libloading is supposed to do; it's only supposed to load it once, not every time it changes on disk. If you want something like that, then try to use some kind of filesystem notification like notify to drop the preexisting library (It keeps a hold on the file) and then to load the new one.

1 Like

Okay, I believe I understand what is wrong, now I just have to figure out how to fix it.

https://github.com/emoon/dynamic_reload

The above looks like a good place to start getting ideas from. It looks like that crate simply copies the file with a unique name using an epoch prefix to a different directory and renames/shadows it everytime.

Does that mean if I reloaded the library 20x that there will be 20 copies in memory? Or how do I drop it?

Thanks!

When you replace the Library in memory, it will be dropped, and looking at the code for dropping it(On unix):

impl Drop for Library {
    fn drop(&mut self) {
        with_dlerror(|| if unsafe { dlclose(self.handle) } == 0 {
            Some(())
        } else {
            None
        }).unwrap();
    }
}

We can see it uses dlclose which has an entry in the man page:

dlclose()

The function dlclose () decrements the reference count on the dynamic library handle handle . If the reference count drops to zero and no other loaded libraries use symbols in it, then the dynamic library is unloaded.

Therefore it will not be loaded 20x on unix systems (and probably not either on windows, glancing at its code it doesn't keep any kind of global state unless the winapi function name is misleading [Which isn't uncommon in my experience]).


Looking at the code for dynamic_reload it appears that it does the job for you.

2 Likes

Does a loop structure in Rust not call drop on each iteration?

Just to be safe, I tried adding a drop to the library, but that did not appear to help.

let render = libloading::Library::new("src/plugins/render/target/debug/librender.dylib")?;
        unsafe {
            let func: libloading::Symbol<extern "C" fn() -> u32> = render.get(b"it_works")?;
            println!("{}", func());
        }
        drop(render);

I am guessing this issue is probably why the author created that other library that had to make fake copies and shadow the dylib using the filesystem.

This might be relevant:

TL;DR - unloading a library loaded via dlopen in macOS seems to not be possible.

EDIT: well, no, I have that backwards. You may be running into similar restrictions in some way, but it seems as though the objective-c runtime is what's unable to unload modules... so perhaps something else is going on!

1 Like

You tipped me off with some good google keywords:

https://github.com/rust-lang/rust/issues/47974

Unfortunately, the issue is closed as "fixed by apple", but that does not appear to be the case for Mojave :frowning:

Here is another example of the hack of renaming the file uniquely over and over:
https://github.com/kurtlawrence/papyrus/issues/16

Ah, so it sounds like you could try building your dylib as a no_std lib, if possible. At least you could confirm that TLS usage is what's causing your library to not be unloaded.

Yeah, no_std is not going to be an option for what I was hoping to do.

I even tried the solution used in papyrus, and it still did not work! I'll try the dynamic_reload library next I guess, but I don't think its going to change the outcome.

I am a Rust beginner, so the concept of attempting a cdylib also scares me as I barely know enough about Rust types to get by, much less C ones.

Just to followup, I attempted to use the dynamic_library but I had the same issue. It appears that macOS it out to kill my project before I can even start :frowning:

Thanks for your help, both of you. I was really hoping to avoid compile times on my raspberry pi, but it seems if I want to do any dylib stuff, I have no choice.

If your targeting Linux, have you considered using Docker or some other virtualized environment for development? It’ll probably be a lot less time consuming than waiting for raspberry pi builds... :slight_smile:

1 Like