Life's too short. How can I prolong it?

help
1

I would like to have something like this:

type Callback<'a> = Symbol<'a, unsafe extern fn() -> i32>;

struct Plugin<'a> {
    library: Library,
    callback: Callback<'a>,
}

impl<'a> Plugin<'a> {
    pub fn new(path: &str, name: &str) -> Plugin<'a> {
        let lib = Library::new(path).unwrap();
        let answer: Callback = unsafe {
            lib.get(name.as_bytes()).unwrap()
        };

        Plugin { library: lib, callback: answer }
    }
}

The problem is that Symbol comes from an external crate and requires an explicit lifetime. Thus, the lifetime propagates through out the struct and the impl. In my new function inside the implementation I can now not assign the lib to the library field inside the struct because lib's lifetime is shorter. That would still make sense, though it is clearly not what I want. What doesn't make any sense to me is that I am even unable to call lib.get(...) because apparently lib does not live long enough. But why? Lib is defined before and leaves the scope later.

I can also not assign the Library directly to the struct field, because then I end up with the use of an uninitialized field. E.g.

let plg: Plugin;
plg.library = Library::new(path).unwrap();

Does not work.

I tried to Box everything. As I understand that should transfer ownership to the Box'ed field. But that doesn't seam to work either. Should a Box'ed value not also take on the lifetime of the Box as well?

How do I have to write my code, so that I can load a library (dll) and store it in a struct?

1 Like
2

I am not sure that I understand the problem fully, but you can try to pass a Library as a parameter to the Plugin::new method, and initialize it elsewhere.

3

That doesn't really work for me. I get the same issue until I end up initializing the Library in fn main(). I am trying to arrange my code so that I don't have to do that. I don't think that fn main() is the only place in which I can allocate resources. I must have a conceptual problem.

The problem is that

let lib = Library::new(path).unwrap();
let answer: Callback = unsafe {
     lib.get(name.as_bytes()).unwrap()
};

only works for me if I do that in fn main(). However, I want to have it elsewhere. But if I do that I get the error that lib.get(...) can't be used because lib does not life long enough. If I were to do it in fn main() I will run into a problem later. Ultimately I want to have a folder with some plugins. The app should load all plugins that are fond in the folder. If I add a new plugin, I don't want to recompile the app. So, I must have a dynamic approach. I won't be able to manually do it in the fn main() anymore.

E.g. This works:

fn main() {
    let lib = Library::new("plugins/plg.dll").unwrap();
    let answer: Symbol<unsafe extern fn() -> i32> = unsafe {
        lib.get(b"answer\0").unwrap()
    };

    let a = unsafe { answer() };
    println!("The answer is: {}", a);
}
4

I think you are hitting the fundamental problem of storing an owner and a reference in the same struct: rust - Why can't I store a value and a reference to that value in the same struct? - Stack Overflow.

I think a practical solution would be to pass references as parameters to new and to organize the reloading logic roughly as follows:

fn main() {
    reload_loop();
}

fn reload_loop() {
    let mut owner = ResourceOwner { resources: vec![] };
    loop {
        owner.reload();
        let handlers = owner.resources.iter().map(|x| {
            ResourceHandler::new(x)
        }).collect::<Vec<_>>();
        main_logic(&handlers);
    }
}

fn main_logic(_handlers: &[ResourceHandler]) {}


struct Library;


struct ResourceOwner {
    resources: Vec<Library>
}

impl ResourceOwner {
    fn reload(&mut self) {
        self.resources.push(Library)
    }
}

struct ResourceHandler<'a> {
    resource: &'a Library
}

impl<'a> ResourceHandler<'a> {
    fn new(resource: &'a Library) -> ResourceHandler<'a> {
        ResourceHandler { resource: resource }
    }
}
1 Like
5

At a first glance I don't quite get what is happening here...

I am back at work now and I will need to get back to this later today when I have a chance to devote some time to it. I'll let you know how it works out.

6

Yep, that was a bit hand wavy... Here is a more complete example: GitHub - matklad/dylib-rs.

Thanks for providing an excuse to play with dynamic loading in Rust! :slight_smile:

7

I have written a crate over here GitHub - emoon/dynamic_reload: Dynamic reloading of shared libraries that does dynamic loading of libs but can also reload them if they have been changed on disk. It may not solve your exact problem though.

1 Like
8

@matklad
First of thank you very much for you help! It is very much appreciated.

Oh man... this was a complicated and heavy birth!
Your link about storing an owner and a reference in the same struct helped me a lot. It explained the issue very well. Your example at github was also very helpful. It contains some nifty techniques I have not even considered (e.g. nested functions).

As I have some plans on how to use this in my app I had some adjustments to make. I will post my solution at the end of the post in case someone else will find it useful. I have a running demo now but it still needs some changes for my specific case later. But it's late now... tomorrow is another day and I will have another issue to solve :slight_smile: I consider this specific problem as solved.

@emoon
Thanks for your link. After you posted it I remembered it from another thread. I have looked at it for some inspiration but ended up sticking to finding a solution for my specific problem as I wanted to learn from it so I know how to solve things in the future.

For reference here is what I ended up with:
(Including a hint for further problems, but that was not the issue of this thread. I have also omitted some app specific parts.)

pub type PluginFunctionCallback = extern fn(&[Value], Element) -> Option<Value>;

pub struct PluginLibraries {
    libraries: HashMap<String, Option<Library>>,
}

impl PluginLibraries {
    fn new() -> PluginLibraries {
        PluginLibraries { libraries: HashMap::new() }
    }

    fn load_plugin_library(&mut self, path: &str) {
        fn load_library(path: &str) -> Option<Library> {
            Some(Library::new(path).unwrap())
        }

        self.libraries.insert(path.to_string(), load_library(path));
    }
}


pub struct Plugins<'a> {
    extensions: HashMap<String, Option<Symbol<'a, PluginFunctionCallback>>>,
}

impl<'a> Plugins<'a> {
    fn new() -> Plugins<'a> {
        Plugins { extensions: HashMap::new() }
    }

    fn load_extensions(&mut self, path: &str, identifier: &str, name: &str, libs: &'a mut PluginLibraries) {
        self.extensions.insert(identifier.to_string(), libs.libraries[path]
                                                        .as_ref()
                                                        .map(|l| unsafe { l.get(name.as_bytes()).unwrap() }));
    }
}


fn main() {
    let mut pluginLibs = PluginLibraries::new();
    pluginLibs.load_plugin_library("plugins/plugin_normal/plugin_normal.dll");

    let mut plugins = Plugins::new();

    plugins.load_extensions("plugins/plugin_normal/plugin_normal.dll",
                            "plugin_normal_script_foo",
                            "script_foo", &mut pluginLibs);

    /* loading second function needs further work, cannot have multiple borrowed ref's
    plugins.load_extensions("plugins/plugin_normal/plugin_normal.dll",
                            "plugin_normal_script_bar",
                            "script_bar", &mut pluginLibs);
    */
}
9

I'm a little surprised there's nothing in the standard library with the shape of fn leak<T>(object: T) -> &'static T. Obviously there are a lot of cases where it'd be completely inappropriate, but if you're loading a bounded number of dynamic libraries "don't unload them" becomes a reasonable option.

fn leak<T>(object: T) -> &'static T {
    use std::mem;
    let box_ = Box::new(object);
    let permaref = unsafe { mem::transmute::<&T, &T>(&box_) };
    mem::forget(box_); /* don't ever free the value */
    permaref
}
10

There was a proposal (got closed in favor of getting more experience out of the tree), and the function now exists as an external crate. It has a different signature (fn leak<T>(v: Box<T>) -> &'static T, and also traits for owned containers) though.

11

That makes me wonder. I have been trying to avoid global variables but would it make sense to have the plugins as global variables? What would be the advantages?