How to load/switch libraries at runtime?

Hi,
I want to write a game and I want to be able to use different map generators or add new generators without recompiling the game.

The different generators should be stored e.g. in a folder and if I create a new game I want to select one of them.

How can I do this in Rust?

If you realy are looking for dynamic library loading, this might help: link

I have never used it, so I can't really help more :slight_smile:

1 Like

At present, dynamic loading is not generally feasible, sadly. To achieve it, one needs separate compilation in before. Separate compilation needs in turn a polymorphic ABI. A polymorphic ABI depends in turn on the generation of runtime polymorphic code. But at present, all polymorphic functions (such ones with type variables) are monomorphized.

1 Like

This isn't necessarily true. Instead of using generics (compile-time generics) you use trait objects (runtime dispatch). You just need to expose a C-style API instead of a Rust one.

I've actually explored this in the past and wrote up an article on my experiences.

1 Like

Your argument is very true, but in the larger picture it comes with an unpleasant aftertaste. A closer look at dyn shows a mechanism for type erasure or so-called upcast. In essence it smashes the type system. We can go further and generate all necessary dispatch tables by hand, which allows for separation of dispatch table and object, and thus for general polymorphic ABIs.

For example, consider the simple generic power function

trait Monoid {
    fn one() -> Self;
    fn mul(&self, x: &Self) -> Self;
}

fn pow<T: Monoid>(x: T, n: u32) -> T {
    let mut y = T::one();
    for _ in 0..n {y = y.mul(&x);}
    return y;
}

impl Monoid for i32 {
    fn one() -> Self {1}
    fn mul(&self, x: &Self) -> Self {self*x}
}

Now we would like pow to be runtime polymorphic instead. At first the structure of the dispatch table:

struct MonoidPtr(Box<dyn std::any::Any>);

struct MonoidType {
    one: &'static dyn Fn() -> MonoidPtr,
    mul: &'static dyn Fn(&MonoidPtr,&MonoidPtr) -> MonoidPtr
}

Implement it for Monoid:

fn monoid_type<T: Monoid + 'static>() -> MonoidType {
    let one = &|| -> MonoidPtr {MonoidPtr(Box::new(T::one()))};
    let mul = &|a: &MonoidPtr, b: &MonoidPtr| -> MonoidPtr {
        if let Some(a) = a.0.downcast_ref::<T>() {
            if let Some(b) = b.0.downcast_ref::<T>() {
                return MonoidPtr(Box::new(a.mul(b)));
            }
        }
        unreachable!();
    };
    return MonoidType{one,mul};
}

The actual polymorphic code (which is monomorphic from the compiler's point of view):

fn pow_poly(t: &MonoidType, x: &MonoidPtr, n: u32) -> MonoidPtr {
    let mut y = (t.one)();
    for _ in 0..n {y = (t.mul)(&y,&x);}
    return y;
}

Finally a type safe wrapper:

fn pow<T: Monoid + 'static>(x: T, n: u32) -> T {
    let t = monoid_type::<T>();
    let px = MonoidPtr(Box::new(x));
    return match pow_poly(&t,&px,n).0.downcast::<T>() {
        Ok(y) => *y,
        _ =>  unreachable!()
    };
}
3 Likes

in response to the original question, it's also possible to use wasm for plugins using wasmer integration:

wasmer itselt is also written in rust and afaik you can even do the plugins in other languages and compile the to wasm (if you so wish)

1 Like

Interesting. How is the performance of WASM compared to native rust?

there is a runtime cost. here's a benchmark from a year ago: https://medium.com/wasmer/benchmarking-webassembly-runtimes-18497ce0d76e

You generally won't notice the difference. I'm guessing map generators are something which will run infrequently (e.g. at the start of a level instead of every frame) and at times when the user is okay with an operation taking 2 or 3 times the normal amount of time.

You'll only really notice a performance hit when you're frequently jumping between native code and WebAssembly because there are some costs associated with calling into the WebAssembly runtime (you need to marshal arguments, etc). Most runtimes use some form of JIT compilation to get near-native performance though, so.

1 Like

I think I give wasmer a try. this looks promising. Thanks

1 Like

Since it hasn't been mentioned yet, there are WIP scripting languages that are designed for hot reloading. Like what lua provides, but implemented in pure Rust. mun is one that I'm keeping an eye on.

1 Like

I know Mun, but it is in an early state and I think it is not ready yet to use it. But I'll keep an eye on it

1 Like