I am a C++ developer at first but I used a lot of languages. The question is about plugins - how to write them in Rust? In java I had a project which could open a .jar at runtime and to extend the program functionality by using these jars. In python we could do almost the same thing. In C++ with Qt we can create a QPlugin which does the same thing. In C we may simply use dlopen. In Rust we may open other libs which symbols are unmangled but how can I make a plugin exactly written in rust, inside an a rust library (rlib file) ?
3 Likes
The way you do it is that you set your crate type to dylib (see here Page Moved)
and in order to export symbols without name mangling you do something like this in your code
#[no_mangle]
pub fn shared_fun() -> i32 {
42
}
Now you can load this library as any other shared library. If you want to load it in Rust I can recommend taking a look at GitHub - nagisa/rust_libloading: Bindings around the platform's dynamic library loading primitives with greatly improved memory safety.
I have written my own lib that I use for loading plugins as well over here GitHub - emoon/dynamic_reload: Dynamic reloading of shared libraries which supports reloading of plugins when they have been recompiled which you may find useful.
6 Likes
So, if I am gonna write a plugin in rust language and use it in rust language my symbols should always be unmangled anyway? I thought it is necessary only for libraries which should provide compatibility to C and other languages which does not know about mangling of the compilers but rust compiler, I am sure, DOES know how it mangle its own names so why do we need #[no_mangle] here?
I don't know about that. The way I do it is to manually load the libraries and have a known (unmanged) symbol to use but that is also because my code supports to load C/C++ plugins so I have to make sure it works in that case also but I'm unsure about the Rust only case.
1 Like
Rust includes the crate name in the mangled symbol. This is good if you're linking an app, because it prevents spurious conflicts, but if you're using the dynamic linker API for extensibility you probably want to have more than one crate providing the same interface using the same symbols, so you can't use builtin mangling.
Also, if you want to support plugins built using a different version of Rust from the main program, you'll need to use #[repr(C)] types and extern "C" fn () function pointers, because current versions of Rust do not have a stable ABI.
So it's not just a matter of "the compiler not knowing how to mangle its own names"; plugin systems have subtly different requirements that are not yet directly supported in the mangler.
2 Likes
Does that mean code like the following:
trait Plugin {...}
fn get_plugin() -> Box<Plugin + 'static> {...}
Will not work or just break across rust minor versions? Presumably the only way to consume that plugin would be unsafe anyways?
Here is a small example of a Rust plugin in my code https://github.com/emoon/ProDBG/blob/master/src/plugins/bitmap_memory/src/lib.rs
This part is what is seen from the outside and the only exported symbol:
#[no_mangle]
pub fn init_plugin(plugin_handler: &mut PluginHandler) {
define_view_plugin!(PLUGIN, b"Bitmap View\0", BitmapView);
plugin_handler.register_view(&PLUGIN);
}
init_plugin will be called from the outside and this is the entry point in which the plugin can registers it's plugins (there can be more than one plugin in one dll file)
In this case the plugin type is "View" and needs to implement the View trait.
then define_view_plugin! is used to setup a C structure which is passed down to plugin_handler register code. How this works can be seen in this file https://github.com/emoon/ProDBG/blob/master/api/rust/prodbg/src/view.rs
It pretty much sets up a bunch of function pointers + user data that is used later on in the main application to call the plugin.
4 Likes
Sorry for jumping into the thread. But I have a question about the same topic. I think it's better than opening another thread about the same thing.
At first I found examples online where dynamic libs are loaded with "std::dynamic_lib::DynamicLibrary". Later I found that this is deprecated. Surely dynamically loading libraries is an important requirement. The above recommended solutions require crates, which require other crates...
It seams that this should be build into the language rather than having to rely on external dependencies. I don't understand the logic about removing a functionality that basically was already there. Especially an important one like this. Am I missing something here? You really have to add crates (dependencies) in order to load a library? Is it not possible to write a dynamic library with rust and then load it dynamically in another rust application? Or is this functionality temporarily removed, because the ABI is not stable yet?
1 Like
Nitpick: 'static is assumed on trait objects nested under Box if no other lifetime is specified (RFC 1156).
Yes. Trait objects are part of the Rust calling convention, and cannot be assumed to be compatible between two rust versions.
If you need ABI compatibility between Rust versions, I think you need DIY vtables:
#[repr(C)]
pub struct PluginVtbl {
free: unsafe extern "C" fn(obj: *mut Plugin),
hello: unsafe extern "C" fn(obj: *mut Plugin),
}
#[repr(C)]
pub struct Plugin {
vtbl: &'static PluginVtbl,
}
#[repr(C)]
struct MyPlugin {
vtbl: &'static PluginVtbl,
counter: usize,
}
unsafe extern "C" fn free_myplugin(ptr: *mut Plugin) {
Box::from_raw(ptr as *mut MyPlugin);
}
unsafe extern "C" fn hello_myplugin(ptr: *mut Plugin) {
let self_ = &mut *(ptr as *mut MyPlugin);
self_.counter += 1;
println!("Hello! {}", self_.counter);
}
static vtbl_myplugin : PluginVtbl = PluginVtbl {
free: free_myplugin,
hello: hello_myplugin,
};
#[no_mangle] pub extern "C" fn create_plugin() -> *mut Plugin {
Box::into_raw(Box::new(MyPlugin {
vtbl: &vtbl_myplugin,
counter: 0,
})) as *mut Plugin
}
(Incidentally, a small tweak on this will get you COM objects, if that matters to you.)
1 Like
The functionality was removed because we didn't want to mark it as stable. Because the standard library is not versioned separately from the language, stabilizing something is a very high committment; we'd have to support it from now until Rust 2.0, which will ideally never happen (because semver, 2.0 means we had to break backcompat in a significant way).
Keeping it as an external crate means the design can be iterated, because cargo's version matching rules will keep all users on compatible versions.
Re-adding it to the standard library would be likely to happen after the design has settled down and the community has standardized on a best-of-breed crate.
Rust/Cargo philosophy strongly encourages crates. This isn't C, where every additional dependency cuts your user's chances of a successful build in half (I exaggerate, but not much); if you have problems with "dependencies" as a concept that's something you're not going to see less of any time soon.
1 Like
It's not that I have a problem with crates. But I think they should extend the functionality. Whereas I would expect loading a library to be a standard feature. I just wanted to check, if that will be built in in a future version. Do you have a link to some sort of a roadmap where we can check upcoming features of rust? Or do you have a time estimate at which point it is reasonable to expect for things to settle down and stabilize? I think I just need to be a bit more patient 
It's not a roadmap as such but the RFC repository holds specifications for new features and other changes that have been agreed upon but not necessarily implemented, and new features (like a dynamic loading system we could feel comfortable stabilizing) are proposed as PRs against that repository.
As a non-core person, I'd guess on the order of 1–2 years before there's a dynamic loading system in std or a blessed rust-lang crate which is required to work on all tier 1 platforms.
Awesome! Thanks for the info and sorry again for hijacking the thread. It seamed to make more sense to ask here rather than opening a new one.
Also you don't need the vtbl if you don want to
https://github.com/emoon/ProDBG/blob/master/api/rust/prodbg/src/view.rs#L72
As the compiler will generate a function (generic) for each of these there is no need for extra indirection.
Can I do the following ?
Implement a dylib in rust with a function without name mangling. This function call other functions which are mangled (norma rust functions).
So the plugin itself is a normal rust library but has one unmangled symbol as "entry point" of the interface.
And one more question about traits: can we write something like this?
dylib:
use std::sync::Arc;
#[no_mangle]
pub fn func(t: Arc<TestTrait>) {
t.messenger().say("omg!");
}
pub trait Messenger {
fn say(&self, s: &str);
}
pub trait TestTrait {
fn messenger(&self) -> Arc<Messenger>;
}
binary:
extern crate dynamic_reload;
use std::sync::Arc;
use dynamic_reload::{
DynamicReload,
Lib,
Symbol,
Search,
PlatformName,
UpdateState,
};
pub trait Messenger {
fn say(&self, s: &str);
}
pub trait TestTrait {
fn messenger(&self) -> Arc<Messenger>;
}
pub struct EchoMessenger;
impl Messenger for EchoMessenger {
fn say(&self, s: &str) {
println!("EchoMessenger: {}", s);
}
}
pub struct Woohaa {
messenger: Arc<Messenger>,
}
impl Woohaa {
pub fn new() -> Woohaa {
Woohaa {
messenger: Arc::new(EchoMessenger{}),
}
}
}
impl TestTrait for Woohaa {
fn messenger(&self) -> Arc<Messenger> {
self.messenger.clone()
}
}
fn main() {
let w = Arc::new(Woohaa::new());
w.clone().messenger().say("TEST");
let mut reload_handler = DynamicReload::new(Some(vec!["target/debug"]),
Some("target/debug"),
Search::Default);
match reload_handler.add_library("dll", PlatformName::Yes) {
Ok(lib) => {
let fun: Symbol<extern "C" fn(Arc<TestTrait>)> = unsafe {
lib.lib.get(b"func\0").unwrap()
};
println!("here is ok");
fun(w.clone());
}
Err(e) => {
println!("Unable to load dynamic lib, err {:?}", e);
return;
}
}
}
Personally for me that does not work but I don't know why. I have a error during calling the function:
error: Process didn't exit successfully: `target/debug/dl-test` (signal: 11, SIGSEGV: invalid memory reference)
Your immediately problem is that you're declaring func as extern "Rust" (this is the default if there is no other ABI spec), but trying to call it through a function pointer typed extern "C". The caller puts the Arc<TestTrait> trait object (two words: the data pointer, and a vtable) on the heap and passes a pointer to said pair in %rdi, while the callee expects the data pointer in %rdi and the vtable in %rsi. (This is on Rust 1.9.0, x86_64-unknown-linux-gnu).
If I change extern "C" to extern "Rust" in the binary, it works fine:
$$ cargo rustc --bin vityafx -- -C prefer-dynamic
$$ cargo rustc --lib -- -C prefer-dynamic --crate-type dylib
$$ cp target/debug/libvityafx.so libdll.so
$$ rustup run stable $PWD/target/debug/vityafx
EchoMessenger: TEST
here is ok
EchoMessenger: omg!
That said, you're still on thin ice regarding rust compiler versions (the layout of trait objects could change at any time, and I don't think having two definitions of the trait is quite kosher; I'd rather have those trait definitions exported from a third crate so there is only one real definition of each).
Okay, I got you then. And thanks a lot for the explanation! One question more - earlier you told that rust will not support plugin normally because it does not have stable abi. When it become stable?
A long-term stable ABI for the platform as whole may never happen. The Glasgow Haskell compiler still seems to require rebuilding the world after major version updates, for instance.
In a few years, you might see one or more of the following, depending on community priorities:
-
A reproducibility guarantee: if Alice and Bob both build crate X from the same source code with the same version of the compiler, their generated
.sofiles are guaranteed to be interchangable subject to $DOCUMENTED_CONSTRAINTS. (I think we already have this, minus the "documented" and "guaranteed" parts.) -
Documented points in time when ABI breaks can occur. Currently any nightly checkin is allowed to break the ABI; I can imagine a future where ABI breaks are only allowed on an N-month schedule.
-
ABI guarantees for highly restricted environments, such as
no_std. (Cross-module inlining is the largest source of possible ABI issues, and a smaller library is better positioned to make promises.)
3 Likes