Build plugin that uses exports from dynamically linked core

Hey all, I'm trying to build a plugin system for my application and not having much succcess. I have a core library that contains structs, implementations, functions and macros that contains most of the logic for the application. This core is consumed by multiple clients such as a Tauri desktop application and a CLI. I also have a plugin that includes the core as a dependency by referencing its file path, and has crate-type = ["dylib"] in its Cargo.toml. Currently I am able to build the plugin, load it into the core with libloading, and call functions in the plugin. However, I have noticed that the entire core library is being included in the plugin's dylib, since it's a regular cargo dependency, and thus the core is being loaded into memory multiple times. Instead, I would like the plugin to include all of its dependencies except for the core, and then when the plugin is loaded by libloading, it uses the structs, functions etc. from the already loaded & running core to do whatever it needs to do. I've done some research and experimenting with different crate types and adding the -C relocation-model=pic rust flag, but have had no success.

If I was doing this in C, I'd create a header file with an outline of the core functionality, link to this header in the plugin, load the plugin at runtime and then the plugin would reference the symbols from the header already loaded into memory, without bringing its own build of the core with it. Obviously Rust doesn't have header files, so this is a bit trickier, and I'm not sure how to go about it.

I'm aware that building a plugin system like this could be error prone because of ABI differences across Rust versions, I would just like to do it as a proof of concept since I think I can standardise the rustc version I use in CI. Thanks for any help!

Having done weird Rust-C interop before, I would make the "core" look like it's written in C. Write that header file you mentioned. You can use cbindgen for that.

Once that is done, you can use bindgen to create the Rust external imports in your plugins.

It seems silly, I know. But in my experience, that is the way to get compatibility, make sure your tools make the right assumptions, and avoid lots of problems.

Good luck, and happy new year!

2 Likes

I tried this suggestion out right after you mentioned it, and spent some time separating parts of my project to be made into C-compatible APIs, but in the end decided to try and go back to using pure Rust since making custom data types and async functions FFI-compatible is an absolute nightmare. With that said, for most use cases a C compatible API is likely the best solution, and so I'm going to mark your suggestion as the accepted answer.

Surprisingly, I've managed to make the package system I wanted without any C bindings or compiler weirdness (ignoring the fact that each package needs its own Tokio runtime). I've leaned into message-passing much more than attempting to share memory, and it's allowed me to separate a good chunk of my project into a lean package-api that is included by each package, and then a core that adds features on top.

It's not quite the solution I detailed in the original question (eg. I still have to include the package-api code in every package), but it's working well enough that I'm willing to deal with some larger binary sizes. The project is MacroGraph btw, feel free to check out what I've done!

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.