Is that true in general? For instance, are shared dependencies supposed to also be dynamically loaded? I can see how this might play out well for dependencies that are designed to be used as shared libraries, but the existing crates ecosystem doesn't lean in that direction. Nor do the JavaScript or Python ecosystems.
If not for any potential benefits from shared libraries, I'm not quite sure where you are going with this in terms of control over dependencies.
Curious if you have measured the WASM overhead? It is amazingly fast and efficient for most purposes, very close to native. Built-in security sandbox with opt-in privileges is great. However if your plugins need full system access, (raw network I/O, full disk access, platform-specific or hardware direct access) then WASM might not be a good fit.
Also, I would look at how plugin systems have been implemented successfully in some other specific instances.
For example, the Language Server Protocol (LSP) is used for rust-analyzer and dozens of other languages to integrate with multiple IDEs. It seems to be a huge success in terms of flexibility and portability. It performs well. This is an interprocess communication protocol, an HTTP-like stream protocol between the host app and plugin process. Easy to test, language-independent.
Some useful discussion, of pro/con of in-process vs IPC and other options, in this Software Engineering Stack Exchange question.
You raise valid points. I should clarify that one of my ideas was to enable re-using a plugin's API in another plugin to reduce code duplication. However, I'm still hesitant about this approach, especially considering the arguments you've cited. It's true that this goes against the crates ecosystem, and it could potentially introduce unnecessary complexity.
I haven't conducted precise measurements, but after reviewing various resources (such as this series of posts), I concluded that WASM would likely introduce some performance overhead, which I really try to avoid since some operations could benefit from optimal speed. Additionally, my use case requires full system access for plugins, as they need to interact with the system and utilize technologies like hypervisors. Sandboxing isn't a priority for my project, which further suggests that WASM might not be the best fit for my specific needs.
Thank you for the resources and additional information! I've already examined some plugin systems (mostly proof-of-concepts with different techniques), but I'll definitely look into what has been done with rust-analyzer and the LSP. Although their model may not fit for my project, it's certainly a good reference.
Depends on the code in question: I'd like to pre-transcode media uploads with ffmpeg, but measurements are that it runs at about 4-8% (!!) speed
I would guess that highly tuned code with lots of CPU-specific SIMD/other acceleration features, like ffmpeg and its codecs, would be the worst case for WASM vs native performance, so not too suprised.
Not to mention GPU offloading fits under the definition of "other acceleration features".
WASM has many strengths. And while direct hardware access is not one of them, I think I agree with you in general that WASM performance overhead is mostly assumed to be much poorer than it is in practice.
I didn't have time to properly read through this thread, but I've spent a lot of time creating a scripting system for games, so it might be worth a look.
I created runtime schema system that works over #[repr(C)] data types that allows languages like Python or Lua ( so far only actually implemented Lua ) to read and write data in Rust types.
Note that this only really covers the data types side of things, not functions, so you still have to figure out how to do that.
In my use-case, it was for games, and I used an ECS to make the game so that once I had exposed the core ECS functions, almost everything else was controlled just by modifying data, that way I don't have to make new function bindings all the time.
I have a talk that gives an overview of how the schema system works:
Not sure if that helps in your case, but I figured I'd leave it here just in case.
Looks like you didn't get any responses on this. I just did a quick search of RFC titles and didn't find anything. If you want to pursue this, I suggest posting on the internals forum.
This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.