What I mean by 'force-kill' here is: even if the module we want to reload is (1) uncooperative or (2) infinite looping, we can still safely kill it and force reload.
To the best of my knowledge, the only way to do this is to have the plugins run in wasm. I.e. to do "rust code -> wasm binary", run wasm binary in wasm container. The wasm container then allows us to do force killing of infinite-looping / un cooperative plugins.
Besides lunatic, is there any other Rust setup that provides this ?
In short, I am looking for a way to (1) compile Rust to wasm32, linking against some API [hard part here: not compiling Rust to wasm32, but what lib to link vs ] (2) some container which then allows us to hot reload / force kill these Rust-wasm32 plugins.
My fault for forgetting to state this. I need to load many concurrent instances of the plugins, enough so that processes / threads are too heavy. I'm looking at things on the level of erlang processes, go routines, java light threads, etc ...
Running arbitrary, uncooperative code within the same process can't be terminated safely without leaking resources. A simple example is scoped threads. If another thread borrows from a stack frame that's supposed to be terminated (unwound/thread-killed) then you get UB if you clear or deallocate that.
Other languages solve these problems by having safepoints at which exceptions can be injected, not having stack-borrowing, having garbage-collectors that won't reap the resources if they're still in use by something else, restricting what code can do, etc.
But all those things are the code being "cooperative" (forced to be cooperative by the runtime or compilation model).
This absolutely can be done if we compile the Rust code to wasm and run it in a wasm container, thus limiting the blast radius. lunatic is one approach to this; I am asking if there are other approaches
note the "arbitrary and uncooperative". wasm is neither arbitrary code nor is it uncooperative. you're running the wasm runtime, not wasm code. The wasm runtime itself is cooperative and has hooks to terminate things.
The question was how to do it without things like lunatic, which - again - relies on wasm.
You could also run some x86 interpreter/emulator or a virtual machine, it would be a similar concept... you stop executing arbitrary code directly and interpose a known, cooperative entity running that code. These entities impose restrictions that prevent things like full shared memory access within the same process.
WASM has the benefit that the binary can be rewritten (with a lot of effort) to include any instrumentation you want. For instance, inject a "watchdog" counter into every loop to stop runaway processes. Check the wall-clock runtime and abort when the timeout expires. Add any resource limits; memory utilization, total network requests, number of concurrent DB connections...
This is the strategy we took with a product long ago to run untrusted code. It was savagely effective. In addition to instrumentation that you control, the process boundary also gives you other capabilities, like cgroups for resource quotas. Just wrap the whole thing in an onion of paranoid caution to add some trust to untrusted code.
The closest I could think of to what you're asking for here is Google NaCl - but again, that's a constrained VM, just where the VM happens to be compatible with a real machine. And also it's a nightmare, apparently?
If you load and execute arbitrary, "uncooperative" code into your process, that code has all the permissions your code has to the OS, so it's pretty tricky to pull off anything reliable.
It's pretty nice and simple technology which was crippled for political reasons.
After it was announced people, quickly, tried to use it to cook up replacement for JavaScript: Python, Tcl, etc.
That spooked “the powers that be” and NaCl was crippled for the first time: NPAPI (which made it possible to embed your plugin into browser) was replaced with PPAPI (entirely asynchronous and thus useless for people who just wanted to port stuff from desktop).
And then some other people on top discovered the fact that NaCl promised to make x86 binaries part of the web. That spooked them, too, and they mandated use of intermediate format. Since they had nothing suitable around they used LLVM bitcode as mandatory format for PNaCl (and they made NaCl not available on the open web).
This, of course made the whole thing really awful (Apple repeated that same mistake and Google also did with RenderScript and AMD and Kronos with HSAIL and SPIR.
If the pressure of these large companies would have managed to cripple LLVM and made is freeze it's IR… maybe NaCl wouldn't have sucked so much (and we would have had crippled LLVM instead).
But in our world... LLVM won, NaCl lost.
But technically there was nothing wrong with NaCl sandboxing. In fact some ideas from it are reused in most WASM runtimes.
Yes, so I think the two most popular wasm runtimes are wasmtime and wasmer.io .
I agree with you that a framework can be built on this (in fact, I think lunatic solutions uses one of them.)
I'm wondering if these is something between wasmtime/wasmer (where I have to build everything myself) and lunatic solutions (where I have to take the entire system).
Have a look at wit-bindgen. It provides a way to communicate with a WebAssembly module without needing to write code for manually reading values out of linear memory and so on. It follows a similar design to gRPC where you'll define the interface in a domain specific language, then the tool will generate (mostly) idiomatic code for the host and the guest that implements that interface.
The glue code it generates is only usable on Wasmtime and the project is still in development, but it's quite promising.
