Has anyone else in the Rust community tried to fix unloading of dynamic libraries? for example, memory leaks due to the thread-locals (on linux), leaked memory in static s (they dont even have destructors in runtime)
I also tried webassembly (wasmtime, wasmer) but they don't work well with multi-threading in wasm modules (I want be able to run normal Rust programs with multi-threading and reload them in runtime without restarting executable)
So, based on fasterthanlime article I made this solution: GitHub - xxshady/dylib-linux-reload at main2 which tries to fix memory leaks caused by thread-locals (I'm not sure about all the unsafe stuff written there, but it seems to be doing it's job, at least there are no memory leaks according to valgrind)
thread-locals is only one issue of all, there are others:
(at least the ones I know of)
memory leaks due to the use of statics or std::mem::forget (can be solved by tracking allocations within the library using #[global_allocator] and deallocating leaked memory manually before unloading the library)
detached threads preventing library from unloading (for example, forgotten std::thread::spawn(...))
panics (not sure if it's possible to handle them before crashing the executable)
FYI there are some informations in this (seemingly unrelated) internals thread on why this is incredibly hard to do safely. For this to work properly you effectively need all your dependencies (including the stdlib!) to support this usecase, which is very very hard to guarantee.
This is an interesting topic, and some languages built around this idea show can be made to work (see e.g. Erlang and other languages running on the Beam VM).
Could it be applied to Rust (or other native languages) though? Not having a managed runtime makes things much harder for sure.
I do not believe thread locals is properly fixable. In fact, avoiding all forms of global state is likely your best bet. Yes there are some interesting hacks you can do as workarounds, but I'm not keen on them and would prefer to fix it properly.
One thing that would be needed for safety is to check that types are compatible. Here we can look to stabby - Rust and abi_stable - Rust are two libraries that take on the stable ABI issue for plugin APIs. As I understand it they will compare ABI and API at load time to make sure things are compatible.
Something like that is absolutely a prerequisite for safe reloading without a managed runtime, but now you also need to compare that your types internal to the library are compatible before and after the reload (or that all instances have been freed).
You could also imagine defining data migration rules like Erlang has to upgrade the state to a newer version. However, in a non-managed language there isn't a good way to know that no instances of custom data structures defined by the plugin have escaped it (or where they ended up and need migrating). This alone is a blocker to safe reloading of plugins (and even worse for the general code that isn't just plugins use case) if all data types and function signatures don't match exactly.
WASM does solve this issue (by making the isolation border clear) and I think the best bet here is to work on improving WASM so it works for more use cases.
So yes, you could do it natively (for small code changes that don't change any data types or signatures), but WASM is a better bet.
Not sure if it's even possible to avoid thread locals and statics in today's Rust, even std's println! uses thread-local, log crate uses static under the hood, so it's everywhere and you have to deal with it if you want people to be able to build their projects with normal Rust
Depending on what you want to achieve with reloading there might be some ways out:
For rapid iterative development we could make do with the risks, if things crash it isn't the end of the world. This makes sense in a library, but is extremely unlikely to get accepted by the main rust project.
If your goal is to apply security fixes without downtime there are a few options:
Serialise your state, exec() the new binary and reload it. At least on Unix you can pass on the open file descriptors etc. I believe systemd does this for the main init daemon. Some servers also support this. You can do this (with additional trickery) so that you don't stop accepting new connections at any point.
Load a module that live patches your code in memory. Obviously unsafe, but perhaps slightly less so. This is used for the Linux kernel itself to support runtime patching without reboot.
If you design from the ground up for reloadable modules it can be done (see e.g. the Linux kernel again). But it isn't safe (in the Rust sense), a poorly written module can easily screw things up.
For the narrower problem of thread locals:
One way out could be go forbid thread locals in the loadable modules, they have to link to the same tokio or tracing instance that the main program links (also linked dynamically). This way the thread local doesn't exist in library. The lack of stable ABI is a problem for this but for some use cases (e.g. development) having all shared libraries exist in the same workspace could be ok.
Again though, wasm is the better option (and it might be better to work on improving wasm).
I also agree with the point about wasm, it's the best way currently, but if you want to use some features thats only available in native Rust (currently) you can't just use it (it's really cool technology though, I really like the wasm component model)
for example, it might not be suitable if you want run your code natively on a server for a multiplayer game