Is it possible to force dylib build in toml when specifying external dependencies of the package? I am trying to build plugin-based application, that supports hot-reloading. In c++ plugins that link static libraries have larger sizes and increased linkage time, and I need hot-reloading to be as fast as possible.
PS. I have all of my crates with:
[dependencies]
[lib]
crate-type = ["dylib"]
If I want to publish some of my crates is it acceptable configuration(I inspected a dozen github projects and none of them specify crate-type)?
No.
Rust doesn't have a stable ABI so if you wanted to use (for example) serde in your application cargo has no choice but to rebuild it from source and pass the newly compiled serde, and your crate's *.o files, to the linker.
Dynamic libraries must typically be used through a C FFI based api.
Can you clarify how unstable ABI connected to inability to specify how you want to build dependent package? It was decided not to provide these options because ABI is still unstable? If I can build dynamic library by hand(without cargo), it seems like a missing features, that I cannot do it with cargo
This might incidentally work, but it is not guaranteed to, nor is it officially supported. The Rust ABI is fully unstable, and it is not designed to be used across dynamically linked boundaries.
For instance, you might run into issues of the same struct, compiled in two different crates, having a different layout. Unlike C with its guaranteed layouts, Rust structures are only guaranteed to have the same bitwise representation when compiled the same. I think you might be able to work around this by ensuring all your dynamic libraries are compiled with the same version of the compiler using the same compilation flags, but again, it's not designed for this. I'd recommend using abi_stable or #[repr(C)] structs whenever passing things across dynamic library boundaries.
The other big problem is multiple allocators. If you try to dynamically link rust libraries, each library will I believe end up with its own allocator instance. This is a problem as rust code assumes that rust things, like Box, have been allocated under its own global allocator. But if you are dynamically linking a library, the Boxes and Vecs from that library will be allocated in that library's allocator, so trying to deallocate them in the downstream crate will crash.
I guess what I'm trying to say is that this is not a supported setup, with or without cargo. These problems are not insurmountable - we could fix them. But to my understanding, this has neither been a design goal of rustc nor of cargo, and so they remain issues if you try to use them like this.
2 Likes
Support of dylib crates in Rust is so unfit for general use that I have proposed an option to disable it in the compiler for use in software integration projects.
One of the very few places where dylib crates are used on a stable basis is the Rust compiler, and it can afford this by installing the libraries it was built against as part of the toolchain. Your application might do the same, but this somewhat defeats the purpose of shared libraries and may result in more code image bloat than linking statically.
The above does not concern cdylib, which is a way to provide dynamic libraries implemented with Rust, but exporting a C-compatible ABI.
2 Likes
Thanks everyone for your replies! Probably, I should provide more background on what I am trying to achieve. I am a rust newbie, with c++ gamedev background. Right now, I am trying to research if it is possible and what are the downside of developing plugin-based game engine in Rust. Basically, plugin would export a number of plain structs(no traits, no dynamic dispatchers and etc) and a number of callbacks. Everything complex will be hidden inside the plugin and not exported. Struct and callbacks will have to be registered through some kind of runtime reflection, but that is a whole another topic.
Plugin changes must be "consumed" by the engine, even layout changes(runtime registration will serve as a state transfer functionality). So even if ABI was stable, it would not help me.
Correct me if I am wrong, but multiple allocator problem happens when some libraries statically link Rust runtime and some dynamically? I do not think it is a language specific problem, since c++ would have issues with this setup too. Anyway I would build everything with prefer-dynamic flag.
I was thinking that I could "reuse" cargo plugin package manager, storing them in cargo registry and and pulling "third-party" libraries from crates.io. So, is there any way to use cargo and build dependencies as cdylib or dylib? Maybe I can use some custom build scripts with cargo?
I'm not that familiar with the topic so maybe this isn't entirely relevant, but I thought you may find Mun (a Lua type language written in Rust) of interest.
This might be useful to read Michael-F-Bryan
1 Like
Unfortunately, each cdylib plugin links all its Rust dependencies, including std, statically. Unless the compiler option -C prefer-dynamic=yes is given, in which case you are back to the ABI issues discussed in this thread.
This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.