Problems building and linking to DLLs


#1

Hello,

I have encountered a few problems trying to build and link against DLLs under Windows. My goal is to have a project in cargo to build both a program and a DLL. The program depends on the DLL as a resource but does not statically link against it. In fact, it will only be loaded dynamically. However building a DLL through rustc/cargo will generate a DLL that exports every single public symbol from every linked crate (over 13000 symbols), creating a huge bloated DLL that takes a long time to build. Not to mention 98% of the code is never used at runtime. My first question is: is it somehow possible to tell rustc which symbols should be “dllexport-ed”?

I have found one workaround:

  1. Move the DLL code to a separate cargo package.
  2. Build this package as a static library. (crate-type = ["staticlib"])
  3. Manually invoke the linker and specify which symbols I want exported (through a DEF file for MSVC).

The problem with this approach is that cargo does not have a way to define a post-build script. I could make a simple Makefile or batch file that first builds the static library through cargo and then links it, but there is no way to invoke this build step from cargo. Of course I can invoke the Makefile/batch file from the main project’s build script and create a “fake” dependency this way, but the main project’s build script will not be rerun if code inside the DLL package changes. This happens even if the DLL package resides in a subdirectory of the main project.

Alternatively, if I were to accept the bloated DLL, I would at least like to reuse some of the code that is shared by the DLL by linking to it. However, specifying: crate-type = ["dylib"] in Cargo.toml and then having the program link against the crate leads to the following error:

error: cannot satisfy dependencies so `std` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `core` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `collections` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `rustc_unicode` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `alloc` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `rand` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `libc` only shows up once
help: having upstream crates all available in one format will likely make this go away
error: cannot satisfy dependencies so `alloc_system` only shows up once
help: having upstream crates all available in one format will likely make this go away

#2

Right now DLLs produced by Rust aren’t really designed to be simple DLLs that export a C ABI, since the DLL is crammed full of metadata and exports not only every public symbol but also every symbol referenced through publicly accessible symbols (or symbols transitively referenced through such) that can be inlined or monomorphized so it can be linked to in the Rust way. This is kinda awful and there seems to be no real strong desire from the Rust team to improve this situation at all, or well any of the linking situation on Windows really.

Regarding that later error, mixing dependencies on rlibs and dylibs is a really bad idea as you have to fully understand which binary is which providing which symbol, which library to link into which library and how to link it, and the worst part is that after you’ve figured that all out Cargo doesn’t give you any way to express that. The only method that Cargo seems to support is building and linking to everything as an rlib with only the very final result being something else. Coming up with a proper solution to allow linking to libraries as dylibs instead of rlibs will also require Rust properly understanding how to use dllimport and dllexport between crates which it currently is incapable of doing so properly and there has been very little movement from the Rust team on coming up with a solution to that either. It’s really weird, Rust dylibs are chock full of metadata and bloated with every possible symbol just so they can be used as a normal Rust dependency yet Cargo is absolutely abysmal at giving you the options to properly link to a dependency as a dylib instead of an rlib.

I’m sure the Rust team is all quite busy with other things that they think are more important than the basics of linking, but it really is frustrating for me to just see so little progress on this front. I feel as if the only way progress will be made is if I create RFCs for everything, and then slap everyone with it repeatedly, and once they’re accepted write the implementations myself.

tl;dr Keep using your workaround and I ranted a bit.


#3

Oh actually, it seems like there is something going on!

Behold, a separation of the two kinds of dylibs so we don’t get stuck with an unhappy compromise!


#4

Yeah I had a similar topic here Dynamic linking of SharedLibs with Cargo so I’m very happy about this RFC.


#5

Thanks, that’s good news. I read the RFC and I think it’s exactly what I need.

I’m still really confused as to what cargo produces when specifying dylib as output and how this output can even be used. It seems to link all dependencies (including std) statically when creating a dylib, but there is no way to link against such a dylib crate with cargo as it would always seem to lead to symbol duplication (e.g. the error above). When linking to a cargo produced dylib, is there no way to tell cargo to try to resolve symbols against dylibs first and only statically link against rlibs when necessary? I assume this is what you meant by:

The cargo documentation, when talking about the crate-type attribute, says:

The available options are dylib, rlib, and staticlib. You should only use this option in a project. Cargo will always compile packages (dependencies) based on the requirements of the project that includes them.

How can I specify such requirements in my project? For example, how do I tell cargo that my dependencies should be built and linked against as dylibs? Am I missing something?


#6

I have the same issue – using “rdylibs” in another crate seems impossible now. I don’t think it’s Cargo’s fault, as just using rustc --extern foo=libfoo.so seems to generate the same kinds of errors. It is a little bothersome, since this is the exact approach that works best for my project.