Wrong calling convention with a Wine DLL?

help
1

I need to call Linux native code written in C from a Rust process running under Wine. I set it up as follows:

  • C code produces a Wine DLL compiled with winegcc.
  • A Rust library dynamically loads the DLL and forwards function calls to it.
  • A Rust binary just directly calls the Rust library.

It works in simple cases but I also hit function signatures where parameters are passed into the C DLL with mangled values. Could it be that code is using wrong calling convention and passing parameters the wrong way?

Details:

  • cargo 1.67.1 (8ecd4f20a 2023-01-10)
  • rustc 1.67.1 (d5a82bbd2 2023-02-07)
  • 64 bit Arch Linux and 64 bit Wine

Demo code: https://github.com/skligys/RustWineDllCall

Output:

----- inside main()
----- inside lib_function(1, 0)
----- inside lib_dll_function(0, 2227184)
----- lib_function() returned true

Any ideas?

2 
bool lib_dll_function (unsigned int arg1, unsigned int arg2) {

Based on the name of the C file I presume you're compiling it for a 64 bit processor.

struct DllLoader {
  lib_dll_function: unsafe extern "C" fn(arg1: c_uint, arg2: c_uint) -> bool,
}

This type will almost always be u32...

Is that enough of a hint?

3

Yes, I am, sorry about not mentioning it.

Could you please elaborate, I don't see a problem. According to https://en.cppreference.com/w/cpp/language/types, unsigned int is 32 bits long in C both on 64 bit Linux and 64 bit Windows. So it seems to me types line up nicely, no?

4

Huh. Sorry about that. Apparently it's been a great while since I used int.

5

Disclaimer that I don't know much about wine, but I was suspicious that the DLL would have symbols which were callable with the Linux ABI. This StackOverflow answer appears to indicate that the function pointer you retrieve will indeed be expecting the Windows ABI and not the Linux ABI that Rust will use by default

6

But the caller Rust binary and library are built under Windows 64 bit cross-compiler (see https://github.com/skligys/RustWineDllCall/blob/main/.cargo/config.toml), and I verified the binary is as expected:

$ file bug_runner.exe
bug_runner.exe: PE32+ executable (console) x86-64, for MS Windows, 24 sections

The Wine DLL is a Linux shared library but in my understanding Wine handles such cross-ABI calls, as this is how it converts Windows API calls into native Linux calls.

$ file libbug64.dll.so
libbug64.dll.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=a5d5f1f392d3069a23a0361cb288b0977fa82220, with debug_info, not stripped

$ file libbug64.dll.fake
libbug64.dll.fake: PE32+ executable (DLL) (console) x86-64, for MS Windows, 2 sections

BTW, I just found this blog about how Wine handles cross-ABI calls: https://blog.hiler.eu/wine-pe-to-unix/. Looks that I need to somehow invoke _wine_unix_call() for cross-ABI calls to work? Then probably it doesn't happen all automagically after all, will look at code links from the blog and try to implement something in that vein.

1 Like
7

If your main program is building for Windows, can't you just build a normal DLL rather than building the wine .so?

8

If your main program is building for Windows, can't you just build a normal DLL rather than building the wine .so?

I am trying to access a USB device that Wine cannot handle. I know how to use it from Linux, so I was hoping this bridge code from Win64 into Linux will help me.

9

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.