FFI: Crash when trying to use the autoitx c++ lib on Windows

help
1

In Go I was able to link to it without swig by removing the default arguments in AutoItX3_DLL.h (you can use this patch (you may need to convert the lines' ending)).

It seems the DLL is found, according to the 'Process Monitor' tool.

I'm able to use the same lib and DLL with Go. I'm wondering if there's a bug. Any ideas?

To download the lib: autoit-v3.zip.

versions

Win 7 64-bit

C:\test-autoit>rustup show
Default host: x86_64-pc-windows-gnu

stable-x86_64-pc-windows-gnu (default)
rustc 1.11.0 (9b21dcd6a 2016-08-15)

code

use std::os::raw::c_int;

#[link(name = "AutoItX3_x64_DLL")]
extern "system" {
	fn AU3_MouseMove(nX: c_int, nY: c_int, nSpeed: c_int) -> c_int;
}

fn main() {
	unsafe {
		AU3_MouseMove(0, 0, 10);
	};
}
2

extern "stdcall" on x86_64? This might be relevant: Was there a recent change to extern "C" functions in nightly Rust? - #11 by retep998

3

Most of the abis in this list are self-explanatory, but the system abi may seem a little odd. This constraint selects whatever the appropriate ABI is for interoperating with the target's libraries. For example, on win32 with a x86 architecture, this means that the abi used would be stdcall. On x86_64, however, windows uses the C calling convention, so C would be used. This means that in our previous example, we could have used extern "system" { ... } to define a block for all windows systems, not only x86 ones.

https://doc.rust-lang.org/book/ffi.html

I think you're right and I should be able to use "C" or "system" but I still have the crash.

4

Using extern "stdcall" on x86_64 is totally valid and will simply result in the same calling convention as "C" "system" "fastcall" and "cdecl". The only convention that is different is "vectorcall". On 32-bit however, it is hugely important that you use the right one.

5

I'm not sure how important this is, but did you remember to call AU3_Init?

6

Yes I tried with or without it. Doesn't seems to help.

7

I also tried with nightly and an old rust version from last year.

I'm thinking about maybe comparing the generated assembly (rust vs go) and maybe creating a simple c++ lib to test.

8

Are you sure that the library is built in such way that it will work with the gnu version of Windows? Maybe testing the msvc one is worth a shot.

9

Maybe testing the msvc one is worth a shot.

I don't have the crash with stable-x86_64-pc-windows-msvc but I also don't have the crash with a C application built using mingw.

You can build it with the modified header file (with the default values removed).

#include <stdio.h>
#include "AutoItX3_DLL.h"

int main()
{
	AU3_MouseMove(0, 0, 10);
}