Appropriate ABI for Windows _setmode

What's the appropriate ABI for declaring _setmode | Microsoft Docs on Windows? I would expect it to be either extern "C" or extern "system", but I'm not sure how to determine which is right in this case.

Not completely certain, but I'd guess extern "system" since that's what winapi uses (though it doesn't have that function AFAICT)

Actually, I'm now convinced that extern "C" is right. Compare that docs page to e.g. GetConsoleMode, which has WINAPI in the declaration. (WINAPI is a macro that afaict just expands to __stdcall, equivalent to extern "stdcall" in Rust.)

The declaration in Wine's io.h also has __cdecl, not __stdcall. I think stdcall is just for things that are part of the Win32 API, and _setmode is more like a nonstandard C library function.

2 Likes

A workaround for this could be to write your own C function that calls _setmode, and compile it with the cc crate. The C compiler would know how to call _setmode, and you'd know how to call your wrapper.

2 Likes