Returning values from Rust to C++


#1

I’m writing an application that is basically a simple C++ shell calling into a Rust library. All computing would happen in Rust; however I got stuck returning a complex data type from Rust.

Given the following simple data structure:

// lib.rs
#[repr(C)]
pub struct Vec2 {
    x: f32,
    y: f32,
}

// main.h
struct Vec2 {
    float x;
    float y;
}

Calling into Rust works:

// lib.rs
#[no_mangle]
pub extern "C" fn vec2_get_x(v: &Vec2) -> i32 {
	v.x as i32
}

// main.cpp
typedef i32 (*vec2_get_x_t)(const Vec2& v);
auto vec2_get_x = (vec2_get_x_t) SDL_LoadFunction(library_handle, "vec2_get_x");

float result = vec2_get_x(Vec2(5, 3));
printf("x: %f\n", result); // Prints "x: 5.00000"

But returning a Vec2 from Rust fails:

// lib.rs
#[no_mangle]
pub extern "C" fn init_vec2() -> Vec2 {
	Vec2 { x: 3.0, y: 5.0 }
}

// main.cpp
typedef Vec2 (*init_vec2_t)();
auto  init_vec2 = (init_vec2_t) SDL_LoadFunction(library_handle, "init_vec2");
Vec2 result = init_vec2(); // This crashes

What am I missing? I think this has something to do with lifetimes and passing values on the stack, but I can’t figure it out. I’ve tried to search for this but all FFI tutorials I’ve seen just return simple integers.


#2

Are you on Windows? If so, you may be running into this nasty issue: https://github.com/rust-lang/rust/issues/38258


#3

I got it to work on Mac. :smile:

I’ll try increasing the struct size by adding random stuff, and see if that helps.


#4

I just tried again on Windows. This works:

#[repr(C)]
pub struct Vec2 {
    x: f32,
    y: f32,
    z: f32,
    w: f32,
}

#[no_mangle]
pub extern "C" fn bloob_init_vec2() -> Vec2 {
	Vec2 { x: 3.0, y: 5.0, z: 42.0, w: 88.0 }
}

But this doesn’t:

#[repr(C)]
pub struct Vec2 {
    x: f32,
    y: f32,
}

#[no_mangle]
pub extern "C" fn bloob_init_vec2() -> Vec2 {
	Vec2 { x: 3.0, y: 5.0 }
}

So it look like it’s caused by the Windows issue you mentioned.

Thanks for helping me debug that! I was pulling my hair out. Is there a way to work around it without adding dummy fields? (Note that in real code, I will probably work with larger / compound structs, so this shouldn’t be too much of an issue.)


#5

Since the issue only affects struct returns, you could change the signature to use a pointer instead:

pub extern "C" fn bloob_init_vec2(vec: &mut Vec2) {
  /* fill in vec */
}

#6

Hmm - I tried to reproduce your crash, but couldn’t. When I compile your init_vec2 (with nightly Rust on x86_64-pc-windows-msvc), the generated code just does

mov     rax, 40A0000040400000h
retn

which can’t crash.

Looking into the issue more, it seems to apply to non-POD structs on the C++ side which Rust treats as POD, not the other way around. If that were the issue, though, I’d expect garbage return, not a crash. And if the C++ version of struct Vec2 in your post is really what you have, it should be POD.

I may have misdiagnosed the issue entirely. Are you sure you’re using the right Rust compiler variant? i.e. msvc not mingw (assuming you’re using MSVC to compile your C++ code). actually, that shouldn’t cause this type of crash either on 64-bit, not sure about 32-bit.


#7

I’ll try sending you the code as a Gist, so you can look at it (I’m not near my Windows machine at the moment, so give me some time).


#8

In trying to reduce it down to the bare minimum, I’ve removed the bug:

My real code does quite a bit more: loading OpenGL, initializing shaders, etc. However all of that happens after the test code is run; so it could be something with the binary itself, or the included headers, or…

I’ll keep investigating. Thanks so much for your help!

I’ve tried both with stable-x86_64-pc-windows-msvc and stable-x86_64-pc-windows-gnu.


#9

After some more digging, I found that the issue comes up if I add a constructor to the Vec2. The Vec4 with constructor keeps working fine, but the smaller data structure no longer works.

What could be the reason for this? Is it because the data structure is no longer a POD and the data layout is different?


#10

Actually, adding a non-trivial constructor or destructor changes the C++ ABI – it forces the parameters and return values to be passed through a pointer instead of in registers. You’re not seeing this behaviour for Vec4, because bigger structures are always passed behind a pointer anyway.

I think that only adding custom copy- or move-constructor will change the ABI, other constructors won’t change it, but I’m not sure about that.


#11

@fdb I’m not sure if it makes a difference but strictly speaking your typedef should be extern C.


#12

Here’s the exact rule:

To be returned by value in RAX, user-defined types must have a length of 1, 2, 4, 8, 16, 32, or 64 bits; no user-defined constructor, destructor, or copy assignment operator; no private or protected non-static data members; no non-static data members of reference type; no base classes; no virtual functions; and no data members that do not also meet these requirements. (This is essentially the definition of a C++03 POD type. Because the definition has changed in the C++11 standard, we do not recommend using std::is_pod for this test.)

https://msdn.microsoft.com/en-us/library/7572ztz4.aspx

So, yeah, it’s the issue I linked after all, which is about non-POD types on the C++ end. I was just getting tripped up because you said it crashed, whereas I’d expect the call to return garbage to C++, but maybe that caused a crash later in the program or something. (And because you omitted the constructor in your original post :slight_smile: )


#13

Now I know why I thought that only copy-and move-constructors affect the POD-ness of the type – the System V ABI, which is used on *nixes indeed doesn’t care if you defined a regular constructor or not. That explains why @fdb had it working on Mac.

The System V ABI also differs on where the floats are returned (in XMMs, not RAX) and on what’s the max struct size for passing in registers (16 bytes (or even 32 when using vector types) instead of 8). The x86_64-pc-windows-gnu is really curious mix of both ABIs, let me show an example:

struct Vec2 {
    float a, b;
    Vec2(float a, float b) : a(a), b(b) {}
};

Vec2 foo() {
    return Vec2 { 1, 2 };
}
  • x86_64-pc-windows-msvc returns a struct through a pointer
  • x86_64-pc-windows-gnu returns a struct in RAX register (like msvc without constructor)
  • x86_64-unknown-linux-gnu returns a struct in XMM0 register.

#14

I need constructors on the C++ side of these objects as well. I want something that works reliably on all platforms (and that I can explain to other members of my team without explaining different ABIs). I’m leaning towards passing a pointer to an object I pre-allocated in C++ (on the stack or heap) like this:

// C++
Vec2 my_vec;
vec2_init(&my_vec);

// Rust
pub extern "C" fn vec2_init(vec: &mut Vec2) { ... }

@comex: I had it crash quite reliably just by accessing the value (maybe because of guard pages in C++ debug mode? Idk) but I remember I got some garbage data in one of my tests. I also didn’t think of including the constructors, because I didn’t realize that would make a difference (ha!).


#15

The call convention for returning big or non-POD structs is (from the link provided by @comex):

Caller allocates memory for Struct1 returned and passes pointer in RCX,
callee returns pointer to Struct1 result in RAX.

So callee basically has to copy RCX to RAX (I think this copy is needed because RCX is considered scratch register and caller cannot assume that RCX will have its original value after the function call). Rust assummess PODness and stores the struct itself in the RAX register. If afterwards the caller treats RAX as a pointer, it results in accessing “random” address in memory. I guess this may be the reason of your crash.

Anyway, I think your approach with passing &mut Vec2 will be the best considering the platform compatibility. *mut Vec would be more accurate, although I don’t see any reason not to use &mut Vec2 here. Also, note that when passing Vec2 as argument, you should probably pass &Vec2 if you don’t want to be bitten by C++ constructors changing ABI.

Another approach would be to treat Vec2 in C++ as a “foreign” type and use it only when calling Rust functions. In the rest of code, use some other Vec2 type with appropriate conversions defined. This way C++ will always consider Vec2 as POD type, and you’ll avoid all the pointer indirections in function call (although it’s really unlikely that the function call overhead will affect the performance of the whole program).