I'm attempting to call into a C wrapper function which wraps a call to OpenCV.
I've observed a case where a function returning the type
cv::Vec3b results in the first argument seemingly being treated as the return value and the other arguments getting shifted left one position.
cv::Vec3b's size as well as my Rust struct are both 3 bytes in size, so I'm not suspecting any issues on that front. It's possible I'm doing something else incorrect, but I've been unable to track it down.
See runnable code (required OpenCV 2.4) and more info here: https://github.com/ryandbair/rust-ffi-return-bug
When you look at the assembly of the C++ code (I'm using release mode, so the code is cleaner):
objdump -dC target/release/build/rust-ffi-return-bug-9c1a9f48b8dd0024/out/native/test.o
you'll see that
call_me2, which returns
Bytes<char, 3> squeezes the whole returned struct in the
%rax register, but the
call_me3, which uses opencv's
Vec3b returns the struct in a location described by a hidden pointer passed as the first argument (that explains the argument shifting).
The behaviour of
call_me3 is really surprising for me, because small structs are passed in registers in C (on
x86_64). The only explanation I can think of is this (from http://refspecs.linuxfoundation.org/elf/x86_64-abi-0.95.pdf, p.27):
If a C++ object has either a non-trivial copy constructor or a non-trivial destructor, it is passed by invisible reference (the object is replaced in the parameter list by a pointer that has class INTEGER)
It's really suspicious why the
Vec/Matx types in opencv would need a non trivial constructors though. Edit: It seems that is in fact the case! You can check it by printing
std::is_trivially_copyable<cv::Vec3b>::value (it's 0). So now I wonder, how the Rust bindings to opencv have been working at all? Unfortunately, I couldn't make them compile, so I haven't tested them. Anyway, Rust can only use
#[repr(C)], which assume register-representation so I have no idea how to work with such structs as
Also, two nits about the code:
libc = 0.1 is old and sometimes incorrect, use
char signedness depends on architecture, so you shouldn't use
u8 to represent it. Either use
libc::c_char, or if you want to use
u8 in Rust, use
uint8_t on C++ side (or even
unsigned char, although that's theoretically incorrect).
That excerpt from the C++ specs shines a lot of light on the subject.
Most of the OpenCV wrapper works by returning pointers to C++ objects. In this case I was hoping to return Vec types directly as they're all pretty small.
A little more research on the C++ specs also indicates that returning C++ classes from extern C functions is undefined and may violate the C ABI which is what we are seeing.
From StackOverflow quoting the C++11 draft (7.5.9):
Linkage from C++ to objects defined in other languages and to objects
defined in C++ from other languages is implementation-defined and
language-dependent. Only where the object layout strategies of two
language implementations are similar enough can such linkage be