Is #[repr(C)] necessary for Rust-to-Rust FFI

I have a project. Some code needs to run fast, other code I would prefer to be able to build it quickly. So I'm experimenting with splitting off the performance-sensitive code into dylibs and compiling it in release mode, then compiling the main project in debug mode and loading the functions.

Anyways, the compiler is complaining that my types, which the extern fns take, are not #[repr(C)]. However, I'm not using extern "C" functions, just extern ("Rust") functions, so it seems like it might be fine. On the other hand, if optimizations can change a struct's layout, that could cause issues.

Should I annotate my types with #[repr(C)]? Or is this dogmatic? And if so, is it valid to wrap external non-#[repr(C)] with #[repr(C)] newtypes?

warning: `extern` fn uses type `game::graphics::BlockVertexEditSubmitter`, which is not FFI-safe
  --> fastcode\src\lib.rs:26:21
   |
26 |     edit_submitter: BlockVertexEditSubmitter,
   |                     ^^^^^^^^^^^^^^^^^^^^^^^^ not FFI-safe
   |
   = help: consider adding a `#[repr(C)]` or `#[repr(transparent)]` attribute to this struct
   = note: this struct has unspecified layout

Yes, #[repr(C)] is necessary. A newtype is not enough.

1 Like

The trouble is that in Rust when a struct is repr(Rust) (the default) it is then free to use whatever layout it thinks is most efficient for your program. So this could (hypothetically) be different between two Rust programs and is even more likely to be different if the two programs are compiled with different versions of the Rust compiler.

So for FFI you need an explicitly specified layout. Currently C and transparent are the only stable layouts for structs. There may be more more in the future but that's currently a ways off.

1 Like

See:

2 Likes

Hold on, when you say a newtype is not enough, does that mean I couldn't do:

#[repr(C)]
struct FfiSender(std::mpsc::Sender<String>);

extern fn f(f: FfiSender) {}

It seems like, if I can't do that, then I would have to basically create my entire new set of types, since most code doesn't annotate all their structs with #[repr(C)]

@gretchenfrage indeed! One way to "quickly" become #[repr(C) is to use indirection (& and &mut usual, but Box for owned values):

extern "C"
fn f (f: Box<FfiSender>)
where
    FFiSender : Sized,
{
    let f: FfiSender = *f;
    // ...
}

And yet that may not be even enough, given that the side the defined the type is the only one allowed to interact with it in a non-opaque way. The other side of the FFI would need to use the opaque type pattern.


Basically, unless you are doing a bit of more advanced shenanigans through the aforementioned ::abi_stable crate, you should be thinking of your situation not as much as a Rust-to-Rust FFI, but rather Rust-to-C-to-Rust FFI, since C is the one providing a stable ABI.

In this fashion, if you use ::safer-ffi, you can create a #[ReprC::opaque] newtype that will let you use a repr_c::Box<_> to box it within #[ffi_export]-ed functions, all in a safe fashion.

The other side of the FFI, however, would then need to cbindgen the generated headers and act as if it was dealing with a C API.

It is very cumbersome, but it will be guaranteed to be stable and without mistakes thanks to ::safer-ffi compile-time checks.


Note that a completely alternative approach / ABI is to use serialization: pick the fastest {,de}serialization format you can think of, and have your ABIs use blobs of bytes / or strings as parameters, that get serialized / deserialized at will.

Basically you replace the #[repr(C)] problem with a #[derive(Serialize, Deserialize)] one, but the latter is more common than the former.

2 Likes

Interesting, thanks. I suppose it wouldn't be too hard to make this FFI safe, by defining #[repr(C)] boundary types with From and Into, and instead of passing a crossbeam::channel or something similar, pass a extern fn + context pointer tuple, which internally calls the crossbeam channel or whatever rust specific thing.

Although it does seem like this could be ameliorated if there was some compiler flag I could use that said "make every struct in every crate repr(C)". That's not a thing, is it?

That's not a great idea. It would break type layouts silently, without any notice in the actual code (and it would also result in non-reproducibility, were someone trying to build your code without cargo, etc.)

Generally, changes and settings that influence the behavior of the code, especially if they have global effects, should be explicitly stated inside the source code itself, not as implicit, magic parts of the build context.

As for your actual problem: given Rust's incremental compilation abilities, you can just split the code into multiple,
independent crates without creating dylibs, and compile in release mode. In this manner, once you built everything (which only takes a long time for the first time you compile), only the changed crates will be rebuilt, so compilation times won't suffer at all that much. Hacking around FFI when all you have is Rust code seems like a weird and needlessly dangerous idea.

1 Like

I like the multi-crate idea, although it would be easier if they could have circular dependencies between them in the same way that modules can.