C FFI Questions: rust methods in callbacks

I’m writing a plugin (shared library) for an application that exposes a C api and I’m using rust + bindgen to do this.

The basic operation of this plugin is that I have to expose a unmangled extern “C” function named specifically after the name of the shared library, and in that I do some registration with the API to setup my plugin. This all works well so far: https://github.com/x37v/puredata-rust/blob/wrapper/examples/helloworld-orig/src/lib.rs

Knowing that I’m working with an unsafe API, but with knowledge of the guarantees of that API as far as thread safety etc etc, my question is this:

Is it okay to pass MyStruct::struct_method(&mut self) to a C API in place of unsafe extern "C" fn(*mut MyStruct) for callback registration where MyStruct is #[repr(C)]?

I am able to do this in practice and it works well so far, though I have to transmute the function pointer when passing to my bindgen generated bindings.

I just want to make sure I’m not shooting myself in the foot before I go much further down this path. I would really like to not have to write a bunch of unsafe extern "C' fn functions that simply call back into my struct methods if possible.

It does look like I should extern "C" my methods so that I can be sure of the calling convention…

BTW, you may notice that I actually have to transmute to a function that takes no arguments… though, the callback actually does take arguments, that is just an artifact of how the API works.

Thanks for any input you can provide!

This is usually a bad idea. Unless you declare otherwise there’s no guarantee that struct_method() will have the "C" ABI. I’m also not sure whether it’s guarantee’d that method invocation in Rust is equivalent to invoking a function who’s first argument is a pointer to self. For example, with some C++ compilers the this pointer is normally passed in via a register. The funny thing about UB is that it can appear to work for your use case, but the slightest change in environment (e.g. a rustc upgrade or changing OS) can result in nasal demons.

A better way is to have a “trampoline” function which takes a *mut c_void then casts that pointer back to *mut MyStruct so it can invoke the method normally. It can get annoying to write these trampolines, but that’s normally the best way to do it without UB. This is what we did when passing callbacks to libsignal-protocol-c.

This “tarmpoline” trick is particularly important when you want to pass a Rust closure to C. I usually make a helper function which will generically “write” the trampoline function and return a (extern "C" fn (*mut c_void), *mut c_void). It shouldn’t be too hard to alter that helper function to take a closure that’ll invoke your struct method.


Uhh… that doesn’t sound like a great idea. I feel like changing a function pointer’s arity with a cast/transmute could mess up with the bookkeeping automatically injected by the compiler for managing the stack…

2 Likes

Given the level of indirection, it is not that much about MyStruct being #[repr(C)] that matters here (it’s mostly about alignment issues or if C directly manipulates the pointee), but really about MyStruct::struct_method having been declared extern "C".

It so happens that in the code you’ve linked to, you do declare these methods as such. But it is important to mention for someone just skimming at your post.

Now, transmuting the arity of a function is something extremely dangerous to do, but after looking at what the C library does, their t_method (i.e., a void (*) (void)) is actually an opaque pointer type that is dynamically casted (back) to the right arity. So this is not really a Rust FFI issue, but an “issue” with the API of the C library. Given that the arity of the t_method type is ignored, I’d personally have expressed that at the type level by requiring an opaque type with the size of a (function) pointer.

1 Like

@Yandros

Yeah, I actually did a search about the calling conventions of rust after my initial post and then updated my methods to be extern "C".

So, with that in mind, is there any real difference, besides the mangling that happens, between:

impl MyStruct { 
pub extern "C" fn my_method(&mut self) {...

and

pub extern "C" fn plainfunc(*mut MyStruct)

As far as the “safety” their being sent to a C API?

…If I can guarantee that methods are extern "C" is there any reason for a trampoline?

Thanks!
I’m likely going to be writing helper macros for this stuff anyways so using trampolines is probably not that big of a deal because I should be able to generate them with my macros … And, this helper function for closures, I’m gonna have to look a little closer to get my head around how this works but that sounds very useful!

1 Like
impl MyStruct { 
    pub
    extern "C"
    fn my_method(&mut self)

gives the function

pub
extern "C"
fn MyStruct::my_method (_: &mut MyStruct)

So, the question is whether such function is equivalent to

extern "C"
fn foo (_: *mut MyStruct)

Or if, at least, the former function can be safely transmuted into the latter.

  • For two types A and B, what does using a f: fn(A) where a fn(B) is expected mean?

    It means that f, a function that expects its first and only argument to be a A, may actually be fed a B. This is known to be sound when transmuting any x: B into a A is sound.

    • (With subtyping notation, it implies that if B : A, then fn(A) : fn(B). It is called contravariance).

So the question becomes: can a *mut MyStruct be transmuted into a &'_ mut MyStruct?

The answer is: no. Not always at least. It is unsafe.

Funnily enough, the fn transmutation can still be made soundly by requiring that the caller of the transmuted function use unsafe. This is achieved by transmuting the function into an unsafe extern "C" fn (*mut MyStruct). This means that the call is sound only as long as the caller respects some invariant (in this case, that the argument is a valid pointer to a (mutable and) unaliased MyStruct).

So, if you know that the C function will only feed such a pointer to the callback, then feeding this transmuted unsafe callback to the C function is sound.

Usually, we cannot really know that about C (the non-aliasing guarantee is non-existant in that language), so in practice you just hope it gives a valid (thus non-NULL) pointer, which in single threaded programs can be an acceptable assumption (Ideally, for guaranteed soundness, using shared & _ references with interior mutability makes it sound in single-threaded contexts, and with an added Sync bound on the pointee makes it sound in any context).

2 Likes