Casting function pointers with different linkage

Considering task::RawWakerVTable that takes Rust function pointers, it makes me wonder if it would be safe to pass function pointer with "C" linkage?

I'm not even sure right now if it is trivial to cast away C linkage, because transmute wouldn't work with function pointers.
But it does make me wonder if it would be safe to cast away "C" linkage and pretend function is like Rust function

Yeah so as I thought transmute no longer works with function pointers:

error[E0591]: can't transmute zero-sized type
 --> src\callback.rs:4:56
  |
4 |     task::RawWakerVTable::new(vtab::on_clone, unsafe { mem::transmute(vtab::on_wake) }, unsafe { mem::transmute(vtab::on_wake) }, vtab::on_drop)
  |                                                        ^^^^^^^^^^^^^^
  |
  = note: source type: unsafe extern "C" fn(*const ()) {callback::vtab::on_wake}
  = note: target type: unsafe fn(*const ())
  = help: cast with `as` to a pointer instead

Makes me sad because it seems to impossible to cast directly now

P.s. you can work it around my casting to usize

1 Like

There are two things here:

  1. callback::vtab::on_wake is a zero-sized type uniquely representing that function (item);

    That is, given fn foo() {} and fn bar() {}, foo and bar have different types.

  2. [unsafe] [extern [ABI]] fn (Args) [-> Ret] is a runtime / dynamic type, no longer zero-sized, but pointer-sized, pointing to the beginning of machine code (while still having the ABI and unsafe-ty encoded at the type-level).

Transmuting between the function pointer types can be done (since they are all equally pointer-sized), it just requires coercing the input zero-sized function item type to the runtime function pointer type:

use ::core::mem;

fn main ()
{
    fn foo () {}

    assert_eq!(mem::size_of_val(&foo), 0);

    let foo_ptr: fn() = foo; // coercion to runtime type (function pointer)

    assert_eq!(mem::size_of_val(&foo_ptr), mem::size_of::<usize>());

    let unsafe_fptr = unsafe {
        mem::transmute::<_, unsafe fn()>(foo_ptr) // works
    };
    // or
    let unsafe_fptr = unsafe {
        mem::transmute::<_, unsafe fn()>(foo as fn()) // explicit coercion from foo to foo_ptr
    };
    // or
    let unsafe_fptr = unsafe {
        mem::transmute::<fn(), unsafe fn()>(foo) // implicit coercion from foo to foo_ptr
    };
    
    // calling foo through an `unsafe fn()` function pointer requires an `unsafe` block
    unsafe {
        unsafe_fptr()
    }
}
  • Playground

  • the same would have happened with foo being extern { fn foo(); } (except for it already being an unsafe fn)

But the only time this transmute is sound is when:

  • transmuting an input parameter to a compatible type that is at least as strict.

    • Example
      fn(*const i32) to fn(&i32)
  • making a non-unsafe fn become unsafe:

    • Example
      [extern _] fn(Args) [-> Ret] to unsafe [extern _] fn(Args) [-> Ret]
  • Sometimes, you may just know that an extern function is not unsafe, i.e., that calling it with any valid input cannot possibly be unsound. Then you can transmute "in the other direction" (i.e., stripping the unsafe).

Something that will never ever be sound, however, is transmuting the call ABI: an extern X function must always remain extern X.


Now, in all this transmute rant, we have downgraded our compile-time zero-sized function items to runtime function pointers (leading to dynamic calls) just for the sake of changing something about the function's signature; and that isn't optimal. Moreover, changing the ABI has not been possible.

It turns out that to keep zero-sized function items, and being even able to "change" function call ABI can very simply be done by wrapping the function call in another function with the ABI and API of our liking.

Changing the ABI
  • instead of

    use ::libc::c_int;
    
    /// Static / compile-time linking
    extern "C" { fn abs (_: c_int) -> c_int; }
    // abs function pointer type is `extern "C"`
    
  • you can do:

    use ::libc::c_int;
    
    /// # Safety: same safety requirements as the extern `abs` function.
    unsafe
    fn abs (x: c_int) -> c_int
    {
        /// Static / compile-time linking
        extern "C" { fn abs (_: c_int) -> c_int; }
    
        abs(x)
    }
    // abs function pointer type is not `extern "C"`
    
Changing the API only (without touching the ABI):

Imagine we have a extern "C" { fn clear (_: *mut c_int); } that we would like to use in Rust, and it taking raw pointers does not grant the compile-time checks that Rust is usually able to give. Then you can just go and do:

extern "C" { // same ABI
    fn clear (_: &'_ mut c_int); // changed API to a more restrictive input
}
Chaning both the API and the ABI

And if wanting to also make the function non-unsafe (and now that we are at it, not have an extern "C" ABI), we can just go and wrap it:

  • #[inline]
    fn clear (x: &'_ mut c_int) // non-`extern "C"` ABI
    {
        extern "C" { fn clear (_: &'_ mut c_int); } // changed API
        unsafe { clear(x); }
    }
    

although to be honest this is usually done within one-time closures:

  •  extern "C" { fn clear (_: &'_ mut c_int); }
    
     fn main ()
     {
         let mut array: [c_int; 4] = [1, 2, 3, 4];
         array.iter_mut().for_each(|x| unsafe { clear(x); });
         assert_eq!(array, [0, 0, 0, 0]);
     }
    
5 Likes

This is indeed clarifies, especially because I didn't know that regular function pointers are different from extern one

In my case I just wanted to be able to have the same API for Rust and C code without a need for extra redirection via Rust callback

1 Like

Yep, the Rust ABI extern "Rust" is implicitly used in task::RawWakerVTable function pointers, so you will need that extra redirection. The other option would be to reimplement a RawWakerVTable using extern "C" function pointers, since Rust functions can be told to use the extern "C" ABI.


In some case you could try to have the extern / linked function be inlined within the Rust "wrapper" function, by managing to perform LTO across FFI boundaries. I have no idea how to do that, but maybe someone else knows more about this (cc @HadrienG).

I am aware of such a thing being stable as of Rust 1.34, but I have never tried to make it work and therefore do not know where its use is documented. The test cases mentioned in the stabilization PR would probably be a good starting point.

What I do know, however, is that cross-language LTO works at the LLVM IR level, so you'll need to build your C code with a version of LLVM that's "close enough" to the one used by rustc for the magic to work out.

2 Likes

Well, you cannot really inline C function pointers in first place so I wouldn't worry about it, but I wonder if Rust function pointers (being zero sized) is special and could be inlined within waker's vtable.
I'd hope it should be possible as long as it is used within the same compilation unit, but somehow I feel like since it uses LLVM it still going to be translated to plain address.

In any case wrapping it into Rust pointer makes things easier, though I suspect transmuting extern "C" to rust pointer is still should be fine