How to convert a Rust closure to a C-style callback?

I have function from C library:

pub type CCallback = unsafe extern fn( *mut c_void ) -> c_uchar;
fn c_func_set_cb(writeCB: CCallback) 

C code (argument):

typedef struct {
  GenObject                  *p_obj;
  const ObjectReference      *p_ref;
  void                        *p_val;
  int32u                       ui_param;
  int16u                       us_param1;
  int16u                       us_param2;
  void                        *p_param;
} CALLBACK_ARGS_T;

typedef STATUS_T (*CCallback)(CALLBACK_ARGS_T*);

How to convert (wrap) to safety Rust function?

What are the safety invariants of the C library?

@alice I don't understand. Could you paraphrase?

Does c_func_set_cb accept any user data for the callback's parameter?

What does the C library require to be true, for it to not cause unsoundness? Stuff like "the function can't be a null pointer" e.g.

In other words, what do you want your safe wrapper api to enforce?

One thing that needs details is that void pointer. What does it point to?

1 Like

@alice @naim I need to check if the arguments (fields) in the structure CALLBACK_ARGS_T are nullptr.

I would like to give a closure function (Rust).

You can do this if your C callback accepts both a function pointer and a data pointer. You box up the closure and pass it in the data pointer. Then you can create a Rust function that takes the data pointer, converts it back to a closure pointer and calls it. From your C library, I don't see that it's possible to pass a data pointer when setting the callback, though.

The traditional C approach when no data pointer is passed would be to use a global variable to hold any state. It's horrible, but you could do that with rust using a Mutex reasonably easily.

First, since you get a pointer to a CALLBACK_ARGS_T, you will need to transpose that definition to Rust:

mod opaque {
    #[repr(C)]
    pub struct GenObject([*const u8; 0]);
    #[repr(C)]
    pub struct ObjectReference([*const u8; 0]);
}

#[repr(C)] // <- Paramount!
struct CallbackArgs {
    p_obj: *mut opaque::GenObject,
    p_ref: *const opaque::ObjectReference,
    p_val: *mut c_void,
    ui_param: u32,
    us_param1: u16,
    us_param2: u16,
    p_param: *mut c_void,
}

// match the C definition of `typedef enum STATUS { ... } STATUS_T;`
/* For the sake of the example, let's say we had:
    typedef enum STATUS {
        STATUS_SUCCESS = 0,
        STATUS_ERROR,
    } STATUS_T;`
*/
mod Status {
    pub type T = ::std::os::raw::c_int;
    pub const Success: T = Helper::Success as _;
    pub const Error: T = Helper::Error as _;

    #[repr(C)]
    enum Helper {
        Success = 0,
        Error,
    }    
}

type CCallback = Option<
    unsafe extern "C" // or extern "system" if on Windows
    fn (Option<&mut CallbackArgs>) -> Status::T
>;

and then, imagining you interact with an API that offers a function to which you have to feed a CCallback (and I suspect that you can feed a void * p_param too):

STATUS_T api_function (
    // ...,
    CCallback cb,
    void * p_param);

You can call it from Rust with:

extern "C" /* or "system" */ {
    fn api_function (
        // ...
        cb: CCallback,
        p_param: *mut c_void,
    ) -> Status::T;
}

api_function(
    // ...,

    /* CCallback_t */
    Some({
        unsafe extern "C"
        fn cb (args: Option<&mut CallbackArgs>) -> Status::T
        {
            // body
        }
        cb
    }),

    /* `void * p_param` */
    ptr::null_mut(),
)

This is for the low-level basic binding. This step is mandatory since now we are dealing with a concrete Rust objective / API, instead of a C one. Now we want to define that callback, void * pair, out of a safe Rust closure:

pub
fn safe_api_function<Closure> (closure: Closure)
  -> Result<(), ()>
where
    Closure : Fn(/* Args */) -> Result<(), ()>,

So, it is currently very hard for us at the forum to know how the CALLBACK_ARGS_T can be used, so I'll just go for a basic thing: I'll just be using ui_param: u32 as my Args:

use ::std::panic;

pub
fn safe_api_function<Closure> (closure: Closure)
  -> Result<(), ()>
where
    Closure : Fn(u32) -> Result<(), ()>,
    Closure : panic::RefUnwindSafe, // unless panic = "abort"
{
    extern "C" /* or "system" */ {
        fn api_function (
            // ...
            cb: CCallback,
            p_param: *mut c_void,
        ) -> Status::T;
    }

    macro_rules! unwrap { ($expr) => (
        match $expr {
            | Some(it) => it,
            | None => return Status::Error,
        }
    )}
    
    let status = unsafe { api_function(
        // ...,
    
        /* CCallback_t */
        Some({
            unsafe extern "C"
            fn cb<Closure> (args: Option<&mut CallbackArgs>)
              -> Status::T
            where
                Closure : Fn(u32) -> Result<(), ()>,
                Closure : panic::RefUnwindSafe, // unless panic = "abort"
            {
                let args = unwrap!(args);
                let at_closure_data: *const Closure = args.p_param.cast();
                let closure: &'_ Closure = unwrap!(at_closure_data.as_ref());
                let ui_param = args.ui_param;
                let result = unwrap!(panic::catch_unwind(|| // unless panic = "abort"
                    closure(ui_param)
                ).ok()); // unless panic = "abort"
                if result.is_ok() { Status::Success } else { Status::Error }
            }
            cb::<Closure>
        }),
    
        /* `void * p_param` */
        &closure as *const _ as *mut c_void,
    )};
    match status {
        | Status::Success => Ok(()),
        | Status::Error => Err(()),
        | _ => unreachable!("C returned an invalid enum discriminant: {}", status),
    }
}
  • This is of course only safe if api_function promises to:

    • never call the callback it has been given once it returns.

      Otherwise you will need to have to change the function signature into taking a &'static Closure (unless you get to have a freeing function, in which case you could take an Arc<Closure> where Closure : 'static, call Arc::into_raw() when creating the p_param, and call drop(Arc::from_raw(...)) in the part of the API that lets you free the closure state).

      • (You can use Rc instead of Arc if you know the freeing function will be called from the same thread that called safe_api_function).
    • call the callback from within the same thread.

      Otherwise you will need to add a Closure : Sync bound, and in the case of also having a freeing function, which itself may be called from another thread, also add a Closure : Send bound.

  • This is conservatively using an Fn (shared access to the closure's environment / captures), to guard against C code being reentrant (since most C code is not even aware of that issue). If you are sure the closure will not be called in a reentrant manner (nor in parallel in the case of a multi-threaded scenario), then you can loosen the Fn bound to an FnMut one (and thus use <*mut _>::cast(&mut closure) when creating the p_param).

    • If the closure is not gonna be run in parallel (otherwise using FnMut would be unsound) but may be nevertheless be run in another thread, you would need a Send bound instead of a Sync one.
  • TL,DR: to be as safe as possible, you take a &'static Closure where Closure : Sync.

2 Likes

I'm trying to write a simple wrapper.

pub type CCallback = unsafe extern "C" fn( *mut c_void ) -> STATUS_T;

pub fn function_rust<F: Fn()>(myfunc: F, handle : &mut HANDLE_t)
{
    unsafe extern "C" fn wrapper<F: Fn()>( args : *mut c_void ) -> STATUS_T
    {
        myfunc();
        println!("wrapper");
        STATUS_T::OK
    };

    c_func_set_cb(wrapper::<F>).unwrap();
}

///...

///Call
   function_rust(||{
        println!("TEST");
    },  &mut v.handle);

Error:

error[E0434]: can't capture dynamic environment in a fn item
  --> src/lib.rs:90:9
   |
90 |         myfunc();
   |         ^^^^^^
   |
   = help: use the `|| { ... }` closure form instead

It is very easy to cause unsoundness when dealing with untyped closures, such as when using C. So I am sorry to say there is no such thing as a simple and correct wrapper. My answer above goes into great detail to describe all the steps required to define such a wrapper, so as to make that task be as simple as possible, while remaining as safe as possible.

Ok, that's more information about your actual API. It turns out, and sadly this happens quite often with C, that the API you have to deal with (c_func_set_cb) has not been well designed: it takes a function pointer (i.e., "code"), but it does not take a data pointer (the *mut c_void from my previous post). This means indeed that you can only give it plain functions, and not (stateful) closures.

And as with any API limitation short-coming, there is no other solution but to get the API to be improved in a further update. That being said, if you really need to, you can do the following hacky workaround (≠ solution): using mutable global state and :crossed_fingers: that callbacks are not concurrently registered (fed to the (limited) API).

  • i.e., in between the moment you register the callback and the moment it is done running there is no other callback registration, such as:

    • no reentrancy: your callback does not itself register another callback;

    • no parallelism: there isn't another callback registration happening in parallel from another thread.

Now, mutable global state requires shared mutability (since a global is, by definition, a shared entity), so for that there are two mechanisms. I'm gonna assume the callback is defined and called within the same thread, hence I'll be using thread_local! storage and ?Sync shared mutability wrappers.

With all that being said, here is the hacky workaround:

use ::std::{
    cell::RefCell,
    ffi::c_void,
};

pub type CCallback = Option< /* Don't forget the `Option` ! */
    unsafe extern "C"
    fn(*mut c_void) -> Status::T
>;

// match the C definition of `typedef enum STATUS { ... } STATUS_T;`
/* For the sake of the example, let's say we had:
    typedef enum STATUS {
        STATUS_SUCCESS = 0,
        STATUS_ERROR,
    } STATUS_T;`
*/
mod Status {
    pub type T = ::std::os::raw::c_int;
    pub const Success: T = Helper::Success as _;
    pub const Error: T = Helper::Error as _;

    #[repr(C)]
    enum Helper {
        Success = 0,
        Error,
    }    
}

pub
fn function_rust<Closure> (
    rust_closure: Closure,
    _handle: &mut HANDLE_T,
) -> Result<(), ()>
where
    Closure : 'static, // to be able to store it in global memory
    Closure : FnMut() -> Result<(), ()>, // Use `FnOnce` if the closure is only gonna be called once, or `Fn` if it may be called concurrently.
{
    thread_local! {
        static CLOSURE_DATA
            : RefCell<Option<Box<dyn FnMut() -> Result<(), ()>>>>
            = None.into()
        ;
    }

    unsafe extern "C"
    fn wrapper (_: *mut c_void)
      -> Status::T
    {
        ::scopeguard::defer_on_unwind!({
            eprintln!("\
                Caught Rust attempting to unwind across \
                an `extern \"C\"` function boundary, which is UB.\n\
                Aborting the process for soundness.\
            ");
            ::std::process::abort();
        });
        let result = CLOSURE_DATA.with(|closure| { // `let closure = &*CLOSURE_DATA;` if lazy_static
            closure
                .borrow_mut() // or `.write()` if `RwLock`; `.borrow()` / `.read()` if `Fn()`
                .as_mut() // or `.as_ref()` if `Fn()`
             // .take() /* if `FnOnce()`
                .expect("Concurrent registration!")
                () // call the closure
        });
        if result.is_ok() { Status::Success } else { Status::Error }
    };

    CLOSURE_DATA.with(|it| {
        it  .borrow_mut() // or `.write()` if `RwLock`
            .replace(Box::new(rust_closure))
    });
    unsafe {
        c_func_set_cb(Some(wrapper))?;
    }
    Ok(())
}

fn main ()
{
    use ::std::{cell::Cell, rc::Rc};

    let mut handle = { /* ... */ };
    let closure_state = Rc::new(Cell::new(0));
    function_rust(
        {
            let closure_state = Rc::clone(&closure_state);
            move || Ok({
                eprintln!("Called start.");
                closure_state.set(
                    dbg!(closure_state.get()) + 1
                );
                eprintln!("Callback end.");
            })
        },
        &mut handle,
    ).unwrap();
    assert_eq!(dbg!(closure_state.get()), 1);
}
4 Likes

@Yandros Thx for reply. I can't use :

pub type CCallback = Option<unsafe extern "C"fn(*mut c_void) -> Status::T>;

because the C library cannot "unwind" the function definition. I'm right?

See https://doc.rust-lang.org/nomicon/ffi.html#the-nullable-pointer-optimization
The idea is that ... fn(...) -> _ pointers cannot be NULL, so if Rust code expects to get such a pointer and the FFI feeds it a NULL, then it is undefined behavior.

That being said, it is quite rare to be receiving an fn() pointer from FFI (usually it is the other way around, such as in your example; you are the one giving the fn() pointer to the FFI). In that case, it is "acceptable" to define the FFI type as a subset of the actual type, and use a non-Option-wrapped pointer. But in general, that is a slippery slope, it is error prone, so the safest choice is to always go with an exact set representation.

2 Likes

I tried to wrap callback to Option <extern fn(...) ...> but the call did not work.

It will depend on where the definition of c_func_set_cb comes from. It may need to be adjusted. If it comes from an automatic tool such as bindgen (I doubt it but we never know), then feel free to remove the Option.

I have a problem.
From: https://doc.rust-lang.org/beta/std/panic/trait.UnwindSafe.html

Types such as &mut T and &RefCell<T> are examples which are not unwind safe. The general idea is that any mutable state which can be shared across catch_unwind is not unwind safe by default. This is because it is very easy to witness a broken invariant outside of catch_unwind as the data is simply accessed as usual.

Types like &Mutex<T> , however, are unwind safe because they implement poisoning by default. They still allow witnessing a broken invariant, but they already provide their own "speed bumps" to do so.

Wrap a closure with a mutex? How to fix it?
Error:

error[E0277]: the type `&mut &Closure` may not be safely transferred across an unwind boundary
   --> src/lib.rs:168:30
    |
168 |                 let result = panic::catch_unwind(|| unsafe_closure(name, val)).ok();
    |                              ^^^^^^^^^^^^^^^^^^^ ---------------------------- within this `[closure@src/lib.rs:168:50: 168:78 unsafe_closure:&mut &Closure, name:std::string::String, val:T]`
    |                              |
    |                              `&mut &Closure` may not be safely transferred across an unwind boundary

Code:

pub fn set_write_callback<T, Closure>(&mut self, callback: Closure) -> Result<(), String>
      where Closure: FnMut(String, T) -> Result<(), String>,
            Closure: 'static + Sized, T: std::fmt::Display  + Default + std::panic::UnwindSafe,
  {
      extern "C" fn std_wr_handler<Closure, T>(args: *mut CallbackArgs) -> Status
          where
              Closure: FnMut(String, T) -> Result<(), String>,
              Closure: 'static + Sized, T: std::fmt::Display  + Default + std::panic::UnwindSafe,
      {
          unsafe {
              let ui_param = match check_cb_arg(args)
              {
                  Ok(ui_param) => {ui_param},
                  Err(_) => { return Status::Error; }
              };

              let mut val: T = T::default();
              let p_val = &mut val as *mut T as *mut c_void;
              if p_val == ptr::null_mut() { return Status::Error; }

              let at_closure_data: *mut Closure = (*args).p_param.cast();
              if at_closure_data == ptr::null_mut() { return Status::Error; }

              let unsafe_closure = match at_closure_data.as_ref()
              {
                 Some(closure) => { closure },
                 None => {
                       println!("TEST closure none");
                       return Status::Error;
                 }
              };

              let name = "test".to_string();
              let result = panic::catch_unwind(|| unsafe_closure(name, val)).ok();

              match result
              {
                 Some(value) => { println!("Result: {:?}", value); }
                 None => {
                      println!("caught panic");
                      return Status::Error;
                  }
              }

              Status::Success
          }
      }

      return match &mut self.callback_handle
      {
          Some(callback_handle) => {
                 IEC_attr_set_cb(callback_handle, std_wr_handler::<Closure, T> as *mut _,
                                 ptr::null_mut(), &callback as *const _ as *mut c_void)
          },
          None => { Err("Cannot set write callback".to_string()) }
      }
  }

Try something like:

pub
fn set_write_callback<T, Closure> (
    self: &'_ mut Self,
    callback: Closure,
) -> Result<(), String>
where
    Closure : Fn(String, T) -> Result<(), String>, // closure is callable by `&`
    Closure : 'static, // closure owns its environment / does not borrow any locals
    Closure : panic::RefUnwindSafe, // closure panicking does not let stuff in a badly corrupted state
    T : Default + panic::UnwindSafe, // value fed to the closure is also unwind safe
{
    unsafe extern "C"
    fn std_wr_handler<Closure, T> (args: *mut CallbackArgs)
      -> Status::T
    where
        Closure : Fn(String, T) -> Result<(), String>, // closure is callable by `&`
        Closure : 'static, // closure owns its environment / does not borrow any locals
        Closure : panic::RefUnwindSafe, // closure panicking does not let stuff in a badly corrupted state
        T : Default + panic::UnwindSafe, // value fed to the closure is also unwind safe
    {
        let ui_param = match check_cb_arg(args) {
            | Ok(ui_param) => ui_param,
            | Err(_) => return Status::Error,
        };

        let at_closure_data: *const Closure = (*args).p_param.cast();
        let at_closure: &'_ Closure = match at_closure_data.as_ref() {
            | None => {
                // Case where `at_closure_data.is_null()`
                return Status::Error;
            },
            | Some(at_closure) => {
                // Non-null Closure ptr
                at_closure
            },
        };

        let result = panic::catch_unwind(|| {
            // this catch_unwind closure captures
            // `&'_ Closure`,
            // and it needs to be `UnwindSafe`.
            // `&'_ Closure : UnwindSafe` if, and only if,
            // `Closure : RefUnwindSafe`.
            // Hence the `RefUnwindSafe` bound.
            at_closure("test".to_string(), T::default())
        });
        match result {
            | Ok( Ok(()) ) => {
                // Closure returned `Ok(_)`
                Status::Success
            },
            | Ok( Err(err) ) => {
                eprintln!("Closure call failed: {}", err);
                Status::Error
            },
            | Err( _ ) => {
                eprintln!("Closure panicked");
                Status::Error
            },
        }
    }

    match self.callback_handle {
        | Some(ref mut callback_handle) => {
            IEC_attr_set_cb(
                callback_handle,
                std_wr_handler::<Closure, T> as *mut _,
                ptr::null_mut(),
                &callback as *const _ as *mut c_void,
            )
        },
        | None => Err("Cannot set write callback".to_string()),
    }
}
2 Likes

@Yandros Thx for reply.
When Closure is Fn(...) then I can't modify variables.
I would like I could pass variables and modify them. And for the function to be thread safe.

If you want mutation and thread-safety, then the mutation needs to happen by virtue of Mutex-ing stuff rather than using exclusive &mut access (the access cannot be exclusive if multi-threaded).

  • or, for simpler types such as integers, by using things like AtomicCell

So, instead of:

let mut counter = 0;
let mut incr = move || {
    let counter = &mut counter;
    *counter += 1;
    println!("{}", *counter);
};

you will need to do stuff like:

let counter = Mutex::new(0);
let incr = move || {
    let mut guard = counter.lock().unwrap();
    let counter = &mut *guard;
    *counter += 1;
    println!("{}", *counter);
};

More generally / if you don't want to be tweaking all the closures already provided, you can define a Mutex-guarded Fn() + Send + Sync + 'static closure that wraps the FnMut() + Send + 'static provided by the user (this may be a bit less eficient than letting the user wrap what needs to be wrapped, but in practice it leads to a simpler API for users):

fn mutexed_closure<Ret> (
    f: impl FnMut(...) -> Ret
          + 'static
          + Send
    ,
) -> impl Fn(...) -> Ret
        + 'static
        + Send + Sync
        + panic::UnwindSafe + panic::RefUnwindSafe
{
    let f = Mutex::new(f);
    move |...| {
        f.lock().unwrap()(...)
    }
}
1 Like

So pass the wrapped closure with Mutex to set_write_callback() -> Mutex<Closure>?