Storing C callbacks in Rust

#1

I am in the process of evaluating languages and frameworks for an upcoming project. The core part of this project is a data parser and transformer which will be a dynamic (or static) library, with various front-ends calling into it. I would like to use Rust for this part of the project. I am a moderately experienced C++ developer.

One of the requirements would be for the front-end application to register event callbacks (of which there could be thousands per second). The way I am thinking about it at the moment is that the calling application would provide a C-style callback (function pointer) as a parameter to a function that stores the function pointer, and calls it with relevant data every time an event occurs. The calling application might written in C++, Rust, Python, or something else.

I am testing the water here to see if this is a valid approach.

#2

Well, the function pointer is the same format in both rust and c/c++:

let x: fn(usize) -> &str;
void*const (*y)(size_t); //Not too sure about the void*const part

I’m pretty sure x and y have the same representation in memory (As long as they’re compiled on the same architecture, but that doesn’t really apply if you’re working with libraries/FFI), so you can pass it like so (Mind, I’m not well versed in c++, much less c++ ffi):

//main.cpp
#include <iostream>
extern "C" {
    void* get_state(int (*callback)(char*));
    int run(void* state);
    void delete_state(void* state);
}

int callBack(char* msg) {
    std::cout << msg;
    return 0;
}

int main(int argc, char** argv) {
    auto state = get_state(&callBack);
    if (run(state) != 0) {
        std::cout << "Error calling callback";
    }
    delete_state(state);
}
//lib.rs
#[no_mangle]
pub struct MyState {
    pub call_back: extern fn(*const u8) -> i32
}

#[no_mangle]
pub extern fn get_state(call_back: fn(*const u8) -> i32) -> *const () {
    let state = MyState { call_back };
    Box::into_raw(Box::new(state)) as *const _
}

#[no_mangle]
pub extern fn run(state: *mut MyState) -> i32 {
    unsafe {
        ((*state).call_back)("ABC".as_ptr())
    }
}

#[no_mangle]
pub extern fn delete_state(state: *mut MyState) {
    unsafe {
        Box::from_raw(state);
    }
}

I’m not too sure about it but I think that you can put the three functions into an impl MyState block along with a #[feature(arbitrary_self_types)].
Also, for some further reading, you might want to look at the Rust ffi omnibus, credit to @shepmaster, it has a lot of mini examples for ffi which help passing various things such as functions, slices and more!

3 Likes
#3

Great - thanks for your detailed reply! That will give me a good starting point.

#4

That call_back type needs to be extern fn too, as this declares the ABI, especially for passing parameters and return values.

3 Likes
#5

Thanks for the catch! I’m not experienced with C++ FFI, so I didn’t know that. I’m sorry, I misunderstood it as you wanted me to make the following change:

- int callBack(char* msg) {
+ extern "C" int callBack(char* msg) {
    std::cout << msg;
    return 0;
}
1 Like
#6

Although Rust is a great language for FFI, it is a very unsafe thing to do, leading very easily to UB.

When doing so, I always use the following

FFI safety belt

  • any pointer from the C world becomes Option<NonNull<_>>, (or Option<unsafe extern "C" fn (...) -> ... > for function pointers);

    • this tackles C code being able to feed NULLs at will, forcing the Rust code to handle them;
  • use ::libc::c_void to represent C’s void. Thus void * becomes Option<NonNull<::libc::c_void>>;

  • even though panic = "abort" is a setting that can be added to Cargo.toml compilation profile, I prefer to have such a guard within the exported code;

  • structs and enums should be #[repr(C)] (or #[repr(transparent)] for newtypes)

  • if receiving an enum from C / FFI, it should be of integer type. If it isn’t, it should be instantly transmuted into an integer and then matched against integer values to get a Rust enum back.

    • this includes booleans:
      let rust_bool: bool = mem::transmute<_, i32>(c_bool) != 0;
  • do not use static muts, not even for FFI; you should use

    • lazy_static! with RwLocks when in doubt,

    • thread_local!s with RefCells for single-threaded programs,

    • or, if you really wanna go down the unsafe path, a static UnsafeSyncCell<_>:

      #[repr(transparent)]
      pub
      struct UnsafeSyncCell<T> (pub UnsafeCell<T>);
      
      unsafe impl<T> Sync for UnsafeSyncCell<T> {}
      
      static MY_STATIC_MUT: UnsafeSyncCell< u8 > = UnsafeSyncCell(UnsafeCell::new(0));
      
      // above is still better than:
      static mut MY_STATIC_MUT: u8 = 0; // DO NOT DO THIS
      

Example: registering callbacks in a single-threaded context

Rust FFI library (crate-type = ["cdylib"] or crate-type = ["staticlib"])

use ::std::{*,
    cell::RefCell,
    ptr::NonNull,
};
use ::libc::c_void; // libc = "0.2.51"

macro_rules! ffi_panic_boundary {($($tt:tt)*) => (
    // /* or */ { ::scopeguard::defer_on_unwind(process::abort()); $($tt)* }
    match ::std::panic::catch_unwind!(|| {$($tt)*}) {//
        | Ok(ret) => ret,
        | Err(_) => {
            // /* or */ return RUST_ERR_PANICKED;
            eprintln!("Rust panicked; aborting process");
            ::std::process::abort() 
        },
    }
)}

type Arg = NonNull<c_void>;

type Callback = unsafe extern "C" fn (mb_arg: Option<Arg>);

thread_local! {
    static CALLBACKS: RefCell<
        Vec< (Callback, Option<Arg>) >
    > = RefCell::new(Vec::new());
}
// or lazy_static! with a RwLock

#[allow(non_camel_case_types)]
#[repr(C)]
pub
enum e_rust_status {
    RUST_OK = 0,
    RUST_ERR_NULL_POINTER,
    // RUST_ERR_PANICKED,
}
use self::e_rust_status::*;

macro_rules! unwrap_pointer {($pointer:expr) => (
    match $pointer {//
        | Some(non_null_pointer) => non_null_pointer,
        | None => return RUST_ERR_NULL_POINTER,
    }
)}

#[no_mangle] pub extern "C"
fn register_cb (
    cb: Option<Callback>,
    arg: Option<Arg>,
) -> e_rust_status
{
    ffi_panic_boundary! {
        let cb = unwrap_pointer!(cb);
        CALLBACKS.with(|slf| { slf
            .borrow_mut()
            .push((cb, arg))
        });
        RUST_OK
    }
}

#[no_mangle] pub unsafe extern "C"
fn call_cbs () -> e_rust_status
{
    ffi_panic_boundary! {
        CALLBACKS.with(|slf| { slf
            .borrow_mut()
            .iter_mut()
            .for_each(|&mut (cb, arg): &mut (Callback, Option<Arg>)| {
                cb(arg);
            })
        });
        RUST_OK
    }
}

#[no_mangle] pub extern "C"
fn clear_cbs () -> e_rust_status
{
    ffi_panic_boundary! {
        CALLBACKS.with(|slf| {
            let mut cbs = slf.borrow_mut();
            cbs.clear();
            cbs.shrink_to_fit();
        });
        RUST_OK
    }
}

Rust FFI library header

#ifndef __RUST_LIB_H__
#define __RUST_LIB_H__

typedef enum rust_status {
    RUST_OK = 0,
    RUST_ERR_NULL_POINTER,
} e_rust_status;

typedef void (*cb_t) (void *);

extern e_rust_status register_cb (cb_t cb, void * arg);

extern e_rust_status call_cbs (void);

extern e_rust_status clear_cbs (void);

#endif //__RUST_LIB_H__

(you can use cbindgen for this step, although it does not like Options (rightfully so: Rust non-C enums layout is undefined))

User of the library: example C code

#include <stdio.h>
#include <stdlib.h>
#include "rust_lib.h"

void rust_try (e_rust_status rust_status)
{
    switch (rust_status) {
        case RUST_OK:
            return;
        case RUST_ERR_NULL_POINTER:
            fprintf(stderr, "Rust call failed: got NULL pointer\n");
            break;
        default:
            fprintf(stderr, "Rust call returned an unknown value\n");
            break;
    }
    exit(EXIT_FAILURE);
}

void inc (int * counter)
{
    ++(*counter);
}

int main (int argc, char const * const argv[])
{
    int counter = 0;
    {
        rust_try(register_cb((cb_t) inc, &counter));
        rust_try(register_cb((cb_t) inc, &counter));
        rust_try(register_cb((cb_t) inc, &counter));
        printf("%d\n", counter);
        rust_try(call_cbs());
        printf("%d\n", counter);
        rust_try(clear_cbs());
    }
    return EXIT_SUCCESS;
}
8 Likes
#7

I’m blown away by the level of detail here, thanks so much

1 Like