I have a callback which a C library calls on rust side
extern "C" fn dataCallback(data_arr: *mut i8, data_size: c_int, arg3: *mut c_void) {
if !data_arr.is_null() {
let data = unsafe { slice::from_raw_parts(audio_data, data_size as usize) };
// want to save this data slice so that it could be reused later in the code
}
}
This callback return data in data_arr.
This callback is called multiple times.
This callback is set in another function like this
setCallback(Some(dataCallback), *mut c_void)
My problem is I want to store all the data_arr into a single slice of i8 as they come in the dataCallback function and use it later. @2e71828 pointed about using Arc<Mutex<Vec<i8>> to get the data you want by passing it to the c_void parameter inside callback function.
I am a little bit new to rust and ffi. Can someone help me by giving some sample example on this
Thanks
Most callback-based C libraries will provide something like a void* user_data argument that's just a pass-through from when the callback was registered. If yours does, then you can use that to pass something like Arc<Mutex<Vec<i8>>> into the callback to fill.
Barring this kind of mechanism provided by the library, I think that a global variable may be your only option.
I do have a void* inside my callback. can you please give a sample example of how to pass Arc<Mutex<Vec<i8>> into the callback. I am relatively new to rust and ffi
I don't have time right now to write up a proper example, but the basic process would look something like this:
When registering the callback, use Arc::into_raw and convert the resulting *const Mutex<Vec<i8>> to void*
Inside the callback, cast the user data pointer back to *const Mutex<Vec<i8>>, and then take a reference to it (&*converted_pointer)
After deregistering the callback, use Arc::from_raw to reconstruct the Arc so that it will decrement the reference counts correctly when it's dropped.
For a slightly more general solution, you can box a closure that captures whatever variables you'll need to access, and use Box::into_raw and Box::from_raw instead. (This method is a bit more complicated to get right and has a few snags you might run into because closures don't have nameable types, but there's probably a crate out there somewhere that implements it for you...)
in most cases, if the callback can access a user provided void* context, you can use rust closures as the callback.
but to provide a concrete example, more details about the C API are needed, e.g. what's the type signature of the "register_callback" routine, and the callback signature, etc.
that is usually called "context" or "user data", the library simply saves it as some opaque data, and it's the key to make a rust wrapper.
here's an example using the FnMut() trait, i.e. rust closures.
// declare the C APIs
extern "C" {
fn setCallback(
callback: unsafe extern "C" fn(data: *const c_char, len: c_int, context: *mut c_void),
context: *mut c_void,
);
// hypothetical main loop
fn run();
}
// the callback shim "un-erases" the `void *` to recover the original closure type
unsafe extern "C" fn callback_shim<F: FnMut(&[c_char])>(
data: *const c_char,
len: c_int,
context: *mut c_void,
) {
let ptr = context.cast::<F>();
let data = std::slice::from_raw_parts(data, len as usize);
(*ptr)(data);
}
// this wrapper "erases" the heap allocated pointer of closure into a `void *`
fn set_callback_rust<F: FnMut(&[c_char])>(callback: F) {
let ptr = Box::into_raw(Box::new(callback));
unsafe {
setCallback(callback_shim::<F>, ptr.cast());
}
}
fn main() {
let mut saved_data = vec![];
// the callback append every chunk of data to `saved_data`
set_callback_rust(|data| {
saved_data.extend_from_slice(data);
});
// now run the ffi library's main loop,
// which should dispatch data to the callbacks
unsafe {
run();
}
// when the main loop returns, can now examine the saved data
for x in saved_data {
println!("data point: {}", x);
}
}
you can also alternatively define a custom trait instead of the special FnMut trait.
I should note, the example I gave is illustrative only, it's not particularly robust or safe to use.
to write a truly safe rust wrapper, there are many factors to consider, for instance, whether the callback will be called from different thread, whether it can re-enter if you call any other APIs, whether it will be called concurrently, when and how the callback is "deregistered", etc. so it heavily depends on the nature of the library.
Yeah it will be called from multiple threads as this rust wrapper is going to be used by a web service.
In that case will wrapping saved_data vector with Arc<Mutex<Vec<>>> help?
if it can be called concurrently, then FnMut() is not safe, you must use Fn(), and you also need to add Sync besides Fn().
if, (althoug very rare, it is possible) it cannot be called concurrently (e.g. the caller, although on multiple threads, will hold a lock to call), I think FnMut() + Send can still be safe.
yes, when Fn() + Sync is used instead of FnMut(), you'll not be able to capture mutable states directly, and you also need thread safety, so you need Arc<Mutex<...>> to be able to mutate states, to save the data, for example.
to be clear, there's always trade-offs to make. the more conservative you go for safety, the more restrictions/inconvenience you'll have when writing the callbacks. for example, the most conservative bounds I can think of is Fn() + Send + Sync + 'static, but you are forced to use a lot more thread safe wrapper types like Arc<Mutex<...>>.
then, change the type of saved_data to Arc<Mutex<Vec<_>>> so it can be safely accessed from within the callback:
let saved_data = Arc::new(Mutex::new(vec![]));
then, add move keyword to the callback closure, also, the closure should capture a clone of saved_data, let's call it saved_data_2:
// this is cheap, it clones the reference counted pointer
let saved_data_2 = Arc::clone(&saved_data);
set_callback_rust(move |data| {
// need to `lock()` the mutex first, to ensure thread safety
saved_data_2.lock().unwrap().extend_from_slice(data);
});
finally, to check the collected results, you also need to lock() the mutex first.
// just a demo, don't actually do this, because
// this code may hold the lock for very long period
for x in saved_data.lock().unwrap().iter() {
println!("data point: {}", x);
}
to be conservative, I added 'static, which might be unnecessary, but it is not trivial to make non-'static safe, if possible at all, for the specific C APIs âŠī¸