Reentrant safety

I have an FFI function that pushes messages to a callback function in my application. This callback can potentially be called multiple times from any thread. The documentation notes that this callback must also be reentrant safe.

The API gives me one pointer-sized value that can be passed straight through to the callback function. This should hopefully allow my callback function to pass the message back to the part of my program that is expecting this particular message without blocking the callback more than absolutely necessary.

However, I'm not certain how to do this safely. A crossbeam channel would be the ideal choice but I need a way to get the right channel to use for a particular message. Simply using a pointer to the channel would work but I would need to ensure that the channel never moves.

What's the best way to approach this?

I think Rust function types that implement Sync have to be re-entrant too. If you require your callback function type to implement Fn() + Send + Sync, Rust will check that for you.

As for data, Box::new() will give you a stable address. Theoretically content of it could be moved out of the box, but Rust won't do that behind your back, so generally it's a non-issue. If C handles lifetime of the data, then use Box::into_raw. Otherwise store the box somewhere on Rust's side and just cast its reference to a pointer.

If you use Box<Box<dyn Fn() + Send + Sync>> as your C user pointer, then you'll be able to use it to jump into any closure, and then that closure will worry how to carry more context data, send data in and out, and how to do re-entrancy. It's double boxed, because closure's box is a fat pointer, so C won't be able to preserve it.

Re-entrancy is one form of concurrency, so any &-based Rust API, such as Fn, will satisfy that requirement.

  • @kornel to nitpick, it's thus the Fn part which allows re-entrancy; consider a Box<dyn FnMut + Sync> or any other absurd / tautological thing of that kind :grinning_face_with_smiling_eyes: (related: let's observe that it is sound to transmute dyn FnMut and dyn FnOnce to dyn FnMut + Sync and dyn FnOnce + Sync)

If you are to heap-allocate a dyn Fn, then I suggest you use {A,}Rc as the heap-allocating pointer (in this instance, Arc), since at the meager cost of two extra usize-sized counters, you get to have Weaks or your own owned clones :slightly_smiling_face:

If using dyn, then double indirection is indeed mandatory if going through an FFI layer, unless you use trait objects with a defined C layout, such as ::safer_ffi's ArcDynFn...

Thanks to you both. To be honest puzzling out this type of concurrency gives me a headache but it sounds like Rust can handle most of it.

One thing I'm a bit fuzzy on is this:

If you require your callback function type to implement Fn() + Send + Sync, Rust will check that for you.

How do I actually enforce that requirement? The callback function itself needs to be callable from C, with the right ABI, but aren't Fn() traits a Rust thing?

You create an extern "C" fn callback_pointer(user_data: *const c_void) that knows how to cast the data to the right type and call the rusty Fn().

Ah I see what you mean! Thank you. This is actually a lot simpler than I feared.

Oh, and also if you need this to be performant, you can avoid the double boxing by making it generic.

Something like that:

extern "C" fn callback_pointer<F>(user_data: *const c_void) where F: Fn() + Send + Sync {
  (user_data as *const F as &F)();
}

and give C callback_pointer::<F> pointer. Store the callback in Box<F>. For the F to be meaningful, it'll have to be in generic code (e.g. impl<F> on your struct) with where F: Fn() + Send + Sync bound.

That will force Rust to create for you a C callback function optimized for each unique kind of closure it's used with.

3 Likes

Interesting. Avoiding the double boxing this way is not even something the standard library uses, at least judging by the implementation of thread::spawn. (IDK, perhaps it’s intentional since it might save some code duplication during compilation, and spawning a thread is lots of overhead anyways... otoh the same code duplication can probably also be avoided differently, without the double-boxing..)

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.