Most general non-generic shareable fun type?

What is the most general non-generic callable reference type (for a specific signature) that can be shared across threads and called multiple times? It should allow funptrs and closures.

Is it: &'static (dyn Fn(SomeArgs) -> SomeReturnType + Sync + 'static)? Or is there something more general?

&'static is the least general possible (assuming you want this in the argument of a higher-order function); it will not allow anything that captures a reference to anything local.

Furthermore, FnMut is more general than Fn – in exchange, you have to do the synchronization yourself.

Most general for whom? There are tradeoffs between closure-writer and closure-consumer.

Sharing how? Scoped threads or spawned? Closures are stored, or passed in, used, and dropped?

4 Likes

Sorry - not enough details.

This function reference type will be part of a crate interface (args of trait methods in the interface), and the functions satisfying this type are stored and called by spawned threads. The function (or closure) writer should be given most freedom. I'm writing the consuming crate, so I can do whatever is needed on my end to make it work.

In general, for sharing an arbitrary callback function with spawned threads, you want Box<dyn … or Arc<dyn …, not &'static, because &'static imposes the requirement that the function(’s captured state) can never be deallocated. But beyond that, there are still choices to make.

  1. You can let the function author write a FnMut, but add a mutex around it so that calls are guaranteed to be non-overlapping in time (no simultaneous access to the mutable state). This gives the function author the ability to freely use mutable state in their function, but means they cannot support parallel execution of the function from multiple threads, which may become a bottleneck for the performance of your program.

    The type for this would be Arc<Mutex<dyn FnMut(SomeArgs) -> SomeReturnType + Send + Sync>> (with implicit + 'static bound).

  2. You can let the function author handle synchronization themselves, by using Fn instead of FnMut. This has no effects on functions that don't need to mutate internal state, but if they do, they have to use Mutex or atomics or something. The advantage in that case is that they can choose the best type of synchronization for their job, and narrow its scope so that contention is as low as possible (or absent).

    The type for this would be Arc<dyn Fn(SomeArgs) -> SomeReturnType + Send + Sync>> (with implicit + 'static bound).

  3. You can require that the function be clonable, F: FnMut(…) + Clone. (Closures implement Clone if their captures do.) This way, the function can be cloned and handed to each thread, which can then call their own clone without any synchronization at all — but the duplication of captured state might be surprising to the supplier of the function.

    You'd want to document this carefully.

    The type for this would be tricky; you'd need to either make all the clones you need at setup time before you type-erased the function to Box<dyn FnMut(…) + Send>, or to use a custom trait object and a helper like dyn-clone.

Arguably option 3 is the most general because you can adapt the other two to it, and option 2 the second-best because you can adapt option 1 by adding a Mutex, but there is no single type that will serve all three; they impose different requirements on the caller.

Personally, I would choose option 2 unless the specific situation suggested one of the others.

6 Likes

In this app, there is no allowed sharing across threads after initialization. In fact, a command-line option switches between multi-threaded and fork subprocesses modes, and the functions have to work identically in both cases (in their own single threaded process, or within a thread of a multi-threaded process). These functions are gathered during initialization, when there is only a single thread, but run in the separate sub-threads/processes after initialization.

I think this means that option 3 might be best.

As a matter of API design, it's possible to offer a generic public method, and immediately inside that method convert the generic argument to a non-generic Box<dyn FnMut(...)> or Arc, etc. This would be a more natural API for users—usually hiding the box/arc/dyn details is good.

Sorry if this is irrelevant—I understand you asked specifically for a non-generic type.

3 Likes

Oh - that might be better. The reason I asked for non-generic was just because I don't want to genericize everything internally that has to store or handle that function type.

Then Send + FnMut(T) -> U is all you need.