Are there any good examples of APIs that use the Fn trait in the standard library?
I'm preparing a talk about closures, Fn traits, and their differences. I would like to give examples of APIs in the Rust standard library that use closures and why they use the traits that they use. It is very easy to think of and find examples for FnMut (lots of examples on Iterator trait) and FnOnce (threads, option functions, etc.) but I can't find any for Fn trait.
I know there are lots of examples for Fn trait in rayon but I would like to stick to the standard library.
Did you know you can actually search for this with Rustdoc? If you search "Fn" (including the quotes), that excludes results which mention FnMut or FnOnce instead. Switching to the arguments tab, we can see that std::panic::take_hook is the only[1] function in std to take a Fn.
FnMut is strictly more flexible, unless you need Fn + Sync for threading like rayon. It's preferable for the standard library to allow that flexibility as much as possible.
Just to note that searching "FnMut" doesn't even shows Iterator::map function. It seems it can only detect if the trait name is directly in the parameter position, like impl Fn or dyn Fn. But most stdlib functions use generics.
I think it's somewhat natural that std rarely requires a Fn bound. It's the most restrictive upon the function implementing it, and the places where I find it useful are often places which are inherently “opinionated” (making more fixed choices than std prefers to), like for example Arc<dyn Fn() + Send + Sync> for some kind of notification mechanism. Similarly, std provides Arc but hardly uses it (only in std::task::Wake which is the kind of situation that needs it).
I searched my local copy of rust standard library using ripgrep and it seems that panic hooks are the only public items that use Fn. It is being used in other places but mostly in tests and some private items.
I don't want to explain panic hooks so I think I'll just use rayon for my examples.
I think rayon is a great example. I’ve liked the way, rayon was used as an example e.g. in this video (the relevant part is starting roughly at the linked time, and goes on for a good 3min), where a comparison to C++ shows once again nicely how good (and almost magical) Rust’s thread-safety features are, and demonstrating how that relates to small differences in the type signatures of Iterator’s for_each method vs. the parallel iterator one.
As it happens, I’m currently living in Kyoto (for a handful more weeks) and I saw your talk announced some place online; I don’t feel like I personally have a lot more to learn about the topic of closures / closure traits though, so I’m uncertain whether or not I should attend, and what the target audience is; on the other hand, I haven’t really attended any in-person meetups about Rust, so it might be worth it anyways? What do you think?
std doesn't actually follow this, but my advice would be to use Fn if you're not going to guarantee the order of calls to the function.
For example, it's not at all obvious to me that someone calling sort_by can do anything particularly useful with it being FnMut(&T, &T) -> Ordering. After all, there's no guarantee on call order, and thus the state in use must not affect the resulting ordering.
(Sure, you can track things like "how many times was it called", but you could do that with a Cell in a Fn too, and personally I think that having to use interior mutability is a helpful "you're doing something a bit odd" reminder.)
And then there's a bunch of things that use FnMut and document the call order because there's plenty of useful stateful things you can do with them. For example, https://doc.rust-lang.org/std/vec/struct.Vec.html#method.retain specifically mentions that things are visited in-order so you can use state to do things like remove every other item.
But really, the most important reason that things use Fn instead of FnMut is for threading, and core doesn't do any of that, so it's almost all FnMut (if it's not just FnOnce).
It would be great to come to the meet up. The topic is geared towards beginners with some experience but it will not take up the full time. I'm expecting at least half of the time will be spent on free form discussion.
I chose an easier topic because my last meetup was extremely advanced and I realized it wouldn't be suitable for most people after no one showed up. I decided to focus on beginner friendly topics until I know roughly the general level of people who join the meetup.
If you're interested I could talk to you about my last topic (implementing iterator on tuples) after the main presentation.
Since mut in Rust serves two purposes (if I understand it right), exclusiveness and semantics of "mutation", wouldn't it be a possible approach to use Fn in cases where the caller strictly expects no side-effects happening?
On the other hand, taking Fn instead of FnMut just because of semantics might be bad because this would, for example, make implementing a cache more difficult (which then would require adding interior mutability again).
I see that Iterator::map also takes an FnMut closure (probably to allow side-effects or remembering the state, or just because it's technically not necessary to share the reference to the closure).
Using Fn would still make it more difficult to add a caching mechanism, for example. So maybe this advice could be extended:
Use Fn if you're not going to guarantee the order of calls to the closure and if the closure might be called from multiple threads in this or later implementations. But I notice this is exactly the example @cuviper mentioned already:
This brings me to the question: Is there any non-semantic reason to use an Fn + ?Sync ever?
Interestingly, that one is also requiring Sync.
Maybe one scenario for Fn + ?Sync could be invoking a closure from a local task set in arbitrary order.
E.g. if you have a closure stored in a thread-local. The Fn then covers the fact that invocations of the closure could itself access the thread-local and thus call the closure a second time, “recursively”.
Or if you want to store a closure in some data structure involving Rcs and don’t want to employ interior mutability for the closure calls themself.
The case you mentioned of sharing a closure between different futures or tasks that are executed concurrently on the same thread is of course also valid; so e.g. multiple Futures being run concurrently via things like select or join, or multiple tasks executed in a local set as you suggested.
I think Rc is a very good example because it doesn't need to cover advanced topics like Futures or thread local storage.
Note, an API might also demand a closure to implement Fn if later implementations might store the closure in an Rc (or run it in a local task set, keyword: coroutines?), even if it's not done in the current implementation.
This brings me back to:
Update:
I.e. if the provider of the closure isn't expected to need state mutation, this leaves room for the caller of the closure to maintain flexibility in storing the closure in an Rc, using it in a local task set, etc.
The “Rc usecase” is not advanced by itself, on the other hand it’s very abstract. You’d still want to come up with an actual use case of needing a bunch of closures in Rc-heavy data structures, or shared-ownership closures behind Rcs themself (i.e. Rc<dyn Fn(…) -> …>). Making this concrete in a non-artificial manner might bring back complexity.
On that point, it is both true that shared references still allow side-effects, and that exclusive references can have good use-cases in side-effect-free situations.
E.g. you can use a &mut reference to access the value contained inside of a RefCell or Mutex without needing to invoke the locking mechanism, via their respective .get_mut methods (or in the case of Cell, without &mut, you would not get any by-reference access to the contained value at all). This can be useful even for read-only access, if you happen to have exclusive access anyways, so it does – in my opinion – seem somewhat suboptimal to take such abilities away from users.
Good documentation is necessary anyways, because with interior/shared mutability, it’s possible to rely on order of effects even with Fn closures, so documentation should be clear about freedoms in execution order.
allowing flexibility for the caller to use shared references to the closure (in the current or future implementations which do not change the API),
allowing more flexibility for the callee (e.g. allowing faster Mutex unlocking, as in your example).
So maybe a possible advice could be the following:
Use Fn + Sync if you need to call the closure from multiple threads.
Use Fn + ?Sync if you need to store shared references to the closure (e.g. in a thread-local or Rc) and call the closure through these shared references or if you might want to do so in future (assuming maintaining this possibility weighs more than giving freedom to the provider of the closure).
Note that it's very normal for a caching mechanism to use interior mutability. After all, the nature of a cache is to be shared, which requires sharable references.
What would be the consequence of this observation? Use Fn where there are no side-effects other than caching expected (such that the caller of the closure maintains more freedom)?