Rationale behind Fn, FnMut and FnOnce design

Why not simply FnOnce, FnFew?

FnMut and Fn both can be called multiple times, but the difference is that FnMut cannot be called concurrently - this is necessary restriction to avoid data races, in case when closure is used to modify its environment.

5 Likes

FnMut acts like a &mut self receiver, and Fn acts like a &self receiver, which matters not only for concurrency, but also for variance.

(Example)

8 Likes

Can you give an example where Rust prevents concurrent use of FnMut but allows that for Fn?

Any place where function is called through shared reference. For example, any rayon::ParallelIterator method takes impl Fn and not impl FnMut, for exactly this reason - it will be shared between several threads, which would be impossible with FnMut.

3 Likes

For example, any rayon::ParallelIterator method takes impl Fn and not impl FnMut , for exactly this reason - it will be shared between several threads

Will Rust prevent sharing FnMut or it is just a convention?

Also:

  • If we have few Fn , one FnMut and they are not synced then we do have problems
  • If we have few Fn and few FnMut and they are synced then we do NOT have problems

So seems Fn does not solve concurrency problem by itself?

PS:

fn for_each<OP>(self, op: OP)
where
    OP: Fn(Self::Item) + Sync + Send, 

This is Sync bound which allows concurrent usage, not just Fn. FnMut can also be Sync:

trait MyTrait {
    fn for_each<OP>(self, op: OP)
    where
        OP: FnMut() + Sync + Send;
}

The simplest way to show concurrency is through parallelism, so you'll mainly see three kind of closures:

  • FnOnce(), for "the callee will need to call the callback at most once" kind of situations, which allows the closure to be optimized by consuming its captured environment when called. Std lib examples:

  • FnMut() for something that is callable multiple times sequentially / non-concurrently.
    Stdlib examples:

    • the whole plethora of Iterator adaptors / consumers

    • Sometimes with Send and/or 'static bounds for things called from another thread or abitrarily late respectively.

  • Fn() + Sync, for something callable in parallel (Fn() -> callable concurrently / through multiple / shared handles; Sync -> these handles can cross thread boundaries. Hence why it can be called from multiple threads at once). In practice, there are also + Send + 'static bounds, since the moment multiple threads are involved, the "end of life" flows can rarely be guaranteed at compile-time, which thus requires 'static bounds and dynamic / multiple ownership, e.g., Arcs.

    • Comparatively to the previous batch of stdlib single-threaded / sequential iterator adaptors which take FnMuts, ::rayon's own set of iterator adaptors / consumers which feature that Fn… + Sync (+ Send) bounds.

But for the sake of the example, you could, quite rarely, see a non-Sync Fn requirement, such as some thread-local hook: since the hook could technically be triggered when the very hook is being run (re-entrancy, another form of concurrency which happens not to require multi-threading to happen), then such a hook would need to be Fn().


If by sharing you mean sharing-and-using, then yes, Rust will prevent it, and that's the whole point about Rust, although not limited to Fn…s: if you are sharing something, you only get & access to it, and thus only have access to the &shared-compatible parts of that element's API, such as the &self-based methods. Calling a Fn is such as method.

If you are not sharing, then you can get a &unique access to it (dubbed &mut in Rust), and thus can use the stronger but more restrictive parts of the item's API: the &mut-based functions, such as &mut self methods, which includes calling a FnMut.


Indeed, and that's a code smell since it's "as useful as nipples on a breastplate": Sync allows &shared access to cross threads, but FnMut requires &unique access to be callable, so while you may be sending &shared-access-yielding handles across threads, you'll never be calling that FnMut().

2 Likes

It will not prevent sharing, but it will prevent calling. To call the FnMut, you must use unique reference to it, which is possible if you have either this (non-shared) reference or direct (also non-shared) ownership.

2 Likes

All in all, think of the Fn… traits as:

  • FnOnce() is CallableThroughOwnedAccess:

    trait CallableThroughOwnedAccess { fn call(self); }
    
  • FnMut() is CallableThroughUniqueAccess:

    trait CallableThroughUniqueAccess { fn call(&mut self); }
    
  • Fn() is CallableThroughSharedAccess:

    trait CallableThroughSharedAccess { fn call(&self); }
    

And since in Rust there are situations where the implementors, or the callback-callers require different levels of access to the callback, it is necessary to feature that trinity / trifecta / troika / triumvirate of traits, in the same fashion that we have self/&mut self/&self, or T/&mut T/&T, etc.

7 Likes

In Rust, soundness is never just a convention.

10 Likes

Right… actually, I think “it is necessary” is a bit too strong. There would’ve been the possibility to just have a single trait. Then e.g. instead of T: Fn(…) -> _, you would simply have &T: FnOnce(…) -> _. Moreover, it may even still be possible to change that in the future, making T: FnMut<Args, Output = O>, and T: Fn<Args, Output = O> mere trait aliases for for<'a> &'a mut T: FnOnce<Args, Output = O>, and for<'a> &'a T: FnOnce<Args, Output = O>, respectively.

Yeah, sure, and Future could be an alias for a HRTB-ed FnOnce, and more generally, you can express all method-based traits (≠ marker traits) as aliases of some FnOnce-based "expression" :grinning_face_with_smiling_eyes:

Be it as it may, the thing is that expressing the & vs. &mut distinction is an API necessity1, no matter whether they are featured as direct traits, HRTBs, or both/aliases.

1 Well, technically, Fn/&self could have been the only trait out there, and use interior mutability and .take() patterns to feature the other two, but I don't consider that introducing panic paths and a performance penalty count as a valid alternative.

Yeah… well, that’s not quite what I was after. See, Fn*-traits are not just about their call method’s signature but also about the fact that function-call syntax works with a type that implements them. If FnMut and Fn didn’t exist as separate things, then it might be e.g. easier (and less weird) to have things like &'a mut T: FnOnce() -> &'a Foo, i.e. closures that return values that borrow from the closure itself. And IIRC, if you tried that with FnOnce as it works today, the function-call syntax doesn’t turn out to work properly. Some testing later… yup, my memory was correct on that, here’s an example in the playground.

1 Like