Clearing up some of my confusions about Closures

help
1

I'm learning closures for the first time; I find it hard so these below are questions to get a sense of my understanding and hopefully expand/correct it.

First, my summary is:

Trait Executions Move Into Move Out Mutation
FnOnce n<=1 Allowed Allowed Allowed (?)
FnMut n >= 0 Allowed Not Allowed Allowed
Fn n >= 0 Allowed Not Allowed Not Allowed
Source Paragraph
  1. FnOnce applies to closures that can be called once. All closures implement at least this trait because all closures can be called. A closure that moves captured values out of its body will only implement FnOnce and none of the other Fn traits, because it can only be called once.
  2. FnMut applies to closures that don’t move captured values out of their body, but that might mutate the captured values. These closures can be called more than once.
  3. Fn applies to closures that don’t move captured values out of their body and that don’t mutate captured values, as well as closures that capture nothing from their environment. These closures can be called more than once without mutating their environment, which is important in cases such as calling a closure multiple times concurrently.

Subsection in the book

More specifically, taking just:

FnMut applies to closures that don’t move captured values out of their body, but that might mutate the captured values. These closures can be called more than once.

I was confused by the "move in and move out" here. And others too.

Snippets and Questions

Here are some examples I made up and were tricky for me to analyse (alongside questions to get help / guidance with):

  • Example 1
fn main(){
  let x = String::from("hello");
  let move_x_into_closure = move || {
      println!("{}", x);
  };
  move_x_into_closure(); 
  move_x_into_closure();
  // println!("{x}"); // fails, x was moved into the closure.
}

So in this case, the ownership of x is moved into the closure (and out of x). One can still run the function many times.

  • Question 1: It's a bit mysterious to me where is the x stored? (Especially given the following example.)

This one would pass though, since the print only takes a borrow (unless we return or use x;):

fn main(){
  let x = String::from("hello");
  let x_stays_alive_outside_closure = || {
     println!("{}", x);
  };
  x_stays_alive_outside_closure();
  x_stays_alive_outside_closure();
  x;
}
  • Question 2: in both cases above the Trait should be Fn ?Since it does not move ownership out (which I understand as return x or just x in the last expression) , and it does not mutate any variable (just takes an immutable borrow.)

But if one writes instead:

fn main(){
  let x = String::from("hello");
  let move_in_out = move || {
     println!("{}", x);
     x
  };
  move_in_out();
//   move_in_out();
//   x;
}
  • Question 3: Now it's FnOnce ? So uncommenting fails.
2

Not too sure what I meant by question 1 so any help is welcome :joy:

3
  1. x is stored in the closure.

You can think of closures (loosely speaking) as structs that contain the values they capture (i.e. x in your example) and a function pointer.

  1. Yup, that's right. Since they capture a value and then keep it, they can be called multiple times.

  2. Exactly right again. It captures x, but returns it, so the second time you try to call it. It no longer owns x, so it can't be called again.

1 Like
4

it is stored inside the closure. if you think the closure type as a struct, x is just a field with type String.

correct, the closure implements Fn, but Fn is a sub-trait of FnMut, and FnMut is a sub-trait of FnOnce, so it actually implements all of them.

if you have difficulty understanding closures as invocable "function"-like objects, you can think as them as sructs which implements the Fn* family traits.

for some, thinking in terms of "moving-in", "moving-out" is easier to grasp, but for others, it may be even more confusing. here's an alternative view.

closures are just data type, like regular structs but defined with special syntax. the captured variables become fields of the struct, with same name.

for the "moving-in" part: when we say a closure captures the environment by-ref, by-mut, or by-value, we are talking about the type of the fields, and the "constructor" of the closure type.

for instance, the variable x has type String, here's the equivalent closure types:

let x: String = String::new();

struct CaptureXByRef<'a> {
    x: &'a String,
}

struct CaptureByMut<'a> {
    x: &'a mut String,
}

struct CaptureByValue {
    x: String,
}

which of form of the closure type will be picked by the compiler? the rules is simple:

  • if there's the move keyword, then capture the variable by value;
  • otherwise, infer it from the usage

for the "moving-out' part: when the closure is called as FnOnce, it receives self by value; when it is called as FnMut, the receiver is &mut self, and when called as Fn, the receiver is &self.

the compiler will synthesize the "least restrictive" impl for the closure automatically. this should be intuitive:

  • if the closure only need access the captured variable x immutably, i.e. the field &self.x, it only needs &self, i.e. Fn is implemented;

  • if it needs exclusive access to x, i.e. &mut self.x, then &self is not enough, so Fn cannot be implemented, instead, at least &mut self is required, so FnMut is implemented

  • if it accesses x by value, i.e. it moves the field self.x, it must use self, so only FnOnce is implemented.

for illustration, the example uses move for all cases, which is not necessary:

// case 1: this closure is `Fn`:
let x: String = String::new();
let c1 = move || {
    // example immutable access to x
    let _ = x.len();
};
// equivalent to:
struct C1 {
    x: String,
}
impl Fn for C1 {
    fn call(&self) {
        let _ = self.x.len();
    }
}

// case 2: `FnMut`, but not `Fn`:
let mut x = String::new();
let mut c2 = move || {
    // example `mut` access of x
    x.push('\n');
};
// equivalent to:
struct C2 {
    x: String,
}
impl FnMut for C2 {
    // need access to `&mut self.x`, cannot use `&self`
    fn call_mut(&mut self) {
        self.x.push('\n');
    }
}

// case 3: only `FnOnce`, neither `FnMut` nor `Fn`
let x = String::new();
let c3 = move || {
    // example `move` access of x
    let _: Vec<u8> = x.into_bytes();
};
// equivalent to:
struct C3 {
    x: String,
}
impl FnOnce for C3 {
    // we need to move `self.x`, so cannot use `&self` or `&mut self` 
    fn call_once(self) {
        let _ = self.x.into_bytes();
    }
}

as a (trivial) exercise, for C1, implement call_mut using call, and implement call_once using call_mut , this can help you understand why Fn is a subtrait of FnMut, and FnMut is a subtrait of FnOnce.

10 Likes
5

I think a part of confusion still remains:

From the table I wrote, I'd expect this:

#[derive(Debug)]
struct Rectangle {
    width: u32,
    height: u32,
}

fn main() {
    let mut list = [
        Rectangle { width: 10, height: 1 },
        Rectangle { width: 3, height: 5 },
        Rectangle { width: 7, height: 12 },
    ];

    let mut sort_operations = vec![];
    let value = String::from("closure called");

    list.sort_by_key(|r| {
        sort_operations.push(value);
        r.width
    });
    println!("{list:#?}");
}

sort_by_key uses FnMut, and to me it should pass the compiler. FnMut is allowed to "Move In" a variable. Only "Move out" isn't allowed.

I do see why the compiler wouldn't accept that (value won't be available at second run.) So it must be FnOnce but the table is synthesised from the paragraphs in the book, which are cited just below it. But here is the key piece I find confusing:

FnMut applies to closures that don’t move captured values out of their body, but that might mutate the captured values. These closures can be called more than once.

Again: I'm not "moving out of the body" just moving into.

6

That closure is moving out of itself. You passed the value into the push() call, so the closure no longer owns it.

“Move out” doesn’t mean the closure returns the value. It means that the closure loses ownership of the value, by any means.

3 Likes
7

Fair enough, I missed that. Thank you.

8

I think the second sentence answers my question. But what does "closure is moving out of itself" mean?

Do you mean that it is moving the value to push, and so it loses ownership; but closure is the function, not the value, right?

I interpret you refer to the value there, in the first few words.

9

When learning Rust, I found it immensely useful to not rely on type inference or lifetime elision; so some parts of the original post could have been answered by simply defining polymorphic functions and seeing if certain closures could be passed to the functions.

FnOnce is about allowing closures to safely move captured data without UB. For example, if a closure captures a String and drops it; it clearly cannot be allowed to be invoked more than once or a double free would happen. Note, that while you will need to declare a closure move sometimes if you need to move the data; the reverse is not true. For example a closure declared move can still implement Fn so long as multiple invocations are not a problem.

FnMut is about allowing non-moved data to be mutated, but this also needs care since you don't want to allow data races from mutating the same variable at the same time.

Here are some examples:

struct NotCopy;
fn main() {
    let f_1 = || {};
    fn_(f_1);
    // `f_1` is `Copy` as well since all of the data it owns is `Copy`, so we can still "use" `f_1`.
    fn_once(f_1);
    let not_copy = NotCopy;
    let f_not_copy = move || {
        _ = &not_copy;
    };
    // Even though we `move`d `not_copy` and thus `f_not_copy` owns it, `f_not_copy` doesn't actually do anything
    // with it that makes multiple invocations problematic; thus it still implements `Fn`.
    fn_(f_not_copy);
    // `f_not_copy` is not `Copy` since at least one of the data it owns (i.e., `not_copy`) cannot be copied.
    //fn_(f_not_copy);
    // Don't mistake local variables for the data that is owned by the closure though. So the below closure
    // will still implement `Fn` and `Copy`.
    let f_copy = || {
        let mut val = String::new();
        val.push_str("hi");
        drop(val);
    };
    fn_(f_copy);
    fn_once(f_copy);
    fn_mut(f_copy);
    let f_2 = move || {};
    fn_(f_2);
    let x = "".to_owned();
    let f_3 = || {
        drop(x);
    };
    fn_once(f_3);
    // The whole point of `FnOnce` is to allow closures to `move` data, but once data has been `move`d; it can't
    // be `move`d again. The way it makes this safe is by ensuring the closure is called at most once; thus
    // `move`s can happen at most once. For example, if I drop a `String`; I can't drop the same `String` due to
    // a double free.
    // Seeing how `FnMut` can be called an arbitrary number of times, we can't allow an `FnOnce` closure to be
    // a subtrait of `FnMut`.
    //fn_mut(f_3);
    // Read above.
    //fn_(f_3);
    let mut y = 0u32;
    let f_4 = || {
        y += 1;
    };
    fn_mut(f_4);
    // `f_4` cannot be copied since it owns a mutable/exclusive reference.
    //fn_mut(f_4);
    // `Fn` cannot mutate data it captures, so this won't compile.
    //fn_(f_4);
    // We could of course `move`/copy a variable into another local variable and mutate it.
    let f_5 = || {
        let mut z = y;
        z += 1;
        let _ = z;
    };
    fn_(f_5);
    // `f_5` can be copied since the captured data (i.e., a shared/immutable reference) is `Copy`.
    fn_(f_5);
}
fn fn_once<T: FnOnce()>(f: T) {
    f();
    // `f` can be invoked at most once (hence the name).
    // If you uncomment below, it won't compile.
    //f();
    // `FnMut` is _not_ a supertrait of `FnOnce` since calling code can do things with `FnMut` closures that they
    // can't with `FnOnce`.
    //fn_mut(f);
    // `Fn` is _not_ a supertrait of `FnOnce` since calling code can do things with `Fn` closures that they
    // can't with `FnOnce`.
    //fn_(f);
}
// If I remove `mut`, then it won't compile since `FnMut` mutates `f`.
fn fn_mut<T: FnMut()>(mut f: T) {
    f();
    f();
    // `FnMut` is a supertrait of `FnOnce` since calling code can do anything with them that they can do
    // with `FnOnce`.
    fn_once(f);
    // `Fn` is _not_ a supertrait of `FnMut` since calling code can do things with `Fn` closures that they can't
    // with `FnMut`.
    // fn_(f);
}
fn fn_<T: Fn()>(f: T) {
    f();
    f();
    // `Fn` is a supertrait of `FnOnce` since calling code can do anything with them that they can do
    // with `FnOnce`.
    fn_once(f);
    // `Fn` is a supertrait of `FnMut` since calling code can do anything with them that they can do
    // with `FnMut`.
    // Ths won't compile purely because `T` is not `Copy` and we moved `f` above. Add `Copy` to `T`, and
    // this will compile just fine. Alternatively comment above and uncomment below, and it'll compile.
    // fn_mut(f);
}
1 Like
10

I can’t quite parse your question, but I'll say this:

A closure expression |r| { sort_operations.push... produces a closure value. The closure value owns a String. When the closure value is called, the closure's code runs, and the closure value stops owning a String because the closure’s code moved the String out of the closure value by passing it to push().

Does that help?

1 Like
11

I think so. I haven't read the reference, just the book, where the explanation at least so far wasn't very much in-depth.

It seems to me that you are saying that the closure expression produces the struct-like thingy named earlier (as value), and this value which is callable is then called – finally executing the function's body and the variables. But it may be nonsense from my part.

At the moment, and assuming the par above is right, it doesn't make much sense that the "struct" is produced every time the closure is called I guess (for cases other than FnOnce), or it would be slow. Maybe it's produced just once. But I don't know. This isn't in the book so far. Nor it is that important right now I guess; I was just pondering about it.

In any case, moving out of itself it's more clear now, I believe. It means moving out of the "struct", in my mind.

I will have to come back to it tomorrow.

12

In the reference there's an example of the struct that we can think of as a closure value:

For me, when learning about closures it helped a lot to think of this struct as the closure value.

13

That is correct.

It is not. You have the closure value, and then call it. You can't call it without already having it. It isn't re-made when you call it.

Side note, not relevant to what is actually happening here: creating a struct is not “slow”. It consists, at most, of moving the values that make up the struct together in memory, if they weren't already.

14

had to skip part of your reply yesterday because it was too advanced. it's a little bit more clear now, although i didn't implement anything, and i doubt i can. thank you.