Why de-structuring lambda arg makes a difference in enumerate + filter + map chain

Hey, beginning Rust after many years in C++ I'm still trying to get my bearings right wrt ownership&borrowing and found the following confusing:

I want to create an iterator that yields the even-indexed elements of the iterable, so I came out with

pub fn evens<T>(iter: impl Iterator<Item = T>) -> impl Iterator<Item = T> {
    iter.enumerate().filter(|(i, v)| i % 2 == 0).map(|(i, v)| v)
}

Then, just for curiosity, I tried to de-structure the filter's lambda parameter since it's a &Self::Item, like so (not the & in front of the tuple)

pub fn evens_destructured<T>(iter: impl Iterator<Item = T>) -> impl Iterator<Item = T> {
    iter.enumerate().filter(|&(i, v)| i % 2 == 0).map(|(i, v)| v)
}

And I get the dreaded E0507

error[E0507]: cannot move out of a shared reference
  --> src/lib.rs:17:18
   |
17 |         .filter(|&(i, v)| i % 2 == 0)
   |                  ^^^^^-^
   |                  |    |
   |                  |    data moved here
   |                  |    move occurs because `v` has type `T`, which does not implement the `Copy` trait
   |                  help: consider removing the `&`: `(i, v)`

For more information about this error, try `rustc --explain E0507`.

As I understand, filter's lambda receives a shared borrow and that doesn't change with me de-structuring it or not, so I'm a bit confused as I can't see why de-structuring makes such a difference.
Without de-structuring I can see the inferred lambda's argument is (i: &usize, v: &T) so there are still shared references at play, but no complaint from the compiler in that case.
My understanding is that

  1. enumerate vends tuples of (i, v)
  2. the filter's lambda gets a reference to each of them
  3. the bool result of the lambda decides which of them gets moved into map's lambda.

So there's always a final move into map and I can't see what difference de-structuring "in the middle" makes despite the seemingly crystal-clear compiler's error message.

To further confuse things, if I omit-with-underscore the "v" (which is actually the right thing to do as I have no use for it in the lambda), then the de-structured version is fine again

fn evens<T>(iter: impl Iterator<Item = T>) -> impl Iterator<Item = T> {
    iter.enumerate()
        .filter(|&(i, _)| i % 2 == 0)
        .map(|(i, v)| v)
}

Seems like the compiler sort of optimized away whatever was causing the problem before?
Any help in visualizing what's going on here to get a deeper understanding?

TIA,
Andrea

This isn't de-structuring per-se. This is known as a pattern in Rust.
When you write |(i, v)|, this pattern is matched against the expected closure argument type (usize, &T). Thus i is matched as usize and v is matched as &T - no moves are made.
The |&(i, v)| pattern when matched against (usize, &T), v receives the type T. How do you go from a &T to a T ? By de-referencing, which means moving out of the reference since T is not Copy. And thus you get the compiler error - and also a help that says how to fix it :slight_smile:

2 Likes

It's a little more complex, since filter passes a reference to the value into the predicate. That is:

  • In the first case, it's (i, v): &(usize, T), which is rewritten under the hood as &(ref i, ref v): &(usize, T) (search keywords are "match ergonomics"; not sure whether it was described somewhere in detail). Therefore, i is &usize, v is &T, all's OK.
  • In the second case, it's &(i, v): &(usize, T), i.e. (i, v): (usize, T)... but this step is only possible when (usize, T): Copy - otherwise, you can't destructure the reference to it.
5 Likes

Yess, that is the term I was looking for.
The RFC for it is here.

1 Like

Oh wow! I think I got it.
The following excerpt from the linked RFC is what made it click it for me

[...]this RFC introduces default binding modes used when a reference value is matched by a non-reference pattern.

In other words, we allow auto-dereferencing values during pattern-matching. When an auto-dereference occurs, the compiler will automatically treat the inner bindings as ref or ref mutbindings.

So, basically, when matching the &(usize, T) passed-in by filter with (i, v), since I'm matching a reference value with a non-reference pattern, the incoming reference is auto-dereferenced and the inner bindings are treated as ref, resulting in (&usize, &T) and hence no move.
When I instead match with &(i, v), the incoming reference-ness is pattern-matched away and I end up with (size, T) causing the move and the error.

Many thanks for the explanation and the link.

Andrea

2 Likes

Note that you can keep the & in the pattern if you don't move the value. Since you didn't actually need the v there, you can just not bind it, and things work again:

pub fn evens_destructured<T>(iter: impl Iterator<Item = T>) -> impl Iterator<Item = T> {
    iter.enumerate().filter(|&(i, _)| i % 2 == 0).map(|(_, v)| v)
                               // ^ If you don't bind it, it doesn't move
}

And while what you have is totally fine, you could also consider using filter_map here to do both steps at once. That gives you ownership of it, so you don't need to worry about the reference in the condition. You could write it like this:

pub fn evens_destructured<T>(iter: impl Iterator<Item = T>) -> impl Iterator<Item = T> {
    iter.enumerate().filter_map(|(i, v)| (i % 2 == 0).then_some(v))
}
2 Likes

Right, I noticed that using underscores (i.e. avoid the binding) would solve the problem, but I thought it was some sort of "optimisation", while from what you say I gather is rather something more structural, in the sense that without the binding the move is entirely not there.
Regarding filter_map, I knew about it, but I was trying to do things at a "lower-level" as a learning exercise.

Ah, yes, this is an important but subtle distinction. The _ pattern is special, because it doesn't just mean "bind and ignore it", it actually means "don't even bind it in the first place".

To demonstrate, note that https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=9cadf1f4380f31616a3e136d989090af

    let s: String = "Hello".to_owned();
    let _unused = s;
    dbg!(s);

Fails with

error[E0382]: use of moved value: `s`
 --> src/main.rs:4:10
  |
2 |     let s: String = "Hello".to_owned();
  |         - move occurs because `s` has type `String`, which does not implement the `Copy` trait
3 |     let _unused = s;
  |                   - value moved here
4 |     dbg!(s);
  |          ^ value used here after move

But if you change it to https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=54db2bedfe482539e6f35200185b408f

    let s: String = "Hello".to_owned();
    let _ = s;
     // ^ just the underscore!
    dbg!(s);

Then it works, because s isn't moved at all.

Admittedly that's often surprising for people at the top level, but it's consistent with how you really want it to work nested. It would be really annoying were if let Some(_) = x { moved the contents of the Option, when all you were doing was looking to see if it was Some or not.

3 Likes

That's very interesting and something I entirely missed from my first read of "the book"

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.