Returning iterator seemingly requiring multiple liftetimes

help
1

Here is a MWE of the problem I am having:

struct Values {
    vals: Vec<u32>,
}

struct Validator {
    valid: u32,
}

impl Validator {
    fn is_valid(&self, value: &u32) -> bool {
        *value == self.valid
    }

    fn valid_values<'a, 'b>(&'a self, values: &'b Values) -> impl Iterator<Item = &'b u32> + 'a {
        values
            .vals
            .iter()
            .filter_map(|v| if self.is_valid(v) { Some(v) } else { None })
    }
}

fn main() {
    let validator = Validator { valid: 5 };
    let values = Values {
        vals: vec![1, 2, 3, 4, 5, 6, 7],
    };

    assert_eq!(
        validator.valid_values(&values).collect::<Vec<&u32>>(),
        vec![&5]
    );
}

What I am trying to do should be obvious: the valid_values function should return an iterator adapter that has a filter applied. However, I cannot get this to compile no matter what I try due to lifetime issues. I understand that, given the lazy nature of iterators, that the returned iterator would ultimately need to contain a reference to the self Validator struct due to the closure, as well as to the values argument. I cannot figure out how to express these lifetime dependencies to make the compiler happy! Anyone know how this can be accomplished?

1 Like
2

You'll need to add the move keyword before the closure

3

This, and in this example you also need to handle the fact that the returned Iterator, capturing both a reference of lifetime 'a and 'b doesn’t not fulfill the + 'a bound. I think the best approach might be to introduce a 'b: 'a relation between the lifetimes, i.e.

    fn valid_values<'a, 'b: 'a>(&'a self, values: &'b Values) -> impl Iterator<Item = &'b u32> + 'a {
        values
            .vals
            .iter()
            .filter_map(move |v| if self.is_valid(v) { Some(v) } else { None })
    }

this should not introduce any significant restrictions on the caller: the returned iterator can only be valid for the shorter of the two lifetimes 'a and 'b anyways.

  • If you have a &'a Validator and a &'b Values where 'a is the shorter lifetime, you can call valid_values and get a impl Iterator<Item = &'b u32> + 'a where + 'a is indeed the shorter of the two lifetimes
  • If you have a &'a Validator and a &'b Values where 'b is the shorter lifetime, a valid_values call will still work because the &'a Validator will just be coerced into a &'b Validator, and you’ll get a impl Iterator<Item = &'b u32> + 'b which is having a + 'b bound, and 'b is indeed the shorter of the two lifetimes in this case

Edit: I meant to answer @kyp44 with this post


For the record, an alternative function signature is (click to expand…) 
    fn valid_values<'a: 'c, 'b: 'c, 'c>(&'a self, values: &'b Values) -> impl Iterator<Item = &'b u32> + 'c {
        values
            .vals
            .iter()
            .filter_map(move |v| if self.is_valid(v) { Some(v) } else { None })
    }

which works by introducing another lifetime which kind-of takes the role of the shorter of the two lifetimes 'a and 'b.

1 Like
4

Great responses, thanks!

I knew that the Iterator could only ever live as long as the shorter of the two references, I just wasn't sure how to express that to the compiler. I also wasn't familiar with the lifetime sub-typing notation, I must have glossed over the relevant section of the book. While more verbose, I think I prefer your alternate solution, since it makes the lifetime of the Iterator relative to the other references more explicit.

I'm actually more confused by the necessity of adding the move keyword to the closure. I think I understand the semantics of that in that captured values are moved into the implicit context struct that accompanies the closure. In this case though the only thing captured by the closure is the &self reference. Since this is itself a borrowed reference, this just means that the reference is moved and not the Validator struct itself, correct? I think this has to be the case since a) the Validator itself was never moved into valid_values and b) I confirmed that I am able to use validator in main after the call to valid_values.

Again, thanks for the great responses, they totally solved the issue and clarified the problem.

5

yes, and without it, the reference would be borrowed (think &&T). Also, “move” for Copy types like references actually means “copy”. The move keyword is pretty much always needed when a closure is returned from a function, otherwise you would try returning a borrow of a local variable. Well, at least move keyword never hurts when retrining a closure from a function, if the closure doesn't capture anything or all uses of captured variables already require owned captured variables, then the move keyword doesn't change anything and thus it isn't needed. In case that isn't clear, in your function the closure is not “directly” returned from a function, but it's placed into an iterator adapter and that iterator is returned from a function.

1 Like
6

Yeah, gotcha. I was also thinking that not moving would result in a &&T, but I also understand the Copy thing when "moving" a reference, though it never really occurred to me that references would be Copy even though that makes total sense (since they are effectively just pointers).

I also get why you'd pretty much always use move when returning a closure (or something that depends on one as in our case) since anything captured from the function would need to outlive the function.

Thanks for the explanation, this has been enlightening!

7

to be clear, in case that isn't obvious: only shared references &T are Copy, while &mut T aren't. This Copy implementation can also be seen in the docs.

8

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.