How to take a generic, repeatedly iterable iterator over mutable data

help
#1

I want to write a function which takes an Iterator<Item = &mut u32> and iterates it repeatedly (the function implements some sort of iterative algorithm). Seems fairly straightforward, but I've been struggling with this for long enough that I'm starting to doubt whether that's even possible. I'll share below what I have tried so far, but it's well possible that I ran off in the wrong direction, in which case you may feel free to ignore my rambling and just show me how it's done.

Since iterators can't be restarted, and since iterators over mutable data can't possibly implement Clone, I thought I could achieve what I wanted by letting the function take an iterator-generating function as input, i.e. I thought I could do something akin to the following.

fn foo<'a, Iter>(mut iter: impl FnMut() -> Iter)
where
    Iter: Iterator<Item = &'a mut u32>,
{
    // Dummy function body
    for _ in 0..3 {
        for v in iter() {
            *v = 42;
        }
    }
}

Unfortunately, I get the following error when I try to call this function.

   |
13 |     let mut vec = vec![1, 2, 3];
   |         ------- variable defined here
14 |     foo(|| vec.iter_mut());
   |          - ---^^^^^^^^^^^
   |          | |
   |          | returns a reference to a captured variable which escapes the closure body
   |          | variable captured here
   |          inferred to be a `FnMut` closure
   |
   = note: `FnMut` closures only have access to their captured variables while they are executing...
   = note: ...therefore, they cannot allow references to captured variables to escape

I understand that what I've written can't work (there's no lifetime restriction on the return value of the closure, so if this were to compile then I could generate an arbitrary number of mutable references to the same data by calling the closure repeatedly), but I'm not sure how to fix it. The only solution I can think of would be to introduce my own trait which mimics the FnMut trait but allows me to restrict the lifetime of the return value, but that's starting to sound complicated enough that the cure might be worse than the disease. Isn't there a simpler way to do this in Rust?

#2

Is collecting an option in your use case ?

fn foo<'a, Iter>(iter: Iter)
where
    Iter: IntoIterator<Item = &'a mut u32>, // IntoIterator is more general than Iterator
{
    // Dummy function body
    let mut iter: Vec<_> = iter.into_iter().collect();
    for _ in 0..3 {
        for v in &mut iter {
            **v = 42;
        }
    }
}
#3

It could be if nothing else works, but not too fond of it due to the overhead.

#4

Maybe you should just accept a &mut [u8] slice? Do you need to call it with anything that you can't get a slice into?

#5

Yes. I think this is essentially the same solution as collecting and imposes the same trade-offs.

#6

Your example calls it with a vector. In that case, a slice is not equivalent to collecting because you do not need to create an extra vector to hold the references.

#7

If something like using a slice doesn't work, then I think the only way you can avoid the collect is to create your own trait to replace the FnMut() -> Iter trait.

1 Like
#8

You could try something like this:

fn foo<T>(collection: &mut T)
where
    for<'a> &'a mut T: IntoIterator<Item = &'a mut u32>, // IntoIterator is more general than Iterator
{
    // Dummy function body
    for _ in 0..3 {
        for v in &mut *collection {
            *v = 42;
        }
    }
}
6 Likes
#9

Thanks to all of you for your input. To summarise, I think we can categorise the proposed solutions into two categories:

  • Restrict the input to &mut [u8] / collect the iterator into a temporary. I'm inclined to treat these two as equivalent since collecting just reduces the general iterator to the slice case.
  • Introduce my own trait / leverage IntoIterator. Again, I'd say these are two minor variations of the same idea.

The trade-offs are as follows:

  • Reduce to slice:
    • Good: relatively straightforward and doesn't require much typing.
    • Bad: introduces an allocation in the general case.
  • Use a trait:
    • Good: no allocations
    • Bad: requires quite a bit of boilerplate in general (e.g., I would probably have to write my own implementation of IntoIterator for all but the simplest cases).

Not particularly happy with either of these solutions, but at least I'm more confident I didn't miss anything obvious, so thanks again to all of you!