Iterators with owned objects with lifetime

I need help to create an iterator that returns an object with lifetime.

Here is what I tried:

struct VecEntryMutable<'a> {
    vec: &'a mut Vec<u8>,
    element: u8,
    index: usize,
}

impl<'a> VecEntryMutable<'a> {
    fn set(&mut self, new_value: u8) {
        *self.vec.get_mut(self.index).unwrap() = new_value;
    }
}

#[derive(Debug)]
pub struct IterMut<'a> {
    vec: &'a mut Vec<u8>,
    index: usize,
}

impl<'a> Iterator for IterMut<'a> {
    type Item = VecEntryMutable<'a>;
    
    fn next(&mut self) -> Option<Self::Item> {
        if self.index >= self.vec.len() {
            return None;
        }
        
        let element = *self.vec.get(self.index).unwrap();
        let entry = VecEntryMutable {
            vec: self.vec,
            element: element,
            index: self.index,
        };
        
        self.index += 1;
        
        Some(entry)
    }
}

Unfortunately, I get a weird error, saying that the lifetime of the object I returned is actually tied to the lifetime of the iterator, which is implicitly defined, rather than the one I provide as 'a. I think the compiler is confused and picks the wrong lifetime for the returned value, as in all the places I can think of, the actual lifetime of an object I returned is 'a and should be. I get this error instead:

error: lifetime may not live long enough
  --> src/main.rs:36:9
   |
19 | impl<'a> Iterator for IterMut<'a> {
   |      -- lifetime `'a` defined here
...
22 |     fn next(&mut self) -> Option<Self::Item> {
   |             - let's call the lifetime of this reference `'1`
...
36 |         Some(entry)
   |         ^^^^^^^^^^^ method was supposed to return data with lifetime `'a` but it is returning data with lifetime `'1`

The example I provided is an MCVE for the problem I have, the example code might not make much sense, but the actual code I am working on does and is a little different, but this is off-topic here, since the problem is isolated and visible very well in this example, and the only problem is tied to the lifetimes confusion.

Not possible, next question.

It's because itratator interface works like that.

The compiler is correct.

Intuitively: someone could just do your_iterator.collect() and get multiple VecEntryMutable, each with its own &mut Vec<u8> that point to the same Vec<u8>, which is invalid because there can be only one &mut reference to a value valid at any point in time.

Regarding the lifetimes you think the compiler wrongly picked: the vec field of IterMut has type &'a mut Vec<u8>, so you may think that doing self.vec would yield you a &'a mut Vec<u8> but that's wrong, it gives you a &'b mut Vec<u8> where the 'b is the lifetime of the &mut self parameter (as if you had written fn next<'b>(&'b mut self) -> ...). The problem is that you don't own that &'a mut Vec<u8>, you only have a mutable reference to it (you can imagine a &'b mut &'a mut Vec<u8>). In the end you only have access to it for 'b, because after that lifetime ends someone else gets back access to the &'a mut Vec<u8>, so you cannot hold onto it for more.

2 Likes

The term for an iterator that can hand out borrows of itself is a lending iterator (or sometimes streaming iterator, but that is too easy to confuse with async streams).

As the other replies said, the Iterator trait in std doesn't support lending iterators.

If you can refactor your code so that the iterator truly gives up its access as it returns items, that may be a way forward. Consider wrapping IterMut, or here's a wordy walkthrough of implementing a mutable slice iterator.

Or just don't use Iterator and write an inherent (lending) next method, usable like so:

while Some(item) = lending_iter.next() {
}