Creating an iterator where the underlying datasource doesn't live long enough

help
1

Hey!
I'm using a piece of code that reads data from a file and parsing the data as protobuf objects using this protobuf crate. I want to expose the data I read via an iterator.
My code looks like this:

pub fn foos_iterator(src_file: &str) -> FoosIterator {

    let mut file = io::file(src_file);
    let stream = CodedInputStream::new(&mut file);

    FoosIterator {
        file,
        stream
    }
}

pub struct FoosIterator<'a> {
    file: File,
    stream: CodedInputStream<'a>
}

impl<'a> Iterator for FoosIterator<'a> {
    type Item = Foo;

    fn next(&mut self) -> Option<Foo> {
        // Return a `Foo` using `self.stream`.
        //
        // Placeholder for now:
        return Some(Foo{});
    }
}

Now, the Rust compiler complains at the line let stream = CodedInputStream::new(&mut file); that the borrowed value does not live long enough. I understand the error and why it's being pointed out by the compiler. My question is, what is the idiomatic way to model this idea in Rust?
(The relevant documentation for CodedInputStream::new is here.)

Any help is appreciated!

2

The straightforward solution would be to add one extra layer of indirection, so to speak; foos_iterator() shouldn't return the iterator, but rather something that owns the File. Then hang an iter<'a>(&'a self) -> impl Iterator<Item = Foo> + 'a method on that, and return the FoosIterator from there. FoosIterator would just have a CodedInputStream inside.

Here is a stripped down illustration of the above.

1 Like
3

@vitalyd Thank you so much for writing in (especially the code sample).

This solution worked for me. :smiley: Although I don't understand this bit: -> impl Iterator<Item = Foo> + 'a

As I understand, this bit of code says 'this function returns some entity that implements the Iterator trait'? Is there some documentation for this pattern?

Also, is it just me or does this look like Rust forcing me to bend over backwards to express the idea I was going for?

4

This feature is called "impl trait in return position" - not sure if it's officially documented in the book just yet as it's a relatively new feature, but you can find a good amount of info on it via google/search.

There's no language support for self-referential structs, which is what your initial attempt boiled down to. There're some crates that support it to some degree (e.g. rental), but I don't know if you'll find it better than just doing the above. So I suppose the answer to your question is "yes", you'll need to bend over backwards if you find yourself in a situation where a self-referential struct would be the "natural" choice.

1 Like
5

Thanks @vitalyd.

For posterity, here's the relevant documentation for impl trait in return position.