Iterator lifetime


#1

I’m trying to rewrite python generator to rust, which forces me to manage state.

Python code looks like this:

def foo():
  # some code that may yield
  for x in my_iter:
    if cond(x):
        yield x

Trivial there, but hard in rust.

So I have s struct, where I’d like to store iterator:

struct Foo {
    my_iter: Option<Iter<Vec<& str>>>
}

and then I can implement Iterator trait with next that manages everything.

Now according to rust I need to put lifetime definition in there somehow… but I can’t figure out the syntax.
I expect this iterator to live as long as the struct, but every attempt to define that is rejected by compiler (I’m trying nightly). Or is there better way to achieve that?


#2

Might be that I don’t understand something, I don’t understand why Iter is there.
Would this be ok?

struct Foo<'a> {
    my_iter: Option<Vec<&'a str>>
}

Note that you have a reference which means that you have to indicate a lifetime if the compiler can’t infer it.
Why I am asking about Iter is because Vec is already iterable…


#3

Its there because I need it. I want to store iterarator itself, not a vector.

so somewhere in implementation of next() (or in initialization) for my structure I want to assign some iterator to self.my_iter, so that subsequent calls can use it. If I would iterate over a vector I could store its index, but I am not,
so this doesn’t work for me.

Or in other words: having an iterator, how do I define a type that can store it. Lifetime definition syntax confuses me and my guesses confuse compiler.


#4

You can implement the Iterator trait for your structure, look into std::iter::Iterator.
You would have only to decide how next() would work on the information you have.
So you would have:

struct Foo<'a> {
    my_iter: Option<Vec<&'a str>>
}

Storing the data, and the implementation of the Iterator trait to offer you the possibility to iterate over the elements in your structure. Of course, may be you need to add more info into the structure, based on your specific needs.

Now I re-read. Sorry, I might not understand this. You want a container to store iterators(references into iterable structures)?
You would probably have to enforce no mutability then, in order to not invalidate the references. That would not be very helpful.
Definitely not understanding the requirements, sorry…


#5

I think I am struggling to communicate what I am doing. You are assuming that I am iterating over a vector. I am not.

Once again this is the logic I am trying to replicate;

def makefoo():
    for item in makebar():
        if i_like(item):
            yield item

Python magically manages state of makebar for me. Subsequent calls to next(makefoo)
will maintain state of makebar. No such magic exist in rust, so I have to keep state of some iterator I am using internally myself.

So within my iterator, I want to use other iterator and so I have to store its state. It is an iterator that calculates something somehow, not a vector. It does not iterate over a vector. Lets consider its a blackbox.

How do I do this?


#6

Just in case you aren’t aware, but you’re describing a filter operation; your makefoo example is basically this:

makebar().filter(i_like)

Anyway, you can look at the various existing iterator adaptors to see how they do this. Or look at itertools which is a crate filled with iterators. They all do more or less the same thing: take the inner iterator type as a type parameter:

struct Foo<It> { my_iter: It }

The alternative is to use a trait object instead; this is slower to execute and requires more memory, but is more flexible at runtime (if you’re adapting Python code, that might be important):

struct Foo { my_iter: Box<Iterator<Item=String>> }

Edit: Just realised I didn’t answer the original question.

Every borrowed pointer (i.e. &_ or &mut _) must have a lifetime associated with it. In most code, you can leave it out and let the compiler infer it, but you can’t do that with interface definitions (i.e. types, functions, traits): you have to specify where the lifetime is coming from.

So in your original example, you’d need something like:

struct Foo<'a> {
    my_iter: Option<Iter<Vec<&'a str>>>
}

It’s worth pointing out that you don’t need to do this if you avoid ever mentioning anything that directly requires a lifetime. Foo<It> doesn’t require any lifetimes because those are hidden inside the It parameter (i.e. It might be a type that involves lifetimes, but Foo doesn’t need to know or care about that).