Wrapping Peekable::peek() as an iterator


#1

Hi! I have a problem where I am computing an infinite sequence of things and checking them against subsequent elements in the sequence. Computing the things themselves is expensive (it’s a cryptographic hash) so I’m trying to use itertools::multipeek to “precompute” them and then re-reveal them later. The natural way to loop over the future ones would be using an iterator, so I’m trying to wrap this peek method in my own iterator. Unfortunately, I can’t get the lifetimes to work out.

I’ve put together an MCVE at https://play.rust-lang.org/?gist=c6c10a02af83c2ba9b949d3ff6b3bc79&version=stable&backtrace=0 . Instead of using the itertools crate, I’m just using Peekable::peek() from the stdlib.

Compiling, I get “cannot infer an appropriate lifetime for autoref due to conflicting requirements”. What’s going on? I thought the lifetime of the return value of peek() would be the lifetime of the Peekable, which has a lifetime determined by the parameter to the PeekFuture (so, 'a in this example). But it seems like the compiler is trying to tie it to the lifetime of the PeekFuture reference ('b), which I don’t think is the same thing? So how do I fix it?


#2

This is a limitation of the Iterator trait - it cannot return values that are borrowed from the iterator itself. Check out the streaming-iterator crate for a workaround. You should be able to implement streaming_iterator::StreamingIterator for your type rather than Iterator.


#3

OK, thanks for the answer. I tried to implement the same thing using streaming-iterator and got mostly the same result. Multipeek::peek() does an advance and a get, so I needed to keep a reference to the most recent peek() result, which I called current_item. But assigning that gives me the same “conflicting requirements” error. Here’s a link to the playground (although it doesn’t work on the playground because crates aren’t allowed I guess):

https://play.rust-lang.org/?gist=b7b2d6ed648e44f08da88a8be49a3e53&version=stable&backtrace=0

It seems like I can get past this particular error by changing the signature of advance from fn advance<'b>(&'b mut self) to fn advance<'b>(&'a mut self) – in other words, by requiring that the lifetime of the item be the same as the lifetime of the PeekFuture iterator (which isn’t allowed by the definition of StreamingIterator). What now?


#4

Oh you’re right, it is exactly what you wrote in the last paragraph. StreamingIterator is for borrowing values from self but you need them borrowed from some other thing, the Peekable / MultiPeek in this case. One approach is fst::Streamer but they can be a bit unwieldy to pass around. The link has a great explanation of what limitations this is working around.

extern crate itertools;
extern crate fst;

use fst::{Streamer, IntoStreamer};
use itertools::structs::MultiPeek;
use std::fmt::Display;

pub struct PeekFuture<'a, I>
    where I: Iterator + 'a
{
    iter: &'a mut MultiPeek<I>,
}

impl<'a, 'f, I> Streamer<'a> for PeekFuture<'f, I>
    where I: Iterator + 'a
{
    type Item = &'a I::Item;
    fn next(&'a mut self) -> Option<Self::Item> {
        self.iter.peek()
    }
}

fn main() {
    let v = vec![1, 2, 3];
    let mut peek = itertools::multipeek(v);

    // cur=1, peek=2,3,
    // cur=2, peek=3,
    // cur=3, peek=
    while let Some(cur) = Iterator::next(&mut peek) {
        print!("cur={}, peek=", cur);
        print_stream(PeekFuture { iter: &mut peek });
        println!();
    }
}

fn print_stream<'f, T, I, S>(s: I)
    where T: Display,
          I: for<'a> IntoStreamer<'a, Into=S, Item=&'a T>,
          S: 'f + for<'a> Streamer<'a, Item=&'a T>
{
    let mut iter = s.into_stream();
    while let Some(elem) = iter.next() {
        print!("{},", elem);
    }
}

#5

Thanks!