Take_while() gives double reference

Hello,

I am wondering while iter().take_while(..) gives double references.

Example code:

fn main() {
    let items = vec![None, None, Some(1), None, Some(2), None];
    println!("{}", items.iter().take_while(|&x| x == None).count());
}

Compiler output:

error[E0277]: can't compare `&Option<{integer}>` with `Option<_>`

It can be fixed with |&&x| x == None, |&x| *x == None or |x| **x == None, but that looks rather excessive and I wonder where the double ref comes from..

iter gives you a borrowing iterator yielding &T items. take_while (and filter) doesn't take ownership of its argument otherwise iteration couldn't continue since its argument would be dropped.

2 Likes

.iter() gives you an Iterator of references to the element of the Vec (this is the first layer). Then take_while (which is generic for any item type!) gives you a reference to the item in order to still be able to yield it (this is the second layer).

(In this particular program, note that you can also write the closure as |x| x.is_none(), which can go through any number of references. The == operator in particular can be somewhat finicky when one side has more layers of references than the other.)

Rust apparently only has a trait Iterator. Would it make sense to have something like RefIterator where the iterated items are references?

Then the implementation of TakeWhile could be written as something like this:

#[stable(feature = "rust1", since = "1.0.0")]
impl<I: RefIterator, P> RefIterator for TakeWhile<I, P>
where
    P: FnMut(I::RefItem) -> bool,
{
    type RefItem = I::RefItem;

    #[inline]
    fn next(&mut self) -> Option<I::Item> {
        if self.flag {
            None
        } else {
            let x = self.iter.next()?;  // x is a reference!
            if (self.predicate)(x) {  // no need to add a reference
                Some(x)  // yield the reference
            } else {
                self.flag = true;
                None
            }
        }
    }
    ...
}

Or alternatively provide different implementations for TakeWhile by differentiating over I::Item. However I am not sure if that is possible in Rust today.

Why do that when you can just do:

list.iter().take_while(|&&item| ...)

(Neither of your proposals are possible)

1 Like

And which trait would you use then? You can't use both at the same time, or you would have conflicts.

It would be possible with specialization, but it would be a nightmare in generic contexts. For example if you have a impl Iterator<Item = T> with a generic T and you don't know the type of T then you also don't know the type of the argument of the closure given to take_while. Would it be &T (like it does right now) or just T (if it was a reference)? IMO this is downside much bigger than simply having to add a & pattern.

AFAIK C++ choose the way where you can only have a single layer of references, and this creates all sorts of problems in templates. I don't think we want to do the same.

4 Likes

I am coming from C++ and while I agree that it is a nightmare, I am not sure where "a single layer of references" is creating problems in templates. In C++ one could write a small template to specialize on the return type of self.iter.next() and then call either self.predicate(x) or self.predicate(&x) as needed. I am sure though that there are always tradeoffs in language design and trying to get one part of the language watertight invites troubles elsewhere.

The additional & or * or - why not both! - in items.iter().take_while(|&x| *x == foo).count() doesn't look like a big deal - "just a character more", but it comes as a surprise hick-up. It looks a bit as if the API of take_while/filter is influenced by its implementation, and not by what makes sense from a design or usage perspective. Probably I am still missing a piece here, and don't see yet why the current choice is important.