Getting errors implementing `unzip` in an iterator adapter

When unzip is called on a custom iterator adapter, I want to print that it was called and then call the underlying iterator's unzip method. I keep getting errors, though. How can I make this work?

[Playground link]

#[derive(Clone, Debug)]
struct IteratorCallAnnouncer<T> {
    iter: T,
}

impl<T> IteratorCallAnnouncer<T> {
    fn new(iter: T) -> Self {
        Self { iter }
    }
}

impl<T> Iterator for IteratorCallAnnouncer<T>
where
    T: Iterator
{
    type Item = T::Item;
    
    fn next(&mut self) -> Option<Self::Item> {
        println!("iter.next()");
        self.iter.next()
    }
    
    fn unzip<A, B, FromA, FromB>(self) -> (FromA, FromB)
    where
        FromA: Default + Extend<A>,
        FromB: Default + Extend<B>,
        Self: Sized + Iterator<Item = (A, B)>,
    {
        println!("iter.unzip()");
        self.iter.unzip()
    }
}

This produces the error:

error[E0271]: type mismatch resolving `<T as Iterator>::Item == (_, _)`
  --> src/lib.rs:30:19
   |
30 |         self.iter.unzip()
   |                   ^^^^^ expected tuple, found associated type
   |
   = note:        expected tuple `(_, _)`
           found associated type `<T as Iterator>::Item`
   = help: consider constraining the associated type `<T as Iterator>::Item` to `(_, _)`
   = note: for more information, visit https://doc.rust-lang.org/book/ch19-03-advanced-traits.html
note: required by a bound in `unzip`

For more information about this error, try `rustc --explain E0271`.
error: could not compile `playground` due to previous error
1 Like

Self: Sized + Iterator<Item = (A, B)>, is incorrect.
It's supposed to be T: Sized + Iterator<Item = (A, B)>, or T::Item = (A, B)

fn unzip<A, B, FromA, FromB>(self) -> (FromA, FromB)
where
    FromA: Default + Extend<A>,
    FromB: Default + Extend<B>,
    Self::Item = (A, B)
{
    println!("iter.unzip()");
    self.iter.unzip()
}

error: equality constraints are not yet supported in `where` clauses
  --> src/main.rs:27:5
   |
27 |     Self::Item = (A, B)
   |     ^^^^^^^^^^^^^^^^^^^ not supported
   |
   = note: see issue #20041 <https://github.com/rust-lang/rust/issues/20041> for more information

Hmm, it doesn't work for now.

A workaround is to define a inherent method:

impl<T> IteratorCallAnnouncer<T> {
    fn unzip<A, B, FromA, FromB>(self) -> (FromA, FromB)
    where
        FromA: Default + Extend<A>,
        FromB: Default + Extend<B>,
        T: Iterator<Item = (A, B)>
    {
        println!("iter.unzip()");
        self.iter.unzip()
    }
}
2 Likes

I tried that, and it worked, but it has the drawback of not being an Iterator method, which means that if it's used in a context where it's considered to be an iterator of an unknown type, it will call the default iterator .unzip() method rather than the one that prints the call.

Your comment saying that Self::Item = (A, B) was needed led me to transmuting it:

impl<T> Iterator for IteratorCallAnnouncer<T>
where
    T: Iterator,
{
    type Item = T::Item;

    fn next(&mut self) -> Option<Self::Item> {
        println!("iter.next()");
        self.iter.next()
    }

    fn unzip<A, B, FromA, FromB>(mut self) -> (FromA, FromB)
    where
        FromA: Default + Extend<A>,
        FromB: Default + Extend<B>,
        Self: Sized + Iterator<Item = (A, B)>,
    {
        println!("iter.unzip()");
        unsafe { recover_item_type(&mut self.iter) }.unzip()
    }
}

#[inline(always)]
unsafe fn recover_item_type<I, ItemType>(iter: &mut I) -> &mut dyn Iterator<Item = ItemType>
where
    I: Iterator,
{
    core::mem::transmute(iter as &mut dyn Iterator<Item = <I as Iterator>::Item>)
}

Thanks for your help!

1 Like

Ew. It seems like a truly terrifying idea to use unsafe for circumventing the type system. Consider the following safe solution instead:

    fn unzip<A, B, FromA, FromB>(mut self) -> (FromA, FromB)
    where
        FromA: Default + Extend<A>,
        FromB: Default + Extend<B>,
        Self: Sized + Iterator<Item = (A, B)>,
    {
        println!("iter.unzip()");
        let (mut a, mut b) = <(FromA, FromB)>::default();
        
        while let Some((i, j)) = self.next() {
            a.extend(Some(i));
            b.extend(Some(j));
        }
        
        (a, b)
    }
}

Edit: see @vague's superior solution below.

3 Likes

Cool! By mocking the implementation of default unzip[1], it's rewritten as

let mut tuple = <(FromA, FromB)>::default();
tuple.extend(self);
tuple

  1. tuples are implemented with Extend ↩ī¸Ž

4 Likes

Neat! I didn't come across the new <(T, U)>::extend() impl; it was apparently only added in 1.56.0.

I even didn't notice it's newly added :laughing:
I'm just learning your tricks and check what std does :handshake:

1 Like

Unfortunately, I can't use that for the original use case because I was trying to print out only all external calls to the outer iterator without printing out the calls to the inner iterator, and this solution will repeatedly call .next() on the outer iterator, announcing that .next() was called repeatedly even though the only external call to the outer iterator was .unzip().

It can also produce a different result if the inner iterator has a custom .unzip() implementation that differs from repeatedly calling .next() for some reason (for example, it might know that it's an infinite iterator and panic or loop { std::thread::park(); }).

You can fix that by storing a flag in the adapter and setting it to false in methods that use self directly instead of immediately forwarding to the wrapped iterator, then only printing in noisy methods when that flag is true.

But calling unzip on an infinite iterator is wrong in the first place; if you keep adding to a container without bounds, it will crash your program in some manner in any case, so it's divergence either way.

1 Like

A version without transmute would would be

let di: *mut (dyn Iterator<Item = I> + 't) = &mut self.iter;
let di: *mut (dyn Iterator<Item = (A, B)> + 't) = di as _;
let di: &mut (dyn Iterator<Item = (A, B)> + 't) = unsafe { &mut *di };
di.unzip()

However, note that this still doesn't actually do what you want, no matter how it's implemented. You're not calling T's unzip implementation here. You're calling &mut dyn Iterator<...>'s unzip implementation... which uses the default method body. (And Box's implementation is similar.)

You can see this if you nest your wrapper. Swap the commented portion of main in to see your unzip avoided altogether (via the same &mut _ implementation mentioned above).

I don't think there's a way to make rustc see that the bounds are satisfied for the inner type within the context of the unzip method (which is what you need in order to do what you want). At least, I've failed to make any headway. I filed #102219.

2 Likes

I'm testing a few ways of passing through method calls to the underlying iterator more frequently, including with &mut I, and I'm using this iterator adapter to check that it works properly.

Sure. Going through &mut I will get to the underlying iterator's implementation (due to the sane implementation for &mut _). But perhaps not to the same methods on underlying iterator's implementation.

Yeah, I'm experimenting with having a trait that allows side-effect-free iterators to be cloned, but with internal state that is equivalent to if .next() was called until it produced exactly one None value.

Then, I can use the &mut I with core::mem::replace to swap the referenced iterator with the apparently-consumed version, receiving ownership of the original iterator, and allowing direct calls to things like .count() that may be far more optimized than the default implementation.

The trait would specialize the default Iterator implementation of &mut I. This call-announcing iterator adapter is useful in debugging things.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.