HKTB: Passing iterators through functions

[daxpedda]

Hi !

This is a strongly simplified example. I used transforming_fun and resulting_fun to give a bit of a picture as to what I'm trying to do.

I believe the issue here lies at Item = &'a (T, &'a T), which can't work because this tuple has a shorter lifetime then &'a T, I have no clue how to assign a separate lifetime to the tuple and to &T.

I doesn't need to be taken by reference, but as I understand HKTB, the lifetime has to be issued at the "root" type, binding a lifetime only to an associated type isn't allowed.

This is a general problem I encountered many times before: how to pass down transformed iterators that hold references through functions. I always found a solution, but this time, the requirement extended to having the same generic with no lifetime passed through, which requires HKTB and comes with limitations I have no experience working around.

use std::fmt::Debug;

pub fn fun1<T: Debug, I>(par1: T, par2: I)
where
    for<'a> &'a I: IntoIterator<Item = &'a T>,
{
    fun2(par1, par2.into_iter().map(|x| (transforming_fun(x), x)))
}

fn fun2<T: Debug, I>(par1: T, par2: I)
where
    for<'a> &'a I: IntoIterator<Item = &'a (T, &'a T)>,
{
    for (x, y) in par2.into_iter() {
        resulting_fun(par1, x, y)
    }
}

fn transforming_fun<'a, T>(par: &'a T) -> T {
    unimplemented!()
}

fn resulting_fun<'a, T>(par1: T, par2: &'a T, par3: &'a T) {
    unimplemented!()
}

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
error[E0271]: type mismatch resolving `for<'a> <&'a I as IntoIterator>::Item == &'a (_, &'a _)`
  --> src/lib.rs:7:5
   |
3  | pub fn fun1<T: Debug, I>(par1: T, par2: I)
   |             - this type parameter
...
7  |     fun2(par1, par2.into_iter().map(|x| (transforming_fun(x), x)))
   |     ^^^^ expected tuple, found type parameter `T`
...
10 | fn fun2<T: Debug, I>(par1: T, par2: I)
   |    ---- required by a bound in this
11 | where
12 |     for<'a> &'a I: IntoIterator<Item = &'a (T, &'a T)>,
   |                                 --------------------- required by this bound in `fun2`
   |
   = note: expected reference `&(_, &_)`
              found reference `&T`

error[E0308]: mismatched types
 --> src/lib.rs:7:16
  |
3 | pub fn fun1<T: Debug, I>(par1: T, par2: I)
  |                       - this type parameter
...
7 |     fun2(par1, par2.into_iter().map(|x| (transforming_fun(x), x)))
  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected type parameter `I`, found struct `Map`
  |
  = note: expected type parameter `I`
                     found struct `Map<<&I as IntoIterator>::IntoIter, [closure@src/lib.rs:7:37: 7:65]>`

Some errors have detailed explanations: E0271, E0308.
For more information about an error, try `rustc --explain E0271`.
error: could not compile `playground` due to 2 previous errors

Thanks!

[steffahn]

Less simplification would probably help; in this playground it’s not clear what’s allowed to be changed and what isn’t.

A trait bound like for<'a> &'a I: IntoIterator<Item = &'a …> more-or-less says that I is the type of a collection with an fn iter(&self) -> impl Iterator<Item = &T> method. In fun1, you try to pass an iterator (created with .into_iter().map(…)) to a function with such a trait bound, it can’t work:

• In particular the for<'a> &'a I: IntoIterator<Item = &'a …> bound requires that you can iterate multiple times over the iterator; you cannot really iterate multiple times over an iterator created with .map

• In neither fun1 nor fun2 is the fact that you can create multiple iterators by borrowing the par2: I actually used.

In conclusion, it might be the best idea to simply move duty of borrowing par2 / creating an iterator with IIem = &'a T to the caller of fun1/fun2.

Another problem that needs to be solved is that the map iterator has owned tuples as items, and it’s not really possible to create an iterator of references to tuples out of that (except if you collect the iterator into e.g. a Vec first).

Here, this almost works

use std::fmt::Debug;

pub fn fun1<'a, T: 'a + Debug, I>(par1: T, par2: I)
where
    I: IntoIterator<Item = &'a T>,
{
    fun2(par1, par2.into_iter().map(|x| (transforming_fun(x), x)))
}

fn fun2<'a, T: 'a + Debug, I>(par1: T, par2: I)
where
    I: IntoIterator<Item = (T, &'a T)>,
{
    for (x, y) in par2.into_iter() {
        resulting_fun(par1, &x, y)
    }
}

fn transforming_fun<'a, T>(par: &'a T) -> T {
    unimplemented!()
}

fn resulting_fun<'a, T>(par1: T, par2: &'a T, par3: &'a T) {
    unimplemented!()
}

the only remaining problem is that you use par1 by value inside of a loop in fun2.

---

Of course, any of these suggestions might be totally useless because your actual, unsimplified code might contain some reason for why you need that HRTB with for<'a> &'a I: IntoIterator<Item = &'a …>. If that’s the case, you’ll need to give more details / avoid oversimplifying things.

[daxpedda]

Thank you, that actually helped, I didn't even try to just apply the lifetimes to the other parameters, so not using HKTB worked out just fine here.

To clarify, the requirements where having I be an iterator over references of T, modify it in fun1 and pass it down to fun2. Everything else has to stay the same. The problem as you said was that you can't iterate multiple times over an iterator created with map(), which wasn't my intention, it was just a constraint I had by HKTB.

I hope this helped clarification. But I realize that this example was bad, I will try to keep that in mind next time! When I'm done with what I was trying to do here I will post a link to it to give a better picture for people encountering this thread.