Idiomatic way to return/receive iterator?

Hi, I would like to write a function to do streaming process, so I hope its argument/return is iterator (instead of Vec) based, I'd like my code like this (pseudo code):

fn foo(Iteratable<i32>)->Iteratable<i32>{
}
fn bar(){
input = Vec<i32>
output0 = foo(input);
output1 = foo(output0);
output2 = foo(output1);
}

and my rust code:

fn foo <'a>(input: impl Iterator<Item = &'a i32>) -> impl Iterator<Item = i32>{
    let mut ret = Vec::new();
        for i in input{
        ret.push(*i);
    }
    ret.into_iter()
}

fn main(){

    let input0 = vec![0,1,2];
    let output0 = foo(input0.iter());
   //let output1 = foo(output0);  this is wrong, Iterator<Item = i32> is not Iterator<Item = &i32>
   // must do a temp convert
    let temp = output0.collect::<Vec<_>>();   
    let output1 = foo(temp.iter());

}

my question is how to makes this code simpler ? if I change foo to

fn foo <'a,'b>(input: impl Iterator<Item = &'a i32>) -> impl Iterator<Item = &'b i32>{
    let mut ret = Vec::new();
        for i in input{
        ret.push(*i);
    }
    ret.iter()
}

is incorrect because ret.iter() didn't take ownership. seems that I need something that takes ownership but iterator only yields reference.

SO suggest use AsRef so I changed my code as

fn foo <'a,I,J>(input: I) -> impl Iterator<Item = i32> 
where I: IntoIterator<Item =&'a J>,
      J: AsRef<i32> + 'a 
{

    let mut ret:Vec<i32> = Vec::new();
    for i in input.into_iter(){
        ret.push(*i);
    }

    ret.into_iter()
}

fn main(){

    let input0 = vec![1,2];
    let output0 = foo(input0);
    let output1 = foo(output0);

}

but rust throws

error[E0308]: mismatched types
 --> src/ask.rs:8:18
  |
8 |         ret.push(*i);
  |                  ^^ expected i32, found type parameter
  |
  = note: expected type `i32`
             found type `J`

error[E0271]: type mismatch resolving `<std::vec::Vec<{integer}> as std::iter::IntoIterator>::Item
== &_`
  --> src/ask.rs:17:19
   |
1  | / fn foo <'a,I,J>(input: I) -> impl Iterator<Item = i32>
2  | | where I: IntoIterator<Item =&'a J>,
3  | |       J: AsRef<i32> + 'a
4  | | {
...  |
11 | |     ret.into_iter()
12 | | }
   | |_- required by `ask::foo`
...
17 |       let output0 = foo(input0);
   |                     ^^^ expected integer, found reference
   |
   = note: expected type `{integer}`
              found type `&_`

error[E0271]: type mismatch resolving `<impl std::iter::Iterator as std::iter::IntoIterator>::Item
== &_`
  --> src/ask.rs:18:19
   |
1  | / fn foo <'a,I,J>(input: I) -> impl Iterator<Item = i32>
2  | | where I: IntoIterator<Item =&'a J>,
3  | |       J: AsRef<i32> + 'a
4  | | {
...  |
11 | |     ret.into_iter()
12 | | }
   | |_- required by `ask::foo`
...
18 |       let output1 = foo(output0);
   |                     ^^^ expected i32, found reference
   |
   = note: expected type `i32`
              found type `&_`

which makes me confused, maybe there is a util simply convert Iterator<Item=i32> to Iterator<Item=&i32> ? how to resolve such problem ? Thanks

Formatting your post properly will make it more likely for people to invest time into helping you. Use ``` before and after code segments.

I will definetelly not go through code fromatted like this, but answering the question in title: probably the most idiomatic way of taking/returning a vector is to go for impl Iterator<Item=...>, or depending on context impl IntoIterator<Item=...>. If the returned (or taken - but this is rare case, I don't remember I've ever faced it) is lifetime bound, then impl Iterator<Item=...> + '_, or explicitely: impl Iterator<Item=...> + 'a.

Thanks for suggestion, I missed the marked down support tip.

Maybe my question is not clear enough , I'd like to know a 'generic' way to pass in iterator no matter Item is value or reference, (so I can passing in Iterator<Item=i32> directly without intermediate collect().iter() convert), for example

fn is_hello<T: AsRef<str>>(s: T) {}

can accept both String and &str. my scenario is similar , but I need the nested type Iterator::Item can be AsRef. I'm not sure if this is doable in rust.

Then maybe:

fn foo(it: impl Iterator<Item=impl AsRef<T>>) -> impl Iterator<Item=impl AsRef<T>> {}

Or:

fn foo(it: impl Iterator<Item=impl Borrow<T>>) -> impl Iterator<Item=impl Borrow<T>> {}

I ommited namespaces for simplicity, check where they are exactly to use them. Remember, that if you go with them, you need to call .as_ref() or .borrow() on elements before processing them.

1 Like

Thanks @hashedone, I tried and confirmed that Borrow version worked. so my code looks like this:

fn foo(it: impl Iterator<Item = impl Borrow<i32>>) -> impl Iterator<Item = i32> {
    let mut ret = Vec::new();
    for i in it {
        ret.push(*i.borrow());
    }
    ret.into_iter()
}
fn bar() {
   let input: Vec<i32> = vec![1, 2, 3];
   let output0= foo1(input.iter());
   let output1 = foo1(output0);
   let output2 = foo1(output1);
}

but that raises me another three questions:

  1. Now I have to use borrow, does this slower runtime performance due to extra call? (or impl makes rust knows exactly type of parameter at compile time, so Borrow only has signature meaning, the borrow call is actually nothing? )

  2. Why AsRef version not work ? why Borrow is auto implemented by i32 but not AsRef ? does & operator just means borrow by reference ?

fn foo(it: impl Iterator<Item = impl AsRef<i32>>) -> impl Iterator<Item = i32> {
    let mut ret = Vec::new();
    for i in it {
        ret.push(*i.as_ref());
    }
    ret.into_iter()
}

fn bar() {
    let input: Vec<i32> = vec![1, 2, 3];
    let output0= foo(input.iter());
    let output1 = foo(output0);
}

rust throws:

error[E0277]: the trait bound `i32: std::convert::AsRef<i32>` is not satisfied
  --> src/ask.rs:17:18
   |
5  | fn foo(it: impl Iterator<Item = impl AsRef<i32>>) -> impl Iterator<Item = i32> {
   | ------------------------------------------------------------------------------ required by `
ask::foo`
...
17 |     let output0= foo(input.iter());
   |                  ^^^ the trait `std::convert::AsRef<i32>` is not implemented for `i32`
   |
   = note: required because of the requirements on the impl of `std::convert::AsRef<i32>` for `&i
32`

error[E0277]: the trait bound `i32: std::convert::AsRef<i32>` is not satisfied
  --> src/ask.rs:18:19
   |
5  | fn foo(it: impl Iterator<Item = impl AsRef<i32>>) -> impl Iterator<Item = i32> {
   | ------------------------------------------------------------------------------ required by `
ask::foo`
...
18 |     let output1 = foo(output0);
   |                   ^^^ the trait `std::convert::AsRef<i32>` is not implemented for `i32`


  1. use Iterator as return type seems makes input parameter borrowed longer than expected, for example, if I return Vec, following code has no problem
    fn foo2(it: impl Iterator<Item = impl Borrow<i32>>)->Vec<i32>{
        let mut v = Vec::new();
        v.push(30);
        v
    }

    fn bar2(){
        let input = vec![1,2,3];
        let output0 = foo2(input.iter());
        let output1 = foo2(input.into_iter());
        let output2 = foo2(output0.iter());
    }

but if I changed code as

    fn foo2(_: impl Iterator<Item = impl Borrow<i32>>)->impl Iterator<Item=i32>{
        let mut v = Vec::new();
        v.push(30);
        v.into_iter()
    }

    fn bar2(){
        let input = vec![1,2,3];
        let output0 = foo2(input.iter());
        let output1 = foo2(input.into_iter());
        let output2 = foo2(output0);
    }

to my surprise, rust throws

error[E0505]: cannot move out of `input` because it is borrowed
  --> src/ask.rs:42:28
   |
41 |         let output0 = foo2(input.iter());
   |                            ----- borrow of `input` occurs here
42 |         let output1 = foo2(input.into_iter());
   |                            ^^^^^ move out of `input` occurs here
43 |         let output2 = foo2(output0);
   |                            ------- borrow later used here


no matter vec/iterator returned, both takes ownership of a new allocated vec, I think that didn't borrow anything from input parameter, why return Iterator makes rust think input is still borrowed by output0 ?

Thanks

So going one by one:

  1. There is nothing in RT if you are using impl Trait. It has obvious upsides - type is known in CT, so if Borrow::borrow or AsRef::as_ref is trivial, it would be most likely inlined. Obviously if there is complex logic behind it, there would be additional call, but idiomatically both those traits should just return reference to existing object.
  2. Semantics. The borrow trait should be implemented if something may be borrowed as something else (it is defined for arrays to be borrowed as slices, String to be borrowed by str, Cow to be borrowed as internal type and some other similar cases). In general borrow should be isomorphic in respect to some key operations - hashing, and ordering in particular. It means "type of same semantic, but without ownership". The semantic of AsRef is loser - it is just defined if borrow of type may be converted to other type borrow. The difference are described in more details in AsRef documentation. However the reason why there is no blanket implementation like AsRef<T> for T is a mystery for me - I have some thoughts, but I am not very much sure about them.
  3. It looks like the return type lifetimes elided to be same for argument and return type, it is very strange for me (as long as return type looks to be 'static). It may be some kind of bug, but maybe someone smarter than me is able to explain this behavior. However looking at IntoIterator impls for Vec, there are three of them: one for owned (giving Iter<T> which is 'static and should cause no problems), and two for borrows (giving Iter<'a, T> and IterMut<a, T>, which are lifetime bound, but which also doesn't match your return type - they both are iterator over borrows, instead of iterator over items). I would like to know what happened here.

Fun fact about (3):

    use std::borrow::Borrow;
    
    fn foo2(_: impl Iterator<Item = impl Borrow<i32>>) -> std::vec::IntoIter<i32> {
        let mut v = Vec::new();
        v.push(30);
        v.into_iter()
    }

    fn bar2(){
        let input = vec![1,2,3];
        let output0 = foo2(input.iter());
        let output1 = foo2(input.into_iter());
        let output2 = foo2(output0);
    }

Works, so it turns out, that v.into_iter() returns somethin else than std::vec::IntoIter.

1 Like

Wow, thanks @hashedone for the detailed explanation!

for my problem 3, I may open another topic for help.