Std::iter::successors fails with overflow error

Hi,

I ran into a problem with code that uses successors. I have boiled it down into a fairly short test case. Here is the code

fn main() {
    println!("test apply");
    let v = apply(std::iter::successors(Some(1), |prev| {
        Some(*prev + 1)
    })
    .inspect(|i| if i > &15 && i < &20 {println!("apply {:?}", i)})
    )
    .inspect(|s| if s.parse::<u8>().unwrap() > 15 && s.parse::<u8>().unwrap() < 20  {println!("main: {}", s)})
    .take(16)
    .collect::<Vec<_>>();
    println!("{:?}", v);
    
    println!("test apply_overflow");
    let v = apply_overflow(std::iter::successors(Some(1), |prev| {
        Some(*prev + 1)
    })
    .inspect(|i| if i > &15 && i < &20 {println!("apply_overflow {:?}", i)})
    )
    .inspect(|s| if s.parse::<u8>().unwrap() > 15 && s.parse::<u8>().unwrap() < 20 {println!("main: {}", s)})
    .take(16)
    .collect::<Vec<_>>();
    println!("{:?}", v);
}

pub fn apply<I>(iter: I) -> impl Iterator<Item = String> 
where
    I: Iterator<Item = u8> + std::fmt::Debug,
{
    iter.map(|n| n.to_string())
}

pub fn apply_overflow<I>(iter: I) -> impl Iterator<Item = String> 
where
    I: Iterator<Item = u8> + std::fmt::Debug,
{
    iter.map(|n| n.to_string()).collect::<Vec<String>>().into_iter()
}


Here is the output from the run:

test apply
apply 16
main: 16
["1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16"]
test apply_overflow
apply_overflow 16
apply_overflow 17
apply_overflow 18
apply_overflow 19
thread 'main' panicked at 'attempt to add with overflow', src\main.rs:16:14

In the overflow situation, successors loops without ever moving on to the next adapter.

Is the overflow situation working as intended?

Thanks

Looks like you're trying to format your code (thanks) but haven't seen the instructions for that.

Thanks. I could see my attempt was failing. I have it now.

In apply_overflow you're trying to eagerly collect the entire iterator, which is infinite, into a vector. It only stops because u8 overflows quickly – had you used u32, it would probably exhausted your available RAM before panicking! Because you're eagerly collecting here, it does not matter that afterwards, once the collection is done (that is, if it did get done which it does not), the new Vector iterator is restricted to 16 items. Iterators are lazy, but not that lazy. collect is eager, it could not possibly materialize the iterator into a vector or anything else otherwise.

iter.map(|n| n.to_string()).collect::<Vec<String>>()
1 Like

I believe the error you're getting is correct, if that's what you're asking.

apply_overflow collects the infinite sequence you're giving it into a Vec, which of course overflows the u8 counter. This happens before it returns the iterator over the elements in the Vec.

1 Like

That's because you're calling .collect::<Vec<String>>() on it inside apply_overflow, which forces it to evaluate all the elements and store them in a Vec before continuing.

Thanks for the replies. It makes sense. I had originally been too focused on returning an
impl Iterator<Item=String>

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.