?-like early return on iterator error

I’m trying to design a simple API for iterating over values retrieved from a file. I’d like to point my users to std::Iterator and be done with it. Unfortunately (but correctly!), my iterator’s Item type is io::Result<u64>, so I need to deal with the possibility of an Err at each step. I don’t want the code to panic and I’d like to stop processing as soon as an Err is encountered.

What I’m after is something like this:

let answer: Result<u64, io::Error> = input
  .iter()
  .take(10)
  .sum();

In many contexts I can use the ? operator to short-circuit processing when an Err variant of Result is encountered, but there doesn’t seem to be an equivalent for working with Iterator.

I’m aware of filter_map(), but I don’t want to ignore failures.

I’m aware of partition(), but the input could be quite long. I don’t want to continue pulling from the iterator after the first Err is returned.

I’m aware of collect(), but users will only very rarely want to collect() the values they’re reading. It’s much more likely that they’ll want to process them as they go, avoiding allocation.

I’m aware of try_fold() and try_for_each(), but these won’t allow me to use the Iterator API – I’ll have to fully define the processing for each individual value within a lambda. At that rate, I may as well use a for loop instead of an iterator.

Is anyone aware of a way to short-circuit iterators?

Thanks!

This should just work, because Result implements Sum:
https://doc.rust-lang.org/std/result/enum.Result.html#impl-Sum<Result<U%2C%20E>>

6 Likes

Huh! Good eye, that’s an interesting observation. While sum() works, however, I believe the larger question about short-circuiting still stands.

let index: Result<Option<bool>, io::Error> = input
  .iter() // We should be able to end iteration if this returns Err
  .position(|id| id == 7); // This returns Option<bool>

I suspect this is not possible without a new language feature (or alternative iterator trait).

True, you can’t unwrap arbitrary Iterator methods this way. I’ll bet you can almost always approximate this with other methods, especially try_fold, but you’re right it won’t be as nice as the ? operator.

I just found @sfackler’s fallible-iterator crate, which appears to offer a version of the Iterator trait that does exactly this! I’ll have to try it out, curious to know whether there’s any performance overhead.

4 Likes

I want a try_collect. Is using the failing iterators really the best option? Or should I use try_fold? Is there any reason the Standard library does not provide failing collection?

Collecting to Result<C, E> or Option<C> should work fine for this.

2 Likes

:man_facepalming: I have asked this question in this forum already a few months ago…

You can use:

  • Iterator::flatten(), to treat an inner Err like a None, taking advantage of impl<T, E> IntoIterator for Result<T, E>
  • Iterator::fuse() to stop at errors

But this hides the error and does not propagate it

Itertools’ process_results may be what you want. In fact, its first version is what is in the standard library to implement sum for Result.

fn example(input: impl Iterator<Item = Result<u64, io::Error>>) -> Result<Option<u64>, io::Error> {
    itertools::process_results(input, |input| input.max())
}
5 Likes