Thoughts about `FallibleIterator`?

I am working on a project, which I was hopping to structure entirely based as combined/chained iterators. However it turned out that one of the most-inner iterators is doing IO, so I have to deal with Result. I started with Iterator<Item=Result<T>>, but I was just hitting problems one by one. It's just not what I want. Eventually it occurred to me that what I really would want is something like an iterator that explicitly handles failure. I thought I should name it "failable iterator". I decided to do a quick search an indeed someone else (sfackler & contributors) must have had the same idea

So I wonder what do other people think about it. I haven't used it yet, but it seems very necessary for handling any IO in a iterator-based API. Does anyone have better approaches? Would this be something that we could RFC and try to add to standard library, or at least had a chance to become a well recognized and supported primitive?

1 Like

From a quick glance:

  • it requires a lot of new types (although they mimic the old ones)
  • it requires reimplementing a lot of existing logic (which has the potential to introduce bugs or inconsistencies)
  • there are design problem such as whether closures should return Result or not
    • and if they return Result then the error should probably not be Self::Error
  • it clashes with existing functionality, like FromIterator<Result<T, E>> for Result<impl FromIterator<T>, E>
  • it makes things easier to write, but it's not strictly necessary to work with Iterator<Item = Result<T>>

I think that an approach adding additional methods to Iterator types that handle Results would make more sense as it makes inter-op easier with libraries that aren't compatible with fallible_iterator (such as Rust standard library). For instance, itertools crate crate adds additional methods that handle results such as map_ok.

1 Like

While I agree with the downsides ...

note that there's a big difference in meaning
between iterator returning Option<Result<Item>> and something that returns Result<Option<Item>>. Iterating over something IO-sourced is not just a sequence of items or errors. It's a sequence of items, that can fail altogether.

Would be great of the normal Iterator unified with that FailableIterator better, but I don't think this is going to work now. Iterator would have to be defined as a special case of FailableIterator that can not fail or something.

My current plan RN is just go take the downsides, and invest into building pieces that I'd need to use FailableIterator.

That can't be expressed with Result<Option<Item>> either though. You're still able to call next, and get an Ok(Some(_)) or Ok(None).

True, but that's not unlike most failable IO operations, where one could make a call again.

I agree that Iterator is not the right trait for the pattern you're describing, and that (&mut self) -> Result<Option<T>, Error>, as opposed to ... -> Option<Result<T, Error>>, is the right signature. I don't know whether it's worth pulling in a crate just to have a different trait with a more appropriate contract to implement. My experience with this kind of thing is implementing an incremental parser, where I had lots of types like PageElements that acted like fallible iterators ("parse the next Element, returning Ok(None) if there are no more elements or Err(_) on a parse error"). I ended up going with a normal method fn parse_next(&mut self) -> Result<Option<T>, Error> and chose not to use the fallible_iterator crate.

By the way, it's worth noting that if your problem is getting Option<Result> rather than Result<Option>, Option has a method called transpose which converts Option<Result<T, E>> into Result<Option<T>, E>.

1 Like

@sfackler Hi! Sorry to bother you, but do you have any plans/thoughts for that crate?

I think that ideally this use case should be handled with generators, see: [Pre-RFC]: Generator integration with for loops - language design - Rust Internals

1 Like

Oh, I see. Indeed, that makes sense. Just as FallibleGenerator seems like a generalization of Iterator a generator seems even more general.

Any idea, what's a realistic time-frame for having it on stable?

I haven't had the time/interest to focus on fallible-iterator for a while - I'd be happy to transfer it over to new owners though!

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.