I am writing a tokenizer. This looks a lot like an Iterator, at one exception: in case the tokenizer fails to parse the input, I would like to give more context than what None can provide. I could return an Option<Result<TokenType>>, but this would be redundant since it would always return Some(...). Is there anything that look like an iterator but where the error case is a type that implements Error instead of being None?
You might be interested in fallible_iterator::FallibleIterator - Rust
Absolutely perfect!
Is there any reason why this is not in std?
That trait uses Result<Option<T>, E> as type, but this makes it equivalent to an ordinary iterator with Result and Option transposed for convenience. You might as well use an ordinary iterator.