Handling errors in FromIterator::from_iter


#1

Hi everyone,

This is my first post to the community. I have been playing with rust for the last couple of weeks and so far I’m enjoying the language very much.

I have a question regarding FromIterator, namely how errors in from_iter should be handled. The trait demands that from_iter returns a Self, that is, a fully constructed object. But it seems overly optimistic, or overly restrictive, to assume/expect that such a construction will always work. I would expect from_iter to return an Option or a Result.

To sum up: what’s the recommended way to handle a failed from_iter conversion?

Thank you very much.


#2

In general, the FromIterator conversion it self should never fail (unless there’s a bug in which case it should just panic). That is, it’s not designed for fallible conversions. What’s your precise use case?


#3

I am on my cell phone right now, I can provide a use case tomorrow from my computer. I am just exploring the language, nothing important.

But this got me thinking, nevertheless: if I copy a huge collection into a new huge collection through into_iter/from_iter, and I run out of memory so the copy cannot succeed, will I get any chance of recovery at all?


#4

Generally, no, out-of-memory conditions are not handled well. There’s a big thread on internals about this right now: https://internals.rust-lang.org/t/could-we-support-unwinding-from-oom-at-least-for-collections/3673


#5

Note that there is a impl<A, E, V> FromIterator<Result<A, E>> for Result<V, E> where V: FromIterator<A>. Meaning, you can collect an iterator of Result<T,E> into a Result<Collection<T>,E> (same thing for Option), for any Collection that supports FromIterator. The first error will be propagated, or if there are no errors, the collection is returned.


#6

Thanks for the pointer, @cuviper, that’s an interesting discussion. I hope we will get OOM handling eventually.


#7

Thanks, @jethrogb, this sounds really interesting. I had seen this piece of advice in other contexts but had not connected it to mine. I will give it some thought. I will post an example of what I am trying to do.


#8

I have a very basic struct with two fields (may be more) of the same type:

    #[derive(Clone, Copy)]
    struct F {
        x: f32,
        y: f32
    }

From a user’s point of view, a Struct is very convenient because it has meaningful tags for its elements: f.x or f.y rather than v[0] or t.0. Because all the elements have the same type, it would also be interesting to be able to treat F as an IntoIterator, and map arbitrary functions to all members of the Struct. This way I could implement algorithms that work on arrays or vectors for Structs (I’m thinking of numeric integration, Runge-Kutta, for example).

So I first implement an iterator for F:

    struct G {
        count: u32,
        f: F
    }

    impl G {
        fn new(f: F) -> G {
            G{count: 0, f: f}
        }
    }

    impl Iterator for G {
        type Item = f32;

        fn next(&mut self) -> Option<Self::Item> {
            self.count += 1;

            match self.count {
                1 => Some(self.f.x),
                2 => Some(self.f.y),
                _ => None
            }
        }
    }

Now implement Fromiter and IntoIter for F:

    impl FromIterator<f32> for F {
        fn from_iter<I>(iter: I) -> Self
            where I: IntoIterator<Item = f32> {

            let mut i = iter.into_iter();

            let x = i.next().unwrap();
            let y = i.next().unwrap();

            F{x: x, y: y}
        }
    }

    impl IntoIterator for F {
        type Item = f32;
        type IntoIter = G;

        fn into_iter(self) -> G {
            G::new(self)
        }
    }

Now this test passes:

    #[test]
    fn from_into_iter() {
        let f = F{x: 1.0, y: 2.0};

        let v: F = f.into_iter().collect();

        assert_eq!(f.x, v.x);
        assert_eq!(f.y, v.y);
    }

But this one does not and panics because of the unwrap on None:

    #[test]
    fn short_from_iter() {
        let f = vec![1.0];

        let v: F = f.into_iter().collect();

        assert_eq!(1.0, v.x);
        assert_eq!(0.0, v.y);
    }

Certainly, I can take care myself inside into_iter of the unwraps, but I am looking for a way to signal this to the user.

Thanks for your help.


#9

[quote=“elferdo, post:8, topic:6704”]```rust
#[test]
fn short_from_iter() {
let f = vec![1.0];

let v: F = f.into_iter().collect();

assert_eq!(1.0, v.x);
assert_eq!(0.0, v.y);

}

[/quote]

I tried to think a bit about the semantics of your `Iterator` implementation.

`FromIterator` consumes the iterator that you pass to it. As a result, each iterator can represent one, and only one `F`. It wouldn't make sense to e.g. "stream" several instances of `F` out of the same iterator.

That means to me that a user of `F` should always know the length of their iterator. In particular, the user should always make sure that their iterator contains enough elements to build an `F` from it.
Consequently, I think an error such as in your function `short_from_iter()` doesn't qualify as "recoverable", but as a "logic error" of the program instead.

Long story short, I think panicking is actually the most reasonable way to react to this particular error condition. (Personally, I'd even panic if the iterator is too long – but that's probably a matter of preference.)

#10

I feel like you should simply be able to implement FromIterator for Option or Result<F, MyError> instead, but on second though I guess this doesn’t work because you can’t implement foreign traits (ie. FromIterator) for foreign types (ie. Result).

A (slightly ugly) workaround would be to add a wrapper struct around Option. This would then look something like this:

struct MaybeF(Option<F>);
impl FromIterator<f32> for MaybeF {
  fn from_iter<I>(iter: I) -> Self where I: IntoIterator<Item = f32> {
    let mut i = iter.into_iter();

    let x = match i.next() { Some(v) => v, None => return MaybeF(None) };
    let y = match i.next() { Some(v) => v, None => return MaybeF(None) };
    
    MaybeF(Some(F{x: x, y: y}))
  }
}

[...]

let v: MaybeF = f.into_iter().collect();
assert!(v.0.is_some());
assert_eq!(1.0, v.0.unwrap().x);
[...]

Not a thing of beauty, but I guess it would work. :wink:


#11

The way, I would solve this problem, is by not using FromIterator at all.
Why not create a function or method that takes an Iterator and returns a Result or an Option?

The FromIterator trait should only be used for operations that must not fail.
In some way like the From trait. That makes it not the best choice for this problem.

Of course, this approach would not be usable through the .collect() method.

fn into_vec<I: Iterator>(i: I) -> Result<Vec<I::Item>, ()> {
    Ok(i.collect())
}

fn main() {
    let iterator = 0..5;
    let vec = into_vec(iterator).unwrap();
    println!("{:?}", vec);
}

For a method like .collect() a trait could be used:

trait CollectVecMaybe<T> {
    fn collect_maybe(self) -> Result<Vec<T>, ()>;
}

impl<T, I: Iterator<Item=T>> CollectVecMaybe<T> for I {
    fn collect_maybe(self) -> Result<Vec<T>, ()> {
        Ok(self.collect())
    }
}

fn main() {
    let iterator = 1..6;
    let vec = iterator.collect_maybe().unwrap();
    println!("{:?}", vec);
}

#12

Thank you guys for all the help and suggestions.

Final take home message: FromIterator must never fail. There are however the impls for Result and Option that I will definitely explore.

Since the original post was only about handling errors in FromIterator::from_iter I wouldn’t like drifting too off-topic. My goal is to build a library with numeric integration algorithms that operate on dynamical systems of arbitrary dimensions (i.e. on arrays or arbitrary sizes, though they will be fixed for each specific problem), while letting the user specify their systems with semantically relevant tags. Therefore my interest in adapting Structs to something iterable.

I will post on separate threads any questions regarding my implementation.

Thanks again!

Cheers,
Fernando