How to collect Vec<Result<Option<T>>> into Vec<T>?

[eee]

I know this works, but it feels so icky iterating twice. Is there a way to call flatten before collect? Rust Analyzer complains because flatten is trying to operate on Result type...

let all_results: Vec<String> = futures::future::join_all(responses)
        .await // Vec<Result<Option<String>>>
        .into_iter()
        .collect::<Result<Vec<_>, _>>()? // Vec<Option<String>>
        .into_iter().flatten().collect::<Vec<_>>();

[leob]

Why can't you just flatten twice?

fn flatten_twice<T, E>(input: Vec<Result<Option<T>, E>>) -> Vec<T> {
    input.into_iter().flatten().flatten().collect()
}

(Rust Playground)

[parasyte]

Because they want Result<Vec<_>> (notice the ? on the .collect::<Result<...>>).

I tried a few things, and the best I could come up with was this:

    let responses = vec![Ok(Some("<3 rust".to_owned()))];
    let all_results = responses
        .into_iter()
        .filter_map(Result::transpose)
        .collect::<Result<Vec<_>, _>>()?;

[leob]

Oh, ok. So the question title differs from the actual intent.

[parasyte]

It's an implementation detail. The code does in fact end up producing Vec<T> after unwrapping.

[eee]

just curious, why filter_map and not flat_map in this case? The differences between the two always get the better of me...

[kpreid]

filter_map can be more efficient than more general iterator flattening, because it knows that for each input, it needs to produce zero or one output, not more than one. Previous discussion:

Where is flatten() skipping None documented?

Yeah, it's a shame, but flatten has to deal with the possibility of an iterator returning multiple items, and thus it's just fundamentally more complicated than filter_map that deals in one item at a time. You can see it right in the type. FilterMap is just where F is a ZST for flattening options, whereas Flatten is because it needs to deal with the possibility of unconsumed front and back iterators.

[parasyte]

In addition to efficiency concerns, the function signatures for filter_map and flat_map differ; The filter_map closure/function returns Option<T>, allowing it to drop items by returning None. Whereas flat_map will happily call Iterator::flatten() on whatever its closure returns, and the effects of that are not always obvious.

Using filter_map(Result::transpose) works nicely in this particular case because it is filtering Result<Option<T>, E>, and the transformation to Option<Result<T, E>> will ensure that all None values are simply ignored while simultaneously unwrapping the Option.

This is very much a functional solution to the problem. And I am deeply aware that finding these solutions is rarely a simple task.

[H2CO3]

@parasyte

he filter_map closure/function returns Option<T>, allowing it to drop items by returning None. Whereas flat_map will happily call Iterator::flatten() on whatever its closure returns, and the effects of that are not always obvious.

What's the difference, though? It yields exactly the same behavior with Option, since its IntoIterator impl yields one element if it's Some, and zero if it's None.

I'm not sure what's "not obvious" in flat_map's effect. It does exactly what it promises – unfolding/concatenating one layer of inner iterators.

[parasyte]

I would argue that readability is a difference of concern. While it is clear to anyone familiar with traits that IntoIterator plays an important role with flat_map, to newcomers I suspect the language-level indirection can be a source of confusion.

"I'm returning something that impls IntoIterator, which means somewhere inside flat_map it is gaining access to something else that impls Iterator and then calls flatten on it."

vs.

"I'm returning Some to keep items or None to discard them."

[H2CO3]

While I agree that the semantically correct method to choose in this specific case is filter_map(), because it expresses the intent of filtering, I don't buy the complexity argument. It really just boils down how complex you want it to sound. It could be formulated to give exactly the opposite impression:

I'm returning an iterable to concatenate 0 or 1 items.

vs.

I'm returning an Option::Some or an Option::None that gets matched somewhere inside filter_map, which in turn somehow decides whether it should be yielded as the next item or not.

There is no real difference in the complexity from the user's perspective.

[parasyte]

That formulation feels even more stretched than what I provided. In one case you can tell what is happening by the signature alone, and in the other you have to click through at least two pages of documentation (or follow interfaces in your IDE or whatever) to discover the implementation details of IntoIterator for Option.

I'm sorry, I just don't buy how you've rephrased the problem statement.