Using flat_map around a Result

help
1

I have some working code that's used to postprocess results from an SQL select. The general idea is that each SQL row contains zero or more UUIDs of interest, and I want to collect up all of those in one flat Vec. So this works:

fn from_row(row: Result<Row, mysql::Error>) -> Result<Vec<Self>, Error> { ... }

//  Do the select and collect up the results.
let result: Result<Vec<Vec::<UuidUsage>>, _> = tx
    .exec_iter(SELECT_UUIDS_SQL, params)?
    .map(UuidUsage::from_row).into_iter()
    .collect();
Ok(result?.into_iter().flatten().collect())

Note that this uses the map feature which can turn a Vec of Result into a Result of Vec. I'm not clear on how that really works. It does work here.

This has two .collect() calls, so it's building up all the data twice. Is it possible to use flat_map to flatten the results in one step? Tried this:

    let result2: Result<Vec::<UuidUsage>, _> = tx
    .exec_iter(SELECT_UUIDS_SQL, params)?
    .flat_map(UuidUsage::from_row).into_iter()
    .collect();

but there, the Vec of Result into a Result of Vec doesn't work. Is that something flat_map doesn't do?

error[E0277]: a value of type `Result<Vec<UuidUsage>, _>` cannot be built from an iterator over elements of type `Vec<UuidUsage>`
  --> src/common/tileassetsgc.rs:80:10
   |
80 |         .collect();
   |          ^^^^^^^ value of type `Result<Vec<UuidUsage>, _>` cannot be built from `std::iter::Iterator<Item=Vec<UuidUsage>>`
   |
help: the trait `FromIterator<Vec<UuidUsage>>` is not implemented for `Result<Vec<UuidUsage>, _>`
      but trait `FromIterator<Result<UuidUsage, _>>` is implemented for it
  --> /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/result.rs:2111:1
   = help: for that trait implementation, expected `Result<UuidUsage, _>`, found `Vec<UuidUsage>`
note: the method call chain might not have had the expected associated types
  --> src/common/tileassetsgc.rs:79:10
   |
77 |           let result2: Result<Vec::<UuidUsage>, _> = tx
   |  ____________________________________________________-
78 | |         .exec_iter(SELECT_UUIDS_SQL, params)?
   | |_____________________________________________- this expression has type `QueryResult<'_, '_, '_, Binary>`
79 |           .flat_map(UuidUsage::from_row).into_iter()
   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ----------- `Iterator::Item` remains `Vec<UuidUsage>` here
   |            |
   |            `Iterator::Item` is `Vec<UuidUsage>` here
note: required by a bound in `collect`
  --> /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/iter/traits/iterator.rs:2015:5

Relevant previous help topic: [Solved] Flat_map and Result issue where answer is "give it up and use a FOR loop". I'm probably getting too cute with functional features here.

2

It's collect that can collect an iterator of Result<T, E> into a Result<Vec<T>, E>, not map doing magic. I take it that UuidUsage::from_row looks something like

fn from_row(r: Result<Row, Err>) -> Result<Vec<UuidUsage>, Err> {

turning your items into Result<Vec<_>, Err>.

Then when you call flat_map, this turns the item type into Vec<UuidUsage>. Each Result either turns into one Vec<_> or no Vec<_> and any errors get discarded, because that's what Result's IntoIter implementation does -- creates an iterator over the Ok variant, if there is one, and discards the error.

You can probably do something like:

    let mut results = vec![];
    for vec in tx.exec_iter(SELECT_UUIDS_SQL, params)?.map(UuidUsage::from_row) {
        results.extend(vec?);
    }
    Ok(results)
2 Likes
3

If you're dealing in iterator-of-results, this is a great page to keep handy:

It lists out a bunch of "staff solutions" for various needs you might have.

4 Likes
4

I'm not familiar with whatever SQL library you're using. exec_iter() apparently returns a Result<T, E> where T implements at least map() and flat_map(). Is T here a std:iter::Iterator (In which case, why are you calling into_iter() on it?), or is it a custom type specific to the SQL library which happens to have iterator-like methods?

Assuming that T is a standard Iterator, the problem with using flat_map() is that the return value of the closure is flattened by means of its IntoIterator::into_iter() method, which for Result<T, E> returns an Iterator<Item=T> of one T value or nothing, so after calling flat_map() you have an iterator of Vec<UuidUsage>s (the Errors having been thrown away), which cannot be collected into a Result<Vec<_>, _>.

As for your original problem, I don't see a way to avoid the double collect() other than a manual for loop.

1 Like
5

I think passing Result as a parameter is a source of trouble. In my opinion, it is preferable that errors are handled where they occur. If this was done to make it more comfortable to map mysql::Error into the module-specific Error type, replace this with a From trait:

impl From<mysql::Error> for Error { ... }

(But I think you have already implemented this trait?)

Then rewrite from_row to:

fn from_row(row: Row) -> Result<Vec<Self>, Error> { ... }

Whenever there are multiple possible failure locations, explicit code like @quinedot has shown gets pretty simple:

    let mut results = Vec::new();
    for row in tx.exec_iter(SELECT_UUIDS_SQL, params)? {
        results.extend(UuidUsage::from_row(row?)?);
    }
    Ok(results)
6

Itertools::flatten_ok could be useful here.

I think you would want .map(UuidUsage::from_row).flatten_ok().collect().

2 Likes