Handling errors inside flat_map iterator?

help
#1

Hi, I'm trying to convert the following piece of code that is using panic! into something with more robust error handling where I'd like to use .context("...") from anyhow. Do you know of a nice way to rewrite this?

    let absolute_paths: HashSet<PathBuf> = globs
        .iter()
        // join expanded globs for each pattern
        .flat_map(|pattern| {
            glob(pattern)
                .unwrap_or_else(|_| panic!(format!("Failed to read glob pattern {}", pattern)))
        })
        // filter out errors
        .filter_map(|x| x.ok())
        // canonical form of paths
        .map(|path| {
            path.canonicalize()
                .unwrap_or_else(|_| panic!(format!("Error in canonicalize of {:?}", path)))
        })
        // collect into a set of unique values
        .collect();
#2

I would go for something like this:

let absolute_paths = globs
    .iter()
    .flat_map(|pattern| glob(pattern))
    .map(|path_result| {
        path_result.map(|path| path.canonicalize())
    })
    .collect::<Result<HashSet<PathBuf>, _>>()?;

To use .context("..."), you can call it on glob(pattern) or path.canonicalize().

1 Like
#3

It seems I'm getting a compilation error because glob(pattern) gives a Result, not handled by canonicalize()

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=37796b00f360fc3d026688a60a5be500

#4

Please note that I have a nested map in the canonicalize part to handle that.

#5

Yes, I thought it would work to but apparently it doesn't (cf play.rust-lang link)

#6

The issues seems to be related with the flat_map not flattening the Result<Paths>

#7

Yes. It is because glob itself returns a Result, so for the flat_map to do the right thing, we need to convert our

Result<impl Iterator<Item = Result<PathBuf, GlobError>>, PatternError>

into an

impl Iterator<Item = Result<PathBuf, anyhow::Error>>
#8

This compiles:

use glob::glob;
use std::collections::HashSet;
use std::path::PathBuf;
use anyhow::Context;

use itertools::Either;

fn get_files_panic(globs: &[&str]) -> anyhow::Result<HashSet<PathBuf>> {
    let absolute_paths = globs
        .iter()
        .flat_map(|pattern| match glob(pattern) {
            Ok(paths) => Either::Left(paths.map(|path| path.context("..."))),
            Err(err) => Either::Right(std::iter::once(Err(err.into()))),
        })
        .map(|path_result| {
            path_result.and_then(|path| path.canonicalize().context("..."))
        })
        .collect::<Result<HashSet<PathBuf>, _>>()?;
    Ok(absolute_paths)
}

I don't know if there's a better way to handle the glob thing.

1 Like
#9

Thanks, that's better that I could do since I didn't manage to get it compiling ^^. I'll see if I find another way avoiding the itertools dep, otherwise I'll do like that. Thanks again!

#10

Of course, one way is to reimplement Either. It is a quite simple utility.

#11

I managed to get something working based on scan. Found the idea from this blog post: Patterns of fallible iteration | More Stina Blog!

Thanks again @alice, I'll keep both solutions in mind for future similar situations.

    let mut glob_pattern_err = Ok(());
    let absolute_paths: HashSet<PathBuf> = globs
        .iter()
        .map(|pattern| (pattern, glob(pattern)))
        // Stop at first glob error
        .scan(&mut glob_pattern_err, |err, (p, gp)| {
            gp.map_err(|e| **err = Err(e).context("...")).ok()
        })
        .flatten()
        .map(|x| x.map_err(anyhow::Error::from)) // Just a type conversion trick for the compiler
        .map(|path_result| path_result.and_then(|path| path.canonicalize().context("...")))
        .collect::<Result<_, _>>()?;
    glob_pattern_err?;
#12

There's also a proposal to add something like this to the itertools crate.

1 Like