Errors in Rust can now be handled more ergonomically, cleanly, and simply: introducing a new error crate

Quick link to the new crate

A few weeks ago, I created a post on the /r/rust subreddit about Rust's error system and its shortcomings compared to corresponding systems other languages, like Java. To summarize it, in Rust, the standard way to handle errors is to either use a crate like Anyhow to simply bubble up any errors to the caller (and hopefully, eventually, to some code that is in a position to handle it) via the ? operator, or to create a complex (compared to Java, see below) custom error type for each module or crate. This error type usually takes the form of an enum with a variant for each kind of error that can take place from that crate or module, or a thin wrapper around that.

The problem with these approaches is that the user of your functions likely doesn't know which errors, exactly, can happen from each function. With Anyhow, this problem is even worse since a function returning the Anyhow error type can return literally any error . This may be acceptable in some situations, but it clearly isn't ideal. When you create a custom error type, it sometimes isn't a trivial undertaking, which is likely why most crates don't have an error type for each function, which would clearly document how each function can fail. Compare this system of poorly-defined ways that a function can error to how it's done in well-written Java (Java does a lot of things poorly, but it does errors well) pseudo-code:

public Object readAndParseFile(Path file) throws IOException, ParseError{
    String contents = Files.read(file); //throws IOException
    Object parsedContents = parseString(contents); //throws ParseError
    return parsedContents;
}

In this example, the set of possible failure modes is clearly documented in the function's signature (RuntimeException can be used to bypass this issue in Java, but this is usually bad practice). Furthermore, it is standard for Javadoc comments to describe under which scenarios each exception is thrown.

In response to this post, many useful objections were raised, such as that treating errors as data has advantages (e.g. threading is easier, and in many parts of the code the error can be handled as data, such as with Vec<Result<…>>). From this, I created a new crate, polyerror, that combines the best of both worlds to create what I hope will make Rust code cleaner, simpler, and better documented.

Polyerror works by making it so trivial (literally a one-line macro call) to define an ergonomic (? works) and correct error type that it is practical to have a separate error type for each function. This way, it is always obvious to the end user in which way a function can error:

use std::str::ParseBoolError;
use std::num::ParseIntError;

create_error!(pub ParseThenCombineError: ParseBoolError, ParseIntError);
pub fn parse_then_combine(a: &str, b: &str) -> Result<String, ParseThenCombineError> {
    let parsed_bool: bool = a.parse()?;
    let parsed_int: i32 = b.parse()?;
    Ok(format!("{} {}", parsed_bool, parsed_int))
}

In this toy function, two errors are possible: ParseBoolError and ParseIntError (both from the standard library). With the traditional model, one would either use something like Anyhow and obscure useful (if not vital) information from the users of your crate, forcing them to delve into your code or hope that it's documented properly (this reminds me of how lifetimes are specified in C and C++), or simply add these two error types to a global error enum. Here, instead, the ParseThenCombineError (you're free to choose less verbose names if that's your style) is to be used only for the parse_then_combine function. Since it's defined with a single line of code before the function, this technique isn't any significant tedium or productivity drain.

To give a precise idea of what's going on, the above create_error! call expands to this (slightly reformatted) source code:

#[derive(Debug)]
pub enum ParseThenCombineError {
    ParseBoolError(ParseBoolError),
    ParseIntError(ParseIntError),
}
impl ::std::fmt::Display for ParseThenCombineError {
    fn fmt(&self, f: &mut ::std::fmt::Formatter<'_>) -> ::std::fmt::Result {
        write!(f, "{:?}", self)
    }
}
impl ::std::error::Error for ParseThenCombineError {}

impl ::std::convert::From<ParseBoolError> for ParseThenCombineError {
    fn from(error: ParseBoolError) -> Self {
        Self::ParseBoolError(error)
    }
}

impl ::std::convert::From<ParseIntError> for ParseThenCombineError {
    fn from(error: ParseIntError) -> Self {
        Self::ParseIntError(error)
    }
}

Just one macro is exported that provides all that you need for robust, easily understandable (for you and your users), and correct error handling in Rust. The generated error type is documented in Rustdoc pages just like any other type, so it's easy for users to quickly understand what's going on, even if they've never heard of this crate before — it is entirely transparent to a user of your API. With this crate, I hope to make Rust code safer, cleaner, simpler, and better documented.

13 Likes

Highly relevant RustConf talk: RustConf 2020 - Error handling Isn't All About Errors by Jane Lusby - YouTube

Obviously in the Java checked exceptions case, adding new checked exceptions is a (source) breaking change. But still, I'd recommend making #[non_exhaustive] an option.

Additionally, for a lightweight "just give me a bundle of error types" error type, I'd expect at least an option to have the source error reported by Error::source(), when upstream errors do implement Error. You should be able to use autoref specialization to implement Error::source to return the source iff it implements Error, since you're already in a macro.

Final note: implementing Display via Debug is almost certainly semantically wrong, especially if you do hook up Error::source. See Jane's video above for more context on why.

See also: cex — Rust library // Lib.rs (though I prefer this approach)

4 Likes

Great feedback, thanks! I'll make some issues on the issue tracker and get to those things when I have time (probably within the next week or two).

Indeed. But I would recommend against putting it in by default, as it can cause silent failures in match exprs when new error enum variants are added.

3 Likes

I disagree. Guarantees from the library author to the library user should always be explicit. Rust made this mistake with traits being ?Sized by default and enums being exhaustive by default. The library author shouldn't be able to trap themself by accident in a stability guarantee they never meant to express.

This causes extreme pain beyond 1.0.0, because revoking any such guarantee involves bumping the version to 2.0.0. Bumping major version numbers splits the crate ecosystem up into groups. Depending on different crates, that depend on the same crate, but incompatible versions leads to confusing errors (expected type A, found type A), binary bloat and increased building times. All of these are undesirable.

As a user, you see immediately, that the match is non-exhaustive and if you'd like for the author to guarantee no new error additions, then it's your task to create an issue or ask them in another way to add a guarantee to their library. If and only if they made this guarantee consciously, you can expect them to not have to eventually break stability by releasing 2.0.0 or even worse, several stability-breaking releases, because the author discovered yet another stability trap, too late.

I'd rather deal with _ => … (and trait is not object-safe) than with breaking releases.

3 Likes

I'm looking into the technique of autoref specialization, and I definitely could do that, but it's a fair bit of extra code complexity for what appears to me to be perhaps not necessary (handling types that don't implement std::error::Error). If a type is to represent an error, which is what my crate is designed to wrap, why would it not implement std::error::Error?

Regardless of whether enums are exhaustive or not, adding a variant to an error enum is a breaking change. The only question is whether the breakage is silent or not. Furthermore, I consider the fact that rustc tells you loudly that there's an unmatched enum variant in a match a feature rather than a bug.
Given that, I say "let's have rustc scream its heart out so that I can fix the issues". Hence, my opinion is no #[non_exhaustive] by default.

It's been behaving this way for about 5 years now, so it doesn't have very many surprises left at this point.
And in fact I'd go a bit further here and say that if you define a closed set of items, and then expand that closed set, that it isn't surprising in the least that that expansion causes breakage. The new items in the set do need to be handled after all. It'd be extremely surprising and annoying if that weren't the case, because then the compiler wouldn't be able to tell you what needs fixing, as it can now.
So if someone writes an error enum and is surprised, then they simply don't understand the concept very well, nor have they thought through the consequences of their choice of representation.
That's on them, same as I'd look at the carpenter (rather than at the hammer) for not knowing the difference between a hammer and a sledgehammer.

They can just as easily happen with a minor version bump as they can with a major one.
So in that sense there's no technological difference between the 2.
I agree it's not the standard behavior, but IMO the explicitness of the breakage is still worth it.
Perhaps cargo fix could even be updated to provide stubbed out match arms at the error locations.

As a user, I like the status quo: I like the flexibility of being able to make that choice myself each time I write a match expression rather than have that choice forced upon me one way or the other.

As I explained above, that's a safe assumption to make by virtue of their choice of error representation i.e. an enum.
Keep in mind that technically there's an alternative in the form of a trait-based approach. It's just much, much less ergonomic to use.

That's purely a matter of opinion.
You see, I happen to consider fixing a (bunch of) missing match arms much easier than having to tangle with headaches like object safety.

5 Likes

Adding a variant to an enum, that is non-exhaustive is never a breaking change.

1 Like

It seems error ergonomics is an ongoing research area for rust for years now. We will get there eventually. Kudos for proposing something that makes it explicit for each function what are the errors that can be returned. In general finer grained errors are preferable.

I just wanted to point out that anyhow is meant to be used by application code, not library code. It's purpose is specifically to easily aggregate error types. So supposedly the application developer can look at their own code to see what the possible errors are. That being said this method might still be more practical even in your own code.

What I have been doing is have an error type per crate and then specify in method documentation which variants can be returned in which circumstances. I suppose the difference is less type explosion, more manual guaranteeing that one documents everything.

One issue I see is that with your approach, there doesn't seem to be an easy way to document the errors. When you return a Result with a specific error type, it will be a clickable link in the docs, so people will check it out to find out how to interpret the error variants, which means they should be documented.

Then there is the question of how these errors are presented to the end user. In the above case, you will have a display that says something like "Failed to parse integer", which gives you absolutely no context about what is actually going wrong, which is bad UX. When having a custom error type with variants, you can give a more appropriate name and display for errors like VersionShouldBeInt if you are parsing a file that has a version number. It can have a source that is ParseIntError for back traces, but it has a more descriptive name and Display implementation.

I could see a sweet spot if something like this could be parsed:

create_error!
{
   /// The first parameter of [`parse_then_combine`] must be a valid `bool`. 
   /// If it cannot be parsed as a `bool`, this error is returned.
   // 
   ParseBoolError as SomeDescriptiveName,

   /// The second parameter of [`parse_then_combine`] must be a valid integer. 
   /// If it cannot be parsed as an integer, this error is returned.
   // 
   ParseIntError as VersionShouldBeInt,
}

And the doc comment could be the Display impl. By now we are well in proc macro land however.

So all that almost brings us close the functionality of thiserror. Note that thiserror also supports adding data to your variants. But then people prefer that not to be used in library crates because the compile time overhead of procedural macros, so that's why I ended up concluding for a manual error per crate and documenting the variants that are used by each part of the API.

1 Like

I suspect you're saying this because rustc doesn't complain.
However, there can still be dependant code out there out there that silently breaks when a new enum variant is added, because control flow is routed to the default arm in any match expressions.

Of course you can then say "that is the intended behavior" and indeed it is. Yet consuming code could still be written without thinking too hard about what would happen if a new enum variant was added. Hence, breakage.

1 Like

"Breakage" because the user assumes something that isn't true is their fault.

1 Like

I think, we're on the same page, now and are moving towards a constructive discussion about the subtle differences between what it means to break stability, i.e. what is formally guaranteed vs. what is practically expected. Indeed, as a user, you can also fall into traps, that causes both compile- and runtime errors upon updating your dependencies, because you didn't expect something to happen, even if the dependency author did nothing wrong (let's assume, they know how to not mess up semver versioning, which is a big topic on its own).

I do think, Rust would heavily benefit from having both a library and binary development guide on how to prevent undesirable breakages by following some simple rules and most importantly, actually throw these guides at us, instead of hiding them somewhere in a dark corner. I expect Rust to have some kind of document/thread/issue, that establishes formal rules on what is considered a breaking change (they must use something for the standard library!?) and what isn't, yet I couldn't find it.

A few pitfalls for users:

  • Using * in use statements, except for designated preludes and exhaustive enums
  • Using extension traits for external (in relation to the trait) types and running into ambiguous method calls (this is by far the most brittle breaking change, that can occur, from my experience)

A few pitfalls for library authors:

  • Add new items to preludes
  • Too liberal dependency version requirements
  • Introducing cutting-edge standard library features (hopefully, we'll get MSRV – minimum supported rustc version – rather sooner than later)

However, at the end of the day, the one who adheres to the rules is the one in the right and in this particular case, adding a variant to a non-exhaustive enum is considered a non-breaking change and that's all we can count on. Nonetheless, it'd be a good idea for the author to document this, so that users don't just do naive stuff in the _-match arm, like using unreachable!(). Documentation is always beneficial to a library and I never noticed anyone complaining about too much documentation for some crate, on the forum.

I agree with almost all of what you said there.

To my original point (which got this entire sub-thread started), because #[non_exhaustive] can cause silent breakage and/or altered behavior, I simply recommended that that be possible to use, yet not on-by-default i.e. that it is opt-in. Which I think is not dissimilar to how it works if it is written by hand.

4 Likes

There should be a warning when you are missing some variants of a non-exhaustive enum (probably opt-in per match). I remember this being mentioned a long time ago on the tracking issue, but I don't think there's been any movement on implementing the lint for it.

2 Likes

I've thought that something like this would be nice. Something I wish was possible was conversion from one error type into another when the target error contains all of the first one's variants:

polyerror::create_error!(SomeError: Err1, Err2);
fn do_stuff() -> Result<(), SomeError> {
    Ok(())
}

polyerror::create_error!(CombinedError: Err1, Err2, Err3);
fn do_other_stuff() -> Result<(), CombinedError> {
    do_stuff()?;
    Ok(())
}

Not sure if there's a way to implement this without it being too cumbersome for the user of the library. I'd be curious to hear if you have any thoughts on a feature like this.

1 Like

Cex author here.

The cex crate simulates of checked exceptions in Rust, which can do this in the following way.

In Cargo.toml:

enumx = "0.4"
cex = "0.5"
use enumx::export::*;
use enumx::predefined::*;
use cex::*;

#[cex]
fn do_stuff() -> Result!( () throws Err1, Err2 ) {
    ret!(());
}

#[cex]
fn do_other_stuff() -> Result!( () throws Err1, Err2, Err3 ) {
    do_stuff()?;
    ret!(());
}

The polyerror crate chooses the "error-per-function" strategy, which is adopted by some previous version of cex. The code above could be written as:

cex! {
    fn do_stuff() -> ()
        throws Foo(Err1)
             , Bar(Err2)
    {
        Ok(())
    }
}

cex! {
    fn do_other_stuff() -> ()
        throws Foo(Err1)
             , Bar(Err2)
             , Baz(Err3)
    {
        do_stuff()?;
        Ok(())
    }
}

The cex!{} macro will produce enum do_stuff::Err and enum do_other_stuff::Err for do_stuff() and do_other_stuff().

But I changed my mind and gave up the "one-error-per-function" strategy in newer versions of cex. Because I noticed a fact that in a library, series of functions may throw the same set of errors, so "one-error-per-function" strategy is only used to track the source of errors which can be done by cex's "logging" mechanism.

Now, the cex library supports even more functionality.

  1. You can call do_other_stuff() inside do_stuff() while propagating errors is still convenient. Notice that the former throws less errors than the latter.

    #[cex]
    fn do_stuff() -> Result!( () throws Err1, Err2 ) {
        do_other_stuff()
        .or_else( |err| #[ty_pat(gen_throws)] match err {
            Err3(_) => /* deal with this error */,
        })
    }
    
    #[cex]
    fn do_other_stuff() -> Result!( () throws Err1, Err2, Err3 ) {
        ret!(());
    }
    
  2. Checked exceptions can fallback as impl std::error::Error or any other traits.

    #[cex]
    fn do_stuff() -> Result!( () throws Err1, Err2 ) {
        ret!(());
    }
    
    fn downstream_api() -> Result<(), impl std::error::Error> {
        Ok( do_stuff()? )
    }
    

@john01dav I'm sorry to introduce cex in your crate's announcement, but many people seems to hate checked exceptions and did not want to discuss in my previous post.

2 Likes

it might be possible to generate the correct from impl for other types to help with the conversion, not sure if this will pass coherence checks

impl<T> From<SomeError> for T
where
    T: From<Err1>,
    T: From<Err2>,
{ ... }

actually no, this probably overlaps with From<T> for T, darn

At least one way to do that would be to implement Into<Err1> and Into<Err2> for SomeError, I think. The rustdoc for Into says that it's bad practice to implement it directly and one should implement From when possible, so I haven't looked into it in depth, but it seems like it may work.

A story of mine:

  1. started with impl From for concrete types.

  2. not satisfied with impl-per-function, began to solve impl From for generic enums.

  3. blocked by "impl overlapping" issue and "phantom generic parameter" came to rescue.

  4. traits with phantom generic parameter e.g. ExchangeFrom, took the place of std::convert::From.

  5. finally, complained that no phantom generic parameter in std::ops::Try.

i think you'd only be able to impl TryInto for those types, since it's an enum and could be one of the other variants.

I feel like the better route might be to tweek the syntax of the macro to let you pass in more information about the other error types

create_error!(SomeError: Err1, Err2);

create_error!(CombinedError: SomeError(Err1, Err2), Err3);

Kinda like you're using a destructuring pattern of SomeError when defining CombinedError. Then you should have enough info to also impl From SomeError for CombinedError.

1 Like