How to wrap errors from a third party library?

Hi all,

I'm writing a library that wraps AWS services and it's fair to say that their SDK errors are ... complex. All services derive from this type defined in the smithy api, and there's a few others (e.g. from "watchers")

#[derive(Debug)]
pub enum SdkError<E, R> {
    ConstructionFailure(ConstructionFailure),
    TimeoutError(TimeoutError),
    DispatchFailure(DispatchFailure),
    ResponseError(ResponseError<R>),
    ServiceError(ServiceError<E, R>),
}
...
impl<E, R> Error for SdkError<E, R>
where
    E: Error + 'static,
    R: Debug,
{
  ...
}

in this case, from SdkError in aws_smithy_runtime_api::client::result - Rust

The main problem is the use of generics, because to be able to include these in my wrapper type I need to make it generic too, and I'd rather avoid that.

My sledgehammer approach is to wrap those with my own error type, and the only thing they all have in common is a big ol Box of tricks... (I'm using some macros from derive_more here in case you're wondering)

#[derive(Debug, Display, Error, From)]
pub enum Fail {
    MyError(#[error(not(source))] String),
    DependencyError {
        source: Box<dyn std::error::Error + 'static + Send>,
    },
}

but I'd like to be a little bit nicer, and also protect myself from including Fail inside Fail (which I've done a few times already, it's easily done) and have an entry in my enum for each one of the services that can fail and preserve their structure. But I'm not getting very far. I thought it would be possible to add an entry and put the most basic constraints on them like here

    SmithyApiError(
        aws_smithy_runtime_api::client::result::SdkError<
            Box<dyn std::error::Error + Send + 'static>,
            Box<dyn std::fmt::Debug + Send + 'static>,
        >,
    ),

so I'd get the Send + 'static that my code needs, and the impl Error for SdkError would get what it needs too. But this fails to compile, which I can fix if I disable the derive_more Error derivation and implement it by hand if necessary... although I still need to write lots of constructor wrappers because the types don't actually line up since I have Box here and the actual errors have types with known sizes. I have no way to convert between a SdkError<E, R> and an SdkError<Box<...>, Box<...>> anyways so I'm not sure if this is even a feasible approach.

A small step I can make is to mark the Fail as #[non_exhaustive] and then only ever create instances via Froms tailored to each of the SDK errors, which at least stops me re-wrapping my own errors or introducing something unexpected.

impl<E, R> From<aws_smithy_runtime_api::client::result::SdkError<E, R>> for Fail
where
    E: Error + Send + 'static,
    R: Debug + Send + 'static,
{
    fn from(e: aws_smithy_runtime_api::client::result::SdkError<E, R>) -> Fail {
        Fail::DependencyError {
            source: Box::new(e),
        }
    }
}

impl<O, E> From<aws_smithy_runtime_api::client::waiters::error::WaiterError<O, E>> for Fail
where
    O: Debug + Send + 'static,
    E: Error + Debug + Send + 'static,
{
    fn from(e: aws_smithy_runtime_api::client::waiters::error::WaiterError<O, E>) -> Fail {
        Fail::DependencyError {
            source: Box::new(e),
        }
    }
}

which does help with error creation, but it still feels like I'm still not giving enough information back to my users. They don't know anything about the sources and can't really inspect them.

Also, maybe I'm asking the wrong question and the question I should be asking is: how should I be idiomatically wrapping up errors that come from third party libraries when I have no choice but to bubble them up to the user? (and also, in this case the user would probably appreciate getting specific AWS service errors because these sorts of things are usually related to security policies and credentials and environments and as much context as possible is always welcome).

(Apologies for this long and rambling post, I'm definitely not really sure what to ask for here :laughing:)

I think the approach can also be informed by your use case. I assume you’ll want to wrap these dependency-errors because of API you’re calling – maybe you can list some examples of concrete functions that you’re calling?

It it’s only a handful of functions that each have some specific SdkError<SpecificEType, SpecificRType> return type signatures, you can create multiple variants and just present those specific error types.

There also seems to be some middle ground, i.e. aws APIs already appear to contain some less specific “Error” types (often huge enums):

E.g. if I pick something random[1] like … say aws-sdk-rds; looking for functions returning SdkError (click “In Return Types”), it looks like they all use HttpResponse for R, and some error type for E which always differs… however, at least within that crate, the errors all appear to at least be convertible to a single “Error” type, which means you could just have an enum variant like this

RdsError(
        aws_smithy_runtime_api::client::result::SdkError<
            aws_sdk_rds::Error,
            HttpResponse,
        >,
    ),

where you can do the conversion via .map_service_error(From::from)

For example, the first search result above, this function, returns SdkError<AddRoleToDBClusterError, HttpResponse> and you can find the relevant conversion impl listed here.


So that would avoid the Boxing and allow any users to still inspect the error in quite some detail.

If you use multiple aws-sdk-… APIs, one approach could be to have a new variant for each of them. (Looking at a few of them, it seems like a common theme each provides at least their own general Error type with a From<…> implementations for all the more specific ones.)


Regarding the question of how to best implement this conversion with or without derive_more, it’s been a while since I’ve last looked into derive_more, so no idea if it has features to allow more customization for specific impls.

A hand-written impl containing the abovementioned map_service_error(From::from) (for the example case of aws rds) call could use E: Into<aws_sdk_rds::Error> to implement a conversion from any SdkError<E, HttpError> for relevant aws_sdk_rds error types.


  1. I know nothing about AWS SDK by the way ↩︎

4 Likes

just gotta say, the quality of this answer is much better than anything I could have hoped for, thank you so much! :ok_hand:

1 Like

I started to follow your suggestion and automate some aspects of it with

pub enum Fail {
    MyError(#[error(not(source))] String),
    KinesisError {
        source: SdkError<aws_sdk_kinesis::Error, HttpResponse>,
    },
    DynamoDbError {
        source: SdkError<aws_sdk_dynamodb::Error, HttpResponse>,
    },
}

and then From impls requiring the Into that you spotted ...

impl<E> From<SdkError<E, HttpResponse>> for Fail
where
    E: Into<aws_sdk_kinesis::Error>,
{
...
}

impl<E> From<SdkError<E, HttpResponse>> for Fail
where
    E: Into<aws_sdk_dynamodb::Error>,
{
...
}

but of course this won't work because it's a conflicting implementation :man_facepalming: aka Workaround for conflicting implementation (negative trait bound)?

This is already much better for the end user. It seems that I might have to compromise on having simpler internal error handling (explicit map_err handling instead of ? being able to call the relevant From for any given third party error) vs retaining specific information for the user... or just suck up the boilerplate and write explicit From instances until we get specialization

impl From<SdkError<GetShardIteratorError, HttpResponse>> for Fail {
    fn from(e: SdkError<GetShardIteratorError, HttpResponse>) -> Fail {
        Fail::KinesisError { source: e.map_service_error(|s| s.into()) }
    }
}

impl From<SdkError<CreateTableError, HttpResponse>> for Fail {
    fn from(e: SdkError<CreateTableError, HttpResponse>) -> Fail {
        Fail::DynamoDbError { source: e.map_service_error(|s| s.into()) }
    }
}

... many many more ...

(tempted to write a macro...)

Ah, yes of course, damn!

That should be quite straightforward actually. You just need to copy the respective list of operations, e.g. the list of modules from this page[1], the rest is easily handled with the help of paste:

use aws_smithy_runtime_api::client::result::SdkError;
use aws_smithy_runtime_api::http::Response as HttpResponse;

pub enum Fail {
    MyError(String),
    KinesisError {
        source: SdkError<aws_sdk_kinesis::Error, HttpResponse>,
    },
    DynamoDbError {
        source: SdkError<aws_sdk_dynamodb::Error, HttpResponse>,
    },
}

macro_rules! impl_conversions {
    ($crt:ident $Variant:ident: $($op:ident)*) => {
        paste::paste!{
            $(
                impl From<SdkError<$crt::operation::$op::[<$op:camel Error>], HttpResponse>> for Fail {
                    fn from(value: SdkError<$crt::operation::$op::[<$op:camel Error>], HttpResponse>) -> Self {
                        Fail::$Variant { source: value.map_service_error(From::from) }
                    }
                }
            )*
        }
    }
}

impl_conversions! { aws_sdk_kinesis KinesisError:
    add_tags_to_stream
    create_stream
    decrease_stream_retention_period
    delete_resource_policy
    delete_stream
    deregister_stream_consumer
    describe_limits
    describe_stream
    describe_stream_consumer
    describe_stream_summary
    disable_enhanced_monitoring
    enable_enhanced_monitoring
    get_records
    get_resource_policy
    get_shard_iterator
    increase_stream_retention_period
    list_shards
    list_stream_consumers
    list_streams
    list_tags_for_stream
    merge_shards
    put_record
    put_records
    put_resource_policy
    register_stream_consumer
    remove_tags_from_stream
    split_shard
    start_stream_encryption
    stop_stream_encryption
    update_shard_count
    update_stream_mode
}

impl_conversions! { aws_sdk_dynamodb DynamoDbError:
    batch_execute_statement
    batch_get_item
    batch_write_item
    create_backup
    create_global_table
    create_table
    delete_backup
    delete_item
    delete_resource_policy
    delete_table
    describe_backup
    describe_continuous_backups
    describe_contributor_insights
    describe_endpoints
    describe_export
    describe_global_table
    describe_global_table_settings
    describe_import
    describe_kinesis_streaming_destination
    describe_limits
    describe_table
    describe_table_replica_auto_scaling
    describe_time_to_live
    disable_kinesis_streaming_destination
    enable_kinesis_streaming_destination
    execute_statement
    execute_transaction
    export_table_to_point_in_time
    get_item
    get_resource_policy
    import_table
    list_backups
    list_contributor_insights
    list_exports
    list_global_tables
    list_imports
    list_tables
    list_tags_of_resource
    put_item
    put_resource_policy
    query
    restore_table_from_backup
    restore_table_to_point_in_time
    scan
    tag_resource
    transact_get_items
    transact_write_items
    untag_resource
    update_continuous_backups
    update_contributor_insights
    update_global_table
    update_global_table_settings
    update_item
    update_kinesis_streaming_destination
    update_table
    update_table_replica_auto_scaling
    update_time_to_live
}

  1. copy paste, then whichever editor tricks you prefer to only keep the first “word” per line ↩︎

1 Like

thanks again!

This ended up getting even more complicated when I considered errors in the aws sdk related to "watchers" (i.e. keep watching this until some condition is met, like "the database exists") ... support WaiterError unwrapping · Issue #3925 · smithy-lang/smithy-rs · GitHub

Steffahn gave you very thorough answer and I don't have anything to add to it, but I'll leave you a link to a great post by Sabrina Jewson -- Modular Errors in Rust, which you can find informative on this topic.

2 Likes