How to expose timeouts from a library?

Hi all, I'm working on a new SDK for a database and wondering about community opinions on how to approach exposing timeouts for async operations?

  • a) Implement timeout values on each operation and have the operations able to return timeout errors
  • b) Leave it to the user to just drop the future whenever they want, which also better supports general cancellation.

From what I can see most libraries follow b. A large consideration to my mind is that even implementing a doesn't prevent users from doing b.

However, in other languages we attach valuable debugging information to timeout errors (like routing information, number of retries, for the operation etc...) and I don't see an obvious way to expose this information via b. Maybe detect the future is being dropped and write the operation context to a log?

4 Likes

Ping -- curious about this too!

from a perspective of composability, I would generally prefer option b. this also motivates you to consider the cancellibility when you authorize the futures.

if I were to choose option a, one problem I usually cannot decide is how to make the API ergonomic? how to deal with optional timeout? etc.

// use two funcitons?
async fn send_request_with_timeout(foo: Foo, bar: Bar, timeout: Duration) -> Response {
    //...
}
async fn send_request(foo: Foo, bar: Bar) -> Response {
    let default_timeout = Duration::MAX;
    send_request_with_timeout(foo, bar, default_time).await
}

// or use one function with `Option` parameter?
async fn send_request_with_timeout(foo: Foo, bar: Bar, timeout: Option<Duration>) -> Response {
    let timeout = timeout.unwrap_or(Duration::MAX);
}

// it's less an issue for builder pattern though
let response = RequestBuilder::new()
        .foo(foo)
        .bar(bar)
        .timeout(timeout)
        .send(&server)
        .await;

that is not to say, I never choose option a. for example, if the operation (some form of IO involved) is fallible by its nature and you must return a Result, chances are the underlying IoError already covers the timeout cases. also, when a fallible result composed with an external timeout, it's likely the return type would look like Result<Result<T, IoError>, TimeoutError>, which feels cumbersome, but this is just minor inconvenience.

now back to why I prefer option b.

in async rust, you can use combinators (such as futures_lite::or(), or futures::Either) to compose Futures.

if you write the Future types with composability in mind, you can easily implement a generic timeout mechanism for them. here's an example using the futures_lite::or() combinator:

// here I wrap the return type with `Result`, you can use `Either` or whatever.
async fn with_timeout<F: Future>(timeout: Duration, f: F) -> Result<F::Output, TimeoutError> {
    or(
        async {
            let value = f.await;
            Ok(value)
        },
        async {
            sleep(timeout).await;
            Err(TimeoutError)
        }
    ).await
}

and this scheme can easily be extended to support other cancellations, or to add whatever contexts to the TimeoutError as needed.

also, for user ergonomics, I don't think it is too bad, it can be implemented as extension traits so the callsite just need to add a method call:

// no timeout
let response = server.send_request(...).await;

// with timeout,
// this example uses question mark to propogate the timeout error 
let response = server.send_request(...).with_timeout(Duration::from_millis(500)).await?;

// for builder style API, this feels natural too:
let response = RequestBuilder::new()
        .parameter_foo(true)
        .parameter_bar("apple")
        .send(&server)
        .with_timeout(Duration::from_millis(500))
        .await
        .inspect_err(|e| log!("timeout: {e:?}")?;
1 Like

Hi @nerditation thanks for your input.

if I were to choose option a, one problem I usually cannot decide is how to make the API ergonomic? how to deal with optional timeout?

We use a pattern similar to the builder pattern which should make it quite ergonomic.

I think that you're correct about option b, it generally just feels more idiomatic to rust. It just leaves us unable to report diagnostics information to the user as they'd own performing the timeout themselves.

I'm wondering if we should implement option a and recommend it to users whilst also ensuring that our futures place nicely with option b (and don't leave library internals in a weird state). That might be the best of both worlds... maybe.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.