Lifetime issues with async trait

roqvist · October 29, 2019, 4:00pm

Hi,

I'm trying to build a client crate, using a builder pattern. The API I'm targeting supports multiple types of authentication so I thought I'd make this a trait. Also not all requests will require authentication so adding it as part of the builder makes sense.

I'd like to use it something like this:

let client = Client::new(...).with_authentication(...);
let result = client.perform_request().await;

I also want the trait method authenticate to be async, as I understand it this can be done by returning a Pinned Future.

I get a chain of errors regarding lifetimes, I tried following the (friendly) compiler messages to resolve the issue but I can't seem to get it right. Partly because I don't fully understand the Pinned Future result. Can anyone explain why this happening and suggestions on how to fix it? (This is a mini/dummy version of the real crate so it may have syntax errors hidden behind the lifetime error)

use core::pin::Pin;
use core::future::Future;

/// Example Client
/// Client should own the authentication
struct Client<'a> {
    url: String,
    authentication: Option<Box<dyn Authentication + 'a>>
}

/// Client can be constructed with `Client::new(...)` and,
/// if authenticated requests are required, `.with_authentication(...)`
impl<'a> Client<'a> {
    /// Create new Client
    pub fn new(url: &str) -> Self {
        Client {
            url: url.into(),
            authentication: None,
        }
    }
    
    // Add authentication to existing client
    pub fn with_authentication(mut self, authentication: impl Authentication + 'a) -> Self {
        self.authentication = Some(Box::new(authentication));
        self
    }
    
    /// Dummy method that requires authentication
    pub async fn dummy_request(&self) -> String {
        let auth_value = self.authentication.as_ref().unwrap().authenticate(self).await;
        
        // Perform the actual request using auth from above
        "Success".into()
    }
}

trait Authentication {
    // Use a Pinned Future to allow async trait
    fn authenticate(&self, client: &Client) -> Pin<Box<dyn Future<Output = String> + '_>>;
}

// Dummy Authentication implementation
struct DummyAuthentication;

impl DummyAuthentication {
    async fn one_way_to_authenticate(&self, client: &Client<'_>) -> Result<String, ()> {
        println!("DummyAuthentication.one_way_to_authenticate() - this would do something async (with .await)");
        Ok("Bearer alksdjalsjdao4234saijoj".into())
    }
}

/// Implement the trait
impl Authentication for DummyAuthentication {
    fn authenticate(&self, client: &Client) -> Pin<Box<dyn Future<Output = String> + '_>> {
        Box::pin(async move {
            let response = self.one_way_to_authenticate(client).await.unwrap();
            response
        })
    }
}

fn main() {
    let dummy_auth = DummyAuthentication;
    let c = Client::new("www.example.com").with_authentication(dummy_auth);
    
    let res = c.dummy_request();

}

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
error[E0623]: lifetime mismatch
  --> src/main.rs:55:9
   |
54 |       fn authenticate(&self, client: &Client) -> Pin<Box<dyn Future<Output = String> + '_>> {
   |                                      -------     ------------------------------------------
   |                                      |
   |                                      this parameter and the return type are declared with different lifetimes...
55 | /         Box::pin(async move {
56 | |             let response = self.one_way_to_authenticate(client).await.unwrap();
57 | |             response
58 | |         })
   | |__________^ ...but data from `client` is returned here

error: aborting due to previous error

error: could not compile `playground`.

To learn more, run the command again with --verbose.

jonh · October 29, 2019, 8:45pm

fn authenticate<'a>(&'a self, client: &'a Client<'a>) -> Pin<Box<dyn Future<Output = String> + 'a>>;

and same on implementation.
The functions output captures all the input borrows.

alice · October 29, 2019, 8:56pm

Basically what's going on here is that the return value borrows from client, and you have to be explicit about that using a lifetime specifier.

tyranron · October 29, 2019, 9:09pm

Consider to use async-trait, which will desugar lifetimes capturing for you automatically.

roqvist · October 30, 2019, 7:54am

Thanks for explaining,
So how should I read/interpret this?

fn authenticate<'a>(&'a self, client: &'a Client<'a>)

I'll give it a try:

<'a>(&'a self, client: &'a Client

A lifetime 'a is introduced, the references self and client are specified with this same lifetime so neither self nor client can outlive one another.

&'a Client<'a>

This confuses me. To me this looks like Client declares another lifetime (for its' members?), to prevent the internals of Client from outliving self or Client.

I thought by declaring my Client like this:

struct Client<'a> {
    url: String,
    authentication: Option<Box<dyn Authentication + 'a>>
}

I already hinted to Rust that the authentication trait object in that struct cannot outlive the Client itself.

And thanks for suggesting async-trait, it really cleans things up and I'll use it as soon as I fully understand the underlying problem.

alice · October 30, 2019, 8:12am

When analyzing the lifetimes on a function, don't skip the ones in the return position! They're vital to understanding what's going on.

fn authenticate<'a>(&'a self, client: &'a Client<'a>)
    -> Pin<Box<dyn Future<Output = String> + 'a>>

In this case there are three places we might borrow something from:

Things stored inside self. This is the &'a self.
Things stored inside Client. This is the &'a Client.
Things stored somewhere else, that Client has a reference to. This is the Client<'a>.

Note that the things in the third case might live much longer than things in case two. This is why you can give them distinct lifetimes, and doing this allows you to write code that takes a copy of the reference stored in Client, while allowing this reference to live longer than the Client as long as the reference inside the Client is longer lived than the Client.

In this case you don't make use of this ability, but it's why there are two distinct lifetimes.

Now what about the return value? Well by writing dyn Trait + 'a you're saying that this is an instance of Trait and that you're okay with this instance containing references with the lifetime 'a. Since you bound the lifetime 'a to all three positions, you're saying that:

The output can contain references into something stored in self.
The output can contain references into something stored directly in Client.
The output can also take copies of references stored inside Client.

Note that it could also be written like this:

fn authenticate<'a, 'b, 'c>(&'a self, client: &'b Client<'c>)
    -> Pin<Box<dyn Future<Output = String> + 'a + 'b + 'c>>

In this case the two are equivalent, but this might make it more clear what it's saying. The reason they're equivalent is that if you have e.g. a Client<'static> in which the Authentication contains no references, then the compiler will automatically shorten the 'static lifetime to whatever lifetime you need.

Note that reusing the same lifetime like this is a bad idea when you're dealing with mutable lifetimes, because mutable lifetimes must be unique, and this can make the requirement that the lifetimes are exactly equal give problems. In this case they're immutable, so it's fine.

Basically: Lifetimes on functions are used to tell the compiler in which ways the output is allowed to borrow from the various inputs.

roqvist · October 30, 2019, 8:30am

This is a really excellent description, thanks a lot! I think for the puzzle pieces to really fall in place I need to find an example where a struct has references that is longer lived than the struct itself (the "ability" you mention above, that Is not used in my case). It sounds like something that can be useful. Do you know of any such examples? Is it described in the Rust book?

In the actual crate the authentication will probably be mutable, since it will update its access_token when needed. So I'll tackle that next.

alice · October 30, 2019, 8:49am

One example of this "ability" would be something like this:

#[derive(Deserialize)]
struct ParsedData<'a> {
    name: &'a str,
    some_other_value: String,
}

#[derive(Debug)]
struct HasName<'a> {
    name: &'a str,
}

fn extract_name<'a>(data: &ParsedData<'a>) -> HasName<'a> {
    HasName {
        name: data.name,
    }
}

fn main() {
    let data = r#"{ "name": "test", "some_other_value": "hello" }"#;
    
    // Parse the above json. Parsed will contain slices into data.
    let parsed = serde_json::from_str(data).unwrap();
    
    // The name value now contains a value it took from parsed.
    let name = extract_name(&parsed);
    
    // Now we drop parsed.
    drop(parsed);

    // However we can stil use name, because it's really borrowing from data, not parsed.  
    println!("{:?}", name);
}

playground

Notice that the reference to ParsedData itself has no lifetime. This implicitly means that we don't borrow from anything stored in the value. For example we would not be able to take a reference to the String without using the lifetime on the reference, because this is owned by ParsedData.

To see what happens if you try to do that, see here.

system · January 28, 2020, 8:49am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.