Returning a reqwest::Response from an async method without lifetime bounds

[jwimberl]

In the following mockup, a struct containing a reqwest::Client can have a method demo_good returning a boxed future reqwest::Response, without any lifetime constraint on the object owning the client. However, if I split this logic up into an async demo_post method to make the actual post request to the client -- a placeholder for more complicated logic that will catch 401s and reauthenticate -- and demo_bad that uses this helper method run a query an return a boxed future reqwest::Response. Logically, the response does not use depend on data in the struct and so the demo_good method compiles without issue. However, there is a compilation error for the demo_post + demo_bad method, as the compiler flags the future returned by the first to be tied to the lifetime of self:

use core::pin::Pin;
use reqwest::{Client, Error, Response, Url};
use std::collections::HashMap;

struct SessionManager {
    client: Client,
}

impl SessionManager {
    // POST to the execute_url, handling reauthentication
    async fn demo_post(
        &mut self,
        url: Url,
        body: HashMap<String, String>,
    ) -> Result<Response, Error> {
        // Run first request
        let response = self.client.post(url).json(&body).send().await;
        // Todo: catch 401s, reauth, and repeat
        dbg!(&response);
        // Return
        response
    }

    pub fn demo_good(
        &mut self,
        url: Url,
    ) -> Pin<Box<dyn futures::Future<Output = Result<Response, Error>> + Send>> {
        let mut map = HashMap::new();
        map.insert("fake_key".to_string(), "fake_value".to_string());
        let response = self.client.post(url).json(&map).send();
        Box::pin(response)
    }

    pub fn demo_bad(
        &mut self,
        url: Url,
    ) -> Pin<Box<dyn futures::Future<Output = Result<Response, Error>> + Send>> {
        let mut map = HashMap::new();
        map.insert("fake_key".to_string(), "fake_value".to_string());
        let response = self.demo_post(url, map);
        Box::pin(response)
    }
}

(Playground)

The error message is:

   Compiling playground v0.0.1 (/playground)
error: lifetime may not live long enough
  --> src/lib.rs:41:9
   |
35 |         &mut self,
   |         - let's call the lifetime of this reference `'1`
...
41 |         Box::pin(response)
   |         ^^^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'static`
   |
help: to declare that the trait object captures data from argument `self`, you can add an explicit `'_` lifetime bound
   |
37 |     ) -> Pin<Box<dyn futures::Future<Output = Result<Response, Error>> + Send + '_>> {
   |                                                                               ++++

error: could not compile `playground` (lib) due to 1 previous error

I'm wary of blindling following the recommendation to add a '_ lifetime bound, because while that makes this playground demo compile I believe that in my actual use case it will just kick the problem further down the road. I have client code right now that uses the returned reqwest::Response from the real version of demo_good just fine, creating and using a bytes stream, but if I switch to the real version of demo_bad with the '_ lifetime bound, there just ends up being a error[E0515]: cannot return value referencing temporary value at some point.

So, if possible I'd like to make this work just by making changes in the SessionManager struct and without making changes in lifetime bounds of the returned responses. Is there a way to make this work?

(Posted originally at https://stackoverflow.com/questions/78697226/returning-and-using-an-reqwestresponse-from-an-async-method before I saw a link this this forum from the Rust Playground)

[jwimberl]

Perhaps this is a duplicate of How to define async methods without capturing lifetime of self - Rust Internals?

[nerditation]

@jwimberl

However, there is a compilation error for the demo_post + demo_bad method, as the compiler flags the future returned by the first to be tied to the lifetime of self:

dyn Trait is implicitly equivalent to dyn Trait + 'static, so your type signature for demo_bad() requires you to return a boxed future with 'static lifetime, but the (opaque) future returned by the async demo_post() actual captures self.

you can change the signature of demo_bad() to explicitly mention the lifetime, like this:

pub fn demo_bad<'a>(
    &'a mut self,
    url: Url,
) -> Pin<Box<dyn futures::Future<Output = Result<Response, Error>> + Send + 'a>> {
    //...
}

note, this compiles for this example, but the "correct" way is to use the "Captures trick". you can read this thread to learn about the "Captures trick":

How to avoid using the Captures trait here?

#![deny(elided_lifetimes_in_paths)] enum Vector<'a, T> { Borrowed(&'a mut [T]), Owned(Vec<T>), } impl<'a, T> Vector<'a, T> { fn iter(&self) -> impl Iterator<Item = &T> { match self { Self::Borrowed(value) => value.iter(), Self::Owned(value) => value.iter(), } } fn iter_mut(&mut self) -> impl Iterator<Item = &mut T> { match self { Self::Borrowed(value) => value.iter_mut(), Self::Owned(value) => value.it…

[jwimberl]

@nerditation Thanks -- but the thing is, I don't want the output of demo_bad to capture self. demo_good doesn't, and logically speaking (though of course not in actually) demo_bad isn't doing anything different and isn't capturing self for any good reason. I tested adding the lifetime signature and it causes problems further up the stack. So, I'm thinking about turning demo_post into a non-async function that returns a future, if I can.

[SkiFire13]

If you want to avoid demo_post from capturing self's lifetime you'll have to avoid using self in the returned Future. Obviously you need to use self to perform the request, however luckily this only happens before the very first .await, meaning you can do that operation before creating the Future. To do this you'll need to avoid using an async fn, as that put the whole body of the function in the future. Instead you'll need to make it return an impl Future<Output = ...>. Something like this:

    fn demo_post(
        &mut self,
        url: Url,
        body: HashMap<String, String>,
    ) -> impl Future<Output = Result<Response, Error>> + Send {
        let response_fut = self.client.post(url).json(&body).send();
        
        async move {
            let response = response_fut.await;
            // Todo: catch 401s, reauth, and repeat
            dbg!(&response);
            // Return
            response
        }
    }

Rust Playground

[jwimberl]

Is there perhaps some third way if I make all the members of the struct automatically reference counted? According to this forum thread:

Fn method(self : Arc<Self>) is it a good idea?

This heavily depends on what a and b do, but one thing you can do is change S. #[derive(Clone)] struct S { t: Arc<Mutex<String>>, } Then you can use &self everywhere and clone it when you want to send it to another thread. You could also leave S alone and wrap it in another struct. #[derive(Clone)] struct S2 { s: Arc<Mutex<S>>, } This allows you to leave b on S, while a is on S2. Or just have a be an associated function instead of a method. async fn a(s: Arc<Mutex<Self>>)

that worked around similar compiler errors. I belatedly learned after all that, while the async move solution could work given what I implemented so far, it might not be easily compatible with the work to be added that's just the Todo: comment right now

[SkiFire13]

@jwimberl

Is there perhaps some third way if I make all the members of the struct automatically reference counted?

You can make demo_post take self (if all the elements are reference counted then you can always call it with .clone().demo_post(url, body).await) or take Rc<Self>/Arc<Self> (in this case you'll have to wrap the SessionManager itself in a Rc/Arc, and you'll still likely need the .clone())

If you combine this with my solution however you can just take &self and then do the .clone() in the function outside the async move {} block.