Moving from struct to trait with tokio::spawn caused has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement

help
#1

Hello,

I am trying to refactor some working code, replacing a struct with a trait so that I can facilitate my life during unit tests.
This operation causes the compiler to complain about the lifetime.

error[E0759]: `app_client` has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement
    |
47  | pub async fn execute(app_client: &dyn WriterAppInitialisation, event: LambdaEvent<SqsEvent>) -> Result<Value, Error> {
    |                      ^^^^^^^^^^  ---------------------------- this data with an anonymous lifetime `'_`...
    |                      |
    |                      ...is used here...
...
52  |     let shared_client = Arc::from(app_client.clone());
    |                                   ---------- ...is used here...
...
59  |         tasks.push(tokio::spawn(async move {
    |                    ------------ ...and is required to live as long as `'static` here
    |
note: `'static` lifetime requirement introduced by this bound
   -->.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/task/spawn.rs:127:28
    |
127 |         T: Future + Send + 'static,

I am reading all around, but I do not get why.

#[derive(Debug, Clone, Builder)]
pub struct WriterAppClient {
    #[builder(setter(into))]
    pub dynamo_db_client: aws_sdk_dynamodb::Client,

    #[builder(setter(into))]
    pub add_query: AddQuery,
}

#[cfg_attr(test, automock)]
#[async_trait]
pub trait WriterAppInitialisation: Send + Sync {
    async fn add_query(&self, item: &MyStruct) -> Result<(), ApplicationError>;
}

#[async_trait]
impl WriterAppInitialisation for WriterAppClient {
    async fn add_query(&self, item: &MyStruct) -> Result<(), ApplicationError> {
        self.add_query.execute(item).await
    }
}


#[tokio::main]
async fn main() -> Result<(), Error> {
    // stuff
    let app_client = WriterAppClient::builder()
        .add_query(query)
        .dynamo_db_client(dynamodb_client.clone())
        .build();

    lambda_runtime::run(service_fn(|event: LambdaEvent<SqsEvent>| {
        execute(&app_client, event)
    }))
    .await?;
    Ok(())
}

//It is all working with the struct WriterAppClient
//pub async fn execute(app_client: &WriterAppClient, event: LambdaEvent<SqsEvent>) -> Result<Value, Error> {

pub async fn execute(app_client: &dyn WriterAppInitialisation, event: LambdaEvent<SqsEvent>) -> Result<Value, Error> {
    let mut tasks = Vec::with_capacity(event.payload.records.len());
    let shared_client = Arc::new(app_client.clone());
    
    for record in event.payload.records.into_iter() {
        let shared_client = shared_client.clone();
        tasks.push(tokio::spawn(async move {
            if let Some(body) = &record.body {
                let request = serde_json::from_str::<MyStruct>(&body);
                if let Ok(request) = request {
                    shared_client.add_query(&request)
                      .await
                      .map_or_else(|e| {
                        println!("ERROR {:?}", e);
                      }, |_| ());
                } 
            }
        }));
    }

    join_all(tasks).await;
 //other stuff
}

I am not really sure why and how to move forward.
Is anyone kind enough to explain what is wrong and why?

#2

The problem is that the closure passed into tokio::spawn is expected to live for "arbitrarily" long (hence the 'static bound). However, app_client need not live that long.
For regular threads the answer would be to use crossbeam and use the scoped-thread.
I am guessing tokio would have something in that vein. @alice would be able to tell.

#3

Using a struct has the same problem, but using Arc makes it works inside the tokio::spawn closure.

Why with the trait is not the same?

#4

Welll, you could use an Arc<Box<dyn Trait>>, if that's what you're looking for.

#5

To be honest, I am not sure if I understand completely what you suggested.

If I want to try your suggestion, you said I should have it here.

pub async fn execute(app_client: Arc<Box<dyn WriterAppInitialisation>>

but this will say:

mismatched types 
expected struct `Arc<Box<(dyn WriterAppInitialisation + 'static)>>`
found struct `Arc<Box<WriterAppClient>>`rustc[E0308](https://doc.rust-lang.org/error-index.html#E0308)

or from this
pub async fn execute(app_client: &dyn WriterAppInitialisation

trying to add the box etc.?

I am sorry for the silly questions. I am not so advanced in Rust and each hint is more study :slight_smile:

#6

Please properly format your error message, it's quite impossible to read.

#7

I formatted the error message

I am reading this article dyn Trait and impl Trait in Rust.
It seems the trait idea is correct, but I think it is missing some types to make it work inside the tokio::spawn closure.

#8

FYI, it doesn't, because there isn't any sound way to offer such an API (yet).

For simple workarounds in this case, defining and using a method fn clone_to_arc(&self) -> Arc<dyn WriterAppInitialisation> on the WriterAppInitialisation trait should be enough to make the code work again.

Another (more general) workaround for the lack of scoped spawning is that often using FuturesUnordered can be an alternative. It doesn't spawn “real” tasks so you don't get actual parallelism for non-async parts of the computation, but otherwise it's still quite capable and efficient at handling a lot of concurrent tasks.

2 Likes
#9

Hello @steffahn

I did as you suggested adding this method to the trait and implementing it in this way:

    fn clone_to_arc(&self) -> Arc<dyn WriterAppInitialisation> {
        Arc::new(self.clone())
    }

and now the code compile:

for record in event.payload.records.into_iter() {
        let shared_client = app_client.clone_to_arc();
        tasks.push(tokio::spawn(async move {
            if let Some(body) = &record.body {
                let request = serde_json::from_str::<MyStruct>(&body);
                if let Ok(request) = request {
                    shared_client.add_query(&request)
                      .await
                      .map_or_else(|e| {
                        println!("ERROR {:?}", e);
                      }, |_| ());
                } 
            }
        }));
    }

Do you mind explaining to me with the most simple words what is happening?

#10

In short, app_client.clone() on app_client: &dyn WriterAppInitialization doesn't do what you might expect in the first place. The type dyn WriterAppInitialization doesn't support cloning (trait objects in general can't easily be cloned unless you add dedicated methods for doing so) so the method resolution would fall back to cloning (i. e. copying) the reference, so you're producing a Arc<&dyn WriterAppInitialization>. This type still has a lifetime, so the Arc doesn't accomplish anything this way.

Instead Arc<dyn WriterAppInitialization> does no longer have any (non-'static) lifetimes, so you can use that type to pass into a spawned task. In order to create it from &dyn WriterAppInitialization you'll need to add a dedicated method to the trait, the Arc::new(self.clone()) in the implementation of that method then works because it's .clone() call acts on the concrete known (sized and clonable) type that implements WriterAppInitialization.

3 Likes