Passing Fn closure to async trait method

I tried to make a follow up on the working example with rocket from the question:

by passing WriterFn to the store_blob function.

trait WriterFn = FnOnce(&Path) -> Pin<Box<dyn Future<Output = Result<(), io::Error>> + Send>>;

#[rocket::async_trait] // I think thats not needed. ???
trait BlobStorage2: Sync + Send {
    async fn store_blob(&self, writer: Box<dyn WriterFn>) -> bool {
        true
    }
}

struct Storage;

#[rocket::async_trait]
impl BlobStorage2 for Storage {
    async fn store_blob(&self, writer: Box<dyn WriterFn>) -> bool {
        writer(&PathBuf::from("path-where-to-store")).await;
        true
    }
}

async fn store(p: &mut TempFile<'_>, i: Arc<dyn BlobStorage2>) {
    let f = Box::new(|r: &Path| Box::pin(p.copy_to(r)));

    i.store_blob(f).await; //////// Does not compile, 
}
.....

The example is here

I am wondering what the compiler tries to tell me and what I should fix. I see this example with a closure WriterFn the same as passing a local struct FileWriter in the original example L62. The original example had a generic trait BlobStorage<W> where as the new Closure approach needs a boxed (allocated) closure (type erased).

Thanks alot for the help and explanations.

The error I get is:

 = help: the trait `std::marker::Send` is not implemented for `dyn FnOnce(&Path) -> Pin<Box<dyn Future<Output = Result<(), Error>> + Send>>`

I cleaned up most of the compiler errors:

trait WriterFn = FnOnce(&Path) -> Pin<Box<dyn Future<Output = Result<(), io::Error>> + Send>> + Send;

/// Simple interface which provides access to blob storage
/// in different components of the application.
#[rocket::async_trait]
trait BlobStorage2: Sync + Send {
    async fn store_blob(&self, writer: Box<dyn WriterFn + Send>) -> bool {
        true
    }
}

/// ======================================================================
pub struct Storage;

#[rocket::async_trait]
impl BlobStorage2 for Storage {
    async fn store_blob(&self, writer: Box<dyn WriterFn + Send>) -> bool {
        writer(&PathBuf::from("path-where-to-store")).await;
        true
    }
}

async fn store(p: &mut TempFile<'_>, i: Arc<dyn BlobStorage2>) {
    let f = Box::new(|r: &Path| Box::pin(p.copy_to(r)) as _);

    i.store_blob(f).await;
}

#[derive(rocket::FromForm, Debug)]
struct JobUpload<'r> {
    file: TempFile<'r>,
}

#[rocket::put("/api/job", data = "<job>")]
async fn submit_job(mut job: Form<JobUpload<'_>>) -> rocket::http::Status {
    let p = Arc::new(Storage {});
    store(&mut job.file, p.clone()).await;

    Status::Continue
}

#[rocket::main]
async fn main() -> Result<(), rocket::Error> {
    let r = rocket::build().mount("/", rocket::routes![submit_job]);
    r.launch().await;
    Ok(())
}

The remaining issue is one that you'll have to deal with by restructuring your code. The thing is that you are passing a reference to store here:

store(&mut job.file, p.clone()).await;

And this reference is captured by closure, which is not allowed since async callbacks cannot take references.

My knowledge of Rust is limited here, maybe someone else can help out.

1 Like

Its strange that the original example somehow works with a local stack object FileWriter which I try now to simulate with a closure but it seems I cannot correctly capture the values in the closure such that the compiler is satisfied with the lifetimes??

The closure is resolved as FnMut ( has a call(&mut self,...)) because I assume that it stores internally a &mut TempFile which it cannot access otherwise (needed in p.copy_to)

One more cleanup: Create the closure at the call site to influence interpretation as a FnOnce

async fn store(p: &mut TempFile<'_>, i: Arc<dyn BlobStorage2>) {
    i.store_blob(
        Box::new(|r: &Path| Box::pin(p.copy_to(r)) as _)
    ).await;
}

Now, what do we have here? store_blob takes a...

Box<dyn WriterFn + Send /* + 'static */>

The outer Box in the call is capturing p, so it can't be 'static. If that's ok it would look like...

    async fn store_blob(&self, writer: Box<dyn WriterFn + Send + '_>) -> bool {

Next, for the inner box we have

trait WriterFn = FnOnce(&Path) -> 
  Pin<Box<
      dyn /* 'static + */ Future<Output = Result<(), io::Error>> + Send
  >> + Send;

As an async fn, TempFile<'_>::copy_to is going to capture all the lifetimes in the sigature, which include its lifetime parameter, the lifetime of the &mut self, and the lifetime of the p you pass in. So, it can't be 'static either.

You can try to capture less things and be more lenient about capturing references...

// The move to `PathBuf` is to avoid trying to encode that the return
// type lifetime is capped by the input parameter lifetime
trait WriterFn<'a> = FnOnce(PathBuf) -> Pin<Box<dyn 'a + Future<Output = Result<(), io::Error>> + Send>> + Send;
// Adjust the rest of the code

But then I end up with

error[E0621]: explicit lifetime required in the type of `writer`
  --> src/main.rs:26:83
   |
26 |       async fn store_blob(&self, writer: Box<dyn WriterFn<'_> + Send + '_>) -> bool {
   |  ________________________________________---------------------------------__________^
   | |                                        |
   | |                                        help: add explicit lifetime `'async_trait` to the type of `writer`: `Box<(dyn FnOnce(PathBuf) -> Pin<Box<(dyn std::future::Future<Output = Result<(), std::io::Error>> + std::marker::Send + 'life1)>> + std::marker::Send + 'async_trait)>`
27 | |         true
28 | |     }
   | |_____^ lifetime `'async_trait` required

error: lifetime may not live long enough
  --> src/main.rs:36:83
   |
36 |       async fn store_blob(&self, writer: Box<dyn WriterFn<'_> + Send + '_>) -> bool {
   |  _____-----______________________------_____________________________________________^
   | |     |                          |
   | |     |                          has type `Box<(dyn FnOnce(PathBuf) -> Pin<Box<dyn std::future::Future<Output = Result<(), std::io::Error>> + std::marker::Send>> + std::marker::Send + '1)>`
   | |     lifetime `'async_trait` defined here
37 | |         writer(PathBuf::from("path-where-to-store")).await;
38 | |         true
39 | |     }
   | |_____^ method was supposed to return data with lifetime `'async_trait` but it is returning data with lifetime `'1`

Which could be as simple as "capturing references isn't supported" or could be something else, hard to say without knowing what the macro does in depth.

1 Like

The following works

which is a bit weird. It seams that we need to be overly lenient with lifetimes, I specified all, and now boom it compiles. Understandable is that however not really...

Here is the same without async_macro.

trait WriterFn<'fA> =
    FnOnce(PathBuf) -> Pin<Box<dyn 'fA + Future<Output = Result<(), io::Error>> + Send>> + Send;

/// Simple interface which provides access to blob storage
/// in different components of the application.
trait BlobStorage2: Sync + Send {
    fn store_blob<'a, 'fB: 'fD, 'c: 'fD, 'fD>(
        &'a self,
        writer: Box<dyn WriterFn<'fB> + Send + 'c>,
    ) -> Pin<Box<dyn Future<Output = bool> + Send + 'fD>> {
        Box::pin(async { true })
    }
}

The constraints mean the following:

  • feature fB outlives feature fD
  • the writer outlives feature fD

Without these constraints it does not compile...

1 Like

Interesting! I have simplified your lifetimes. It seems that I was missing a bound when I gave it a try yesterday.


#![feature(trait_alias)]
use core::future::Future;
use rocket::{self, form::Form, fs::TempFile, http::Status, routes};
use std::{
    io,
    path::{Path, PathBuf},
    pin::Pin,
    sync::Arc,
};

trait WriterFn<'a> =
    FnOnce(PathBuf) -> Pin<Box<dyn 'a + Future<Output = Result<(), io::Error>> + Send>> + Send;

/// Simple interface which provides access to blob storage
/// in different components of the application.
#[rocket::async_trait]
trait BlobStorage2: Sync + Send {
    async fn store_blob<'a, 'b>(&self, writer: Box<dyn WriterFn<'a> + Send + 'b>) -> bool;
}

/// ======================================================================
pub struct Storage;

#[rocket::async_trait]
impl BlobStorage2 for Storage {
    async fn store_blob<'a, 'b>(&self, writer: Box<dyn WriterFn<'a> + Send + 'b>) -> bool {
        writer(PathBuf::from("path-where-to-store")).await;
        true
    }
}

async fn store<'a>(p: &mut TempFile<'_>, i: Arc<dyn 'a + BlobStorage2>) {
    i.store_blob(Box::new(|r: PathBuf| Box::pin(p.copy_to(r)) as _))
        .await;
}

#[derive(rocket::FromForm, Debug)]
struct JobUpload<'r> {
    file: TempFile<'r>,
}

#[rocket::put("/api/job", data = "<job>")]
async fn submit_job(mut job: Form<JobUpload<'_>>) -> rocket::http::Status {
    let p = Arc::new(Storage {});
    store(&mut job.file, p.clone()).await;

    Status::Continue
}

#[rocket::main]
async fn main() -> Result<(), rocket::Error> {
    let r = rocket::build().mount("/", rocket::routes![submit_job]);
    r.launch().await;
    Ok(())
}
``
1 Like

Nice: I cleaned it up more: Removing some unnecessary lifetimes and replacing them with '_:

Here

1 Like