HashMap of async function pointers

Hi,

I'm trying to compile the following minimal example and have trouble understanding the error message, can anyone please help?

Playground link: Rust Playground

use std::collections::HashMap;
use std::future::Future;
use std::io::Result;

pub struct Handlers{}
impl Handlers{
    pub async fn get(input: &str) -> Result<()>{
        println!("{}",input);
        Ok(())
    }
}

pub fn schema_handlers<Fut>(
    key: &str,
    get_handler: impl Fn(&str) -> Fut,
) -> HashMap<&str, Box<dyn Fn(&str) -> Fut>>
where
    Fut: Future<Output = Result<()>>,
{
    let mut m = HashMap::new();
    m.insert("http", Box::new(Handlers::get));
    m
}

fn main() {}

The error message is:

   Compiling playground v0.0.1 (/playground)
error[E0308]: mismatched types
  --> src/main.rs:22:5
   |
16 | ) -> HashMap<&str, Box<dyn Fn(&str) -> Fut>>
   |      --------------------------------------- expected `HashMap<&str, Box<(dyn for<'r> Fn(&'r str) -> Fut + 'static)>>` because of return type
...
22 |     m
   |     ^ expected trait object `dyn Fn`, found fn item
   |
   = note: expected struct `HashMap<&str, Box<(dyn for<'r> Fn(&'r str) -> Fut + 'static)>>`
              found struct `HashMap<&str, Box<for<'_> fn(&str) -> impl Future<Output = Result<(), std::io::Error>> {Handlers::get}>>`

For more information about this error, try `rustc --explain E0308`.
error: could not compile `playground` due to previous error

My understanding is that a Fn trait is expected, but I provided a fn pointer.
Yet the fn implements the trait because of its signature, no?

For putting async fn into a data structure, you usually have to resort to using something like BoxFuture as the return type.

use futures::future::BoxFuture;
use futures::FutureExt;

use std::collections::HashMap;
use std::io;

pub struct Handlers {}
impl Handlers {
    pub async fn get(input: &str) -> io::Result<()> {
        println!("{}", input);
        Ok(())
    }
}

pub fn schema_handlers(key: &str) -> HashMap<&str, Box<dyn Fn(&str) -> BoxFuture<io::Result<()>>>> {
    let mut m = HashMap::<&str, Box<dyn Fn(&str) -> BoxFuture<io::Result<()>>>>::new();
    m.insert(key, Box::new(|input| (Handlers::get(input)).boxed()));
    m
}
2 Likes

Some further explanation: The error you encountered is because the function wasn't coerced soon enough. The coercion of the Box to a boxed trait object has to happen at a moment when the Box isn't already wrapped in some other type (the HashMap).

You needed to either explicitly insert a coercion of the Box (Box::new(Handlers.get) as _) or to tell the compiler that the HashMap contains Box<dyn Fn>s, not Box<{Handlers::get}>s, but you left the type of the HashMap unspecified.

Separately from that, <Fut> would not have worked — every async function has its own accompanying Future type, so the future needs to be boxed just like the function needs to be boxed, as @steffahn showed. In general, the <F, Fut> where F: Fn() -> Fut pattern only works when you're dealing with one async function being passed in.

3 Likes

Thank you all, that's a very nice explanation!

Follow-up:
I'm trying to take the outcome of the minimal example and get it into my usecase.

Here's my code:

type BoxedHandlerFut = Box<dyn Future<Output = Result<(), ValidateError>>>;
type GetHandler<'a, Return> =
    Box<dyn Fn(&'a str, &'a str, &'a mut WrappedBar, &'a str) -> BoxFuture<'a, Return>>;
type PutHandler<Return> = Box<dyn Fn(&str, &str, &WrappedBar) -> Return>;

pub fn schema_handlers<'a, Fut>() -> HashMap<&'a str, GetHandler<'a, BoxedHandlerFut>> {
    let mut m = HashMap::<&str, GetHandler<BoxedHandlerFut>>::new();
    m.insert(
        "http",
        Box::new(|a: &_, b: &_, c: &mut _, d: &_| {
            (crate::https::HTTPSHandler::get(a, b, c, d)).boxed()
        }),
    );
    m
}

Which produces the following compiler errors:

error: future cannot be sent between threads safely
   --> src/driver.rs:25:59
    |
25  |             (crate::https::HTTPSHandler::get(a, b, c, d)).boxed()
    |                                                           ^^^^^ future returned by `get` is not `Send`
    |
    = help: the trait `std::marker::Send` is not implemented for `dyn std::io::Write`
note: future is not `Send` as this value is used across an await
   --> src/https.rs:84:20
    |
74  |           let (mut out, mut downloaded) = io::get_output(output, bar.silent);
    |                ------- has type `Box<dyn std::io::Write>` which is not `Send`
...
84  |               .send()
    |  ____________________^
85  | |             .await
    | |__________________^ await occurs here, with `mut out` maybe used later
...
105 |       }
    |       - `mut out` is later dropped here
note: required by a bound in `futures_util::FutureExt::boxed`
   --> /home/mihai/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/future/future/mod.rs:520:23
    |
520 |         Self: Sized + Send + 'a,
    |                       ^^^^ required by this bound in `futures_util::FutureExt::boxed`

error[E0308]: mismatched types
  --> src/driver.rs:25:13
   |
25 |             (crate::https::HTTPSHandler::get(a, b, c, d)).boxed()
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `Box`, found enum `Result`
   |
   = note: expected struct `Pin<Box<dyn futures_util::Future<Output = Box<(dyn futures_util::Future<Output = Result<(), ValidateError>> + 'static)>> + std::marker::Send>>`
              found struct `Pin<Box<dyn futures_util::Future<Output = Result<(), ValidateError>> + std::marker::Send>>`

How can I go around this?
Seems FutureExt::boxed requires the Send trait be implemented but as you can see in the error output, I'm using Box<dyn std::io::Write> internally in HTTPSHandler::get() and that thing is not Send..

Any time you have a dyn, you need to specify exactly what traits it should visibly have. Box<dyn Write> isn't Send but Box<dyn Write + Send> is.

1 Like

Thank you for you patience.
I think we're almost there:

type BoxedHandlerFut<'a> = Box<dyn Future<Output = Result<(), ValidateError>> + 'a>;
type GetHandler<'a, Return> =
    Box<dyn Fn(&'a str, &'a str, &'a mut WrappedBar, &'a str) -> BoxFuture<'a, Return>>;

pub fn schema_handlers<'a, Fut>() -> HashMap<&'a str, GetHandler<'a, BoxedHandlerFut<'a>>>
where
    Fut: Future<Output = Result<(), ValidateError>> + 'a,
{
    let mut m = HashMap::<&str, GetHandler<BoxedHandlerFut>>::new();
    m.insert(
        "http",
        Box::new(move |a: &_, b: &_, c: &mut _, d: &_| {
            Box::new(crate::https::HTTPSHandler::get(a, b, c, d)).boxed()
        }),
    );
    m
}

The types don't match:

28 |             Box::new(crate::https::HTTPSHandler::get(a, b, c, d)).boxed()
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `Box`, found enum `Result`
   |
   = note: expected struct `Pin<Box<dyn futures_util::Future<Output = Box<dyn futures_util::Future<Output = Result<(), ValidateError>>>> + std::marker::Send>>`
              found struct `Pin<Box<dyn futures_util::Future<Output = Result<(), ValidateError>> + std::marker::Send>>`

I probably need a second .boxed() somehow?

You have a boxed future that returns a boxed future. Try this instead: GetHandler<'a, Result<(), ValidateError>>

I understood that BoxedFuture is necessary if you need to store different handlers into your collection since they all return different rust types. I tried simplifying the templating like you suggested, Alice, no luck.

It works though.

Your understanding is correct. However, you only need one box. Your code has a box within a box.

Ok, here's what I ended up with:

type GetHandler<'a> = dyn Fn(
    &'a str,
    &'a str,
    &'a mut WrappedBar,
    &'a str,
) -> BoxFuture<'a, dyn Future<Output = Result<(), ValidateError>>>;

pub fn schema_handlers<'a, Fut>() -> HashMap<&'a str, GetHandler<'a>>
where
    Fut: Future<Output = Result<(), ValidateError>> + 'a,
{
    let mut m = HashMap::new();

    m.insert(
        "http",
        Box::new(move |a: &_, b: &_, c: &mut _, d: &_| {
            crate::https::HTTPSHandler::get(a, b, c, d).boxed()
        }),
    );
    m
}

Output:

error[E0277]: the size for values of type `(dyn Fn(&'a str, &'a str, &'a mut WrappedBar, &'a str) -> Pin<Box<(dyn futures_util::Future<Output = (dyn futures_util::Future<Output = Result<(), ValidateError>> + 'static)> + std::marker::Send + 'a)>> + 'static)` cannot be known at compilation time
   --> src/driver.rs:21:38
    |
21  | pub fn schema_handlers<'a, Fut>() -> HashMap<&'a str, GetHandler<'a>>
    |                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
    |
    = help: the trait `Sized` is not implemented for `(dyn Fn(&'a str, &'a str, &'a mut WrappedBar, &'a str) -> Pin<Box<(dyn futures_util::Future<Output = (dyn futures_util::Future<Output = Result<(), ValidateError>> + 'static)> + std::marker::Send + 'a)>> + 'static)`
note: required by a bound in `HashMap`
   --> /home/mihai/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/collections/hash/map.rs:209:23
    |
209 | pub struct HashMap<K, V, S = RandomState> {
    |                       ^ required by this bound in `HashMap`

Ok, here's what I have with this variant:

type BoxedHandlerFut = Result<(), ValidateError>;
type GetHandler<'a, Return> =
    Box<dyn Fn(&'a str, &'a str, &'a mut WrappedBar, &'a str) -> BoxFuture<'a, Return>>;

fn schema_handlers<'a, Fut>() -> HashMap<&'a str, GetHandler<'a, BoxedHandlerFut>>
where
    Fut: Future<Output = Result<(), ValidateError>> + 'a,
{
    let mut m = HashMap::new();

    m.insert(
        "http",
        Box::new(move |a: &_, b: &_, c: &mut _, d: &_| {
            crate::https::HTTPSHandler::get(a, b, c, d).boxed()
        }),
    );
    m
}

Output:

error[E0308]: mismatched types
  --> src/driver.rs:40:5
   |
28 |   fn schema_handlers<'a, Fut>() -> HashMap<&'a str, GetHandler<'a, BoxedHandlerFut>>
   |                                    ------------------------------------------------- expected `HashMap<&'a str, Box<(dyn Fn(&'a str, &'a str, &'a mut WrappedBar, &'a str) -> Pin<Box<(dyn futures_util::Future<Output = Result<(), ValidateError>> + std::marker::Send + 'a)>> + 'static)>>` because of return type
...
36 |           Box::new(move |a: &_, b: &_, c: &mut _, d: &_| {
   |  __________________-
37 | |             crate::https::HTTPSHandler::get(a, b, c, d).boxed()
38 | |         }),
   | |_________- the found closure
39 |       );
40 |       m
   |       ^ expected trait object `dyn Fn`, found closure

Sorry, I forgot to specify the type for the resulting HashMap. Comibining your solution and input from Alice, I got it running! Thank you manifold!

Here's the final form:

type BoxedHandlerFut = Result<(), ValidateError>;
type GetHandler<'a, Return> =
    Box<dyn Fn(&'a str, &'a str, &'a mut WrappedBar, &'a str) -> BoxFuture<'a, Return>>;

fn schema_handlers<'a, Fut>() -> HashMap<&'a str, GetHandler<'a, BoxedHandlerFut>>
where
    Fut: Future<Output = BoxedHandlerFut> + 'a,
{
    let mut m = HashMap::<&str, GetHandler<BoxedHandlerFut>>::new();

    m.insert(
        "http",
        Box::new(move |a: &_, b: &_, c: &mut _, d: &_| {
            crate::https::HTTPSHandler::get(a, b, c, d).boxed()
        }),
    );
    m
}

I typed my previous suggestions on a phone, and it was evidently not clear what I meant. My original suggestion was this:

-type BoxedHandlerFut<'a> = Box<dyn Future<Output = Result<(), ValidateError>> + 'a>;
 type GetHandler<'a, Return> =
     Box<dyn Fn(&'a str, &'a str, &'a mut WrappedBar, &'a str) -> BoxFuture<'a, Return>>;
 
-pub fn schema_handlers<'a, Fut>() -> HashMap<&'a str, GetHandler<'a, BoxedHandlerFut<'a>>>
+pub fn schema_handlers<'a, Fut>() -> HashMap<&'a str, GetHandler<'a, Result<(), ValidateError>>
 where
     Fut: Future<Output = Result<(), ValidateError>> + 'a,
 {
     let mut m = HashMap::<&str, GetHandler<BoxedHandlerFut>>::new();
     m.insert(
         "http",
         Box::new(move |a: &_, b: &_, c: &mut _, d: &_| {
             Box::new(crate::https::HTTPSHandler::get(a, b, c, d)).boxed()
         }),
     );
     m
 }

Your last snippet "works", but its a bit weird to have a type alias called BoxedSomethingFut that isn't a boxed future.

That said, it seems like you should be doing this instead:

type GetHandler<Return> =
    for<'a> Box<dyn Fn(&'a str, &'a str, &'a mut WrappedBar, &'a str) -> BoxFuture<'a, Return>>;

pub fn schema_handlers() -> HashMap<&'static str, GetHandler<Result<(), ValidateError>> {
    let mut m = HashMap::new();
    m.insert(
        "http",
        Box::new(move |a: &_, b: &_, c: &mut _, d: &_| {
            Box::new(crate::https::HTTPSHandler::get(a, b, c, d)).boxed()
        }),
    );
    m
}

Your current version will cause lifetime issues when you attempt to call it.