HashMap of Vec<async function with a single &Any argument>, key'ed by the argument TypeId

Hi!

I want to create an event dispatcher, to begin, I want to be able to register into it multiple listener for Any events

the handlers should be implemented like this:

async fn (event: &SomeEvent) -> Result<(), ...> { todo!() }

To begin, I'm focusing on storing the handlers, for that, I'm trying to put them inside an HashMap using the Any TypeId for keys, and a Vec<Handler<...>> value. But I'm having issues with the lifetimes

here is my latest attempt : Rust Playground
The compiler is telling me that I could cast it to the correct type but I couldn't find how. I suppose it means I'll have to do some unsafe downcast later on to be able to call those handlers which is not great but I have the feeling I won't be able to avoid it.

I've read a lot of documentation & related questions here and tried many things. But I'm stuck now. :man_shrugging:

Can this approach work? If so, what do I need to change? Or should I implement this differently entirely?


For context, once this part is implemented, I'd like to wrap this into its own Handlers struct, and then implement an EventDispatcher using it, could be done like this:

async fn main() {
  let handlers: Handlers = get_handlers_from_plugins(get_plugins_from_somewhere());
  let event_dispatcher = EventDispatcher::new(handlers);
  
  event_dispatcher.dispatch(&SomeEvent {}).await.unwrap();
  event_dispatcher.dispatch(&AnotherEvent {}).await.unwrap();
}

thank you!

There are a couple layers of problems here, but the fundamental problem is that you're assuming a function that takes some type can be cast to a function pointer that takes an Any. Rust doesn't have that kind of dynamic subtyping.

The compiler error is talking about casts because there's some classic function pointer vs. fn item confusion happening but that's not really relevant to whats wrong with your code.

If you add this function to your code the error might help illustrate the issue better

fn check() {
    let a: Handler<dyn std::any::Any, Result<(), Error>> = Box::new(h_first_1);
}
error[E0308]: mismatched types
   --> src\bin\main.rs:26:69
    |
26  |     let a: Handler<dyn std::any::Any, Result<(), Error>> = Box::new(h_first_1);
    |                                                            -------- ^^^^^^^^^ expected trait object `dyn Any`, found struct `FirstEvent`
    |                                                            |
    |                                                            arguments to this function are incorrect
    |
    = note: expected fn pointer `fn(&dyn Any) -> Pin<Box<dyn futures::Future<Output = Result<(), Error>> + std::marker::Send>>`
                  found fn item `for<'a> fn(&'a FirstEvent) -> impl futures::Future<Output = Result<(), Error>> {h_first_1}`

You have the same problem with the return type of your functions. They all return opaque future types because they're async fns, but you need them to be boxed according to your type signature.


I removed the async bits, and some of the lifetimes to simplify things, but here's a working version that uses type erasure to get around the event type problem

Playground

use std::any::Any;
use std::collections::HashMap;

#[derive(Debug)]
enum Error {
    WrongEventType,
}

/// Each event type will have its own type of EventHandler
type EventHandler<Event, Return> = Box<dyn Fn(&Event) -> Return>;

/// The individual EventHandlers will be wrapped in a Handler which does the work of downcasting the Any to the expected type.
type Handler<Return> = Box<dyn Fn(&dyn Any) -> Return>;

/// Create a closure the takes the typed EventHandlers and converts to the expected event type from the given Any.
fn create_handler<Event: Any>(
    handlers: Vec<EventHandler<Event, Result<(), Error>>>,
) -> Handler<Result<(), Error>> {
    Box::new(move |any| {
        let Some(event) = any.downcast_ref() else {
            return Err(Error::WrongEventType)
        };

        for handler in &handlers {
            handler(event)?
        }

        Ok(())
    })
}

struct FirstEvent;
struct SecondEvent;

fn h_first_1(_event: &FirstEvent) -> Result<(), Error> {
    println!("h_first_1");
    Ok(())
}

fn h_first_2(_event: &FirstEvent) -> Result<(), Error> {
    println!("h_first_2");
    Ok(())
}

fn h_second(_event: &SecondEvent) -> Result<(), Error> {
    println!("h_second");
    Ok(())
}

fn schema_handlers() -> HashMap<std::any::TypeId, Handler<Result<(), Error>>> {
    let mut m = HashMap::<std::any::TypeId, Handler<Result<(), Error>>>::new();

    m.insert(
        std::any::TypeId::of::<FirstEvent>(),
        create_handler(vec![Box::new(h_first_1), Box::new(h_first_2)]),
    );
    m.insert(
        std::any::TypeId::of::<SecondEvent>(),
        create_handler(vec![Box::new(h_second)]),
    );
    m
}

fn main() {
    let handlers = schema_handlers();

    let event = FirstEvent;
    handlers.get(&event.type_id()).unwrap()(&event).unwrap();
}

Also note there's no reason to box function pointers. If you wanted to accept closures too the type should be Box<dyn Fn(Arg) -> Return>. Box<fn(Arg) -> Return> can just be replaced with fn(Arg) -> Return most of the time

1 Like

First, some adjustments in your type alias. You have

type Handler<'a, Event, Return> = Box<fn(&'a Event) -> BoxFuture<'a, Return>>;

But you don't really want this to be lifetime specific, you want it to work for any lifetime. Also there's no point in boxing a fn pointer (maybe you meant dyn Fn?). So from there we get

type Handler<Event, Return> = fn(&Event) -> BoxFuture<'_, Return>;

From there, you're trying to coerce a

fn(&SomethingSpecific) -> impl Future<...> + '_

into a

fn(&dyn Any) -> BoxFuture<'_, ...>; // aka
fn(&dyn Any) -> Pin<Box<dyn Future<...> + '_ + Send>>;

But

  • The later's inputs are more general than the former, so there's no way a coercion could work
    • What happens when someone passes in &() as &dyn Any?
  • Even in the other direction, you can't coerce a fn(&dyn Any) into a fn(&Something), you need another function (or closure) to do the coerceion of the input argument for you
  • And similarly but moreso, Rust won't build some shim function for you that pins and boxes up the return value of a function as a fn coercion

So you need something like

//  let hf1: fn(&dyn std::any::Any) -> BoxFuture<'_, Result<(), Error>> =
    let hf1: Handler<dyn std::any::Any, Result<(), Error>> =
        |evnt| Box::pin(h_first_1(evnt.downcast_ref().unwrap()));

(And take care to only call it when you've tested the &dyn Any's TypeId.)

Playground.


However, I think there's probably a simpler design here. Something like a

// n.b. untested, I'm winging this example
struct Map<Res> {
    inner: HashMap<TypeId, Box<Any>>,
    // the keys are each T's TypeId
    // the values are Vec<Handler<T, Res>>
}

impl<Res> Map<Res> {
    fn handlers<T: ?Sized + Any>() -> Option<&[Handler<T, Res>]> {
        self.inner
            .get(T::type_id())
            .and_then(Any::downcast_ref::<Vec<Handler<T, Res>>>)
            .as_deref()
    }
}

So that your hash lookup and downcasting are encapsulated away in the API.

If that works for your use case, you still need the pin-and-box shims, but not the other downcasting.

I might have overlooked something since I didn't actually try this route -- my main doubts are (a) does it even do what you want but also (b) the opacity of async fn return types make them hard or impossible to work with in generic context, particularly higher-ranked generic contexts, so the pin-and-box shims perhaps can't be hidden in the API (or at least, not with more boiler plate).

1 Like

Nerd sniped myself... probably not perfect but I didn't hit any blockers like I thought I might.

3 Likes

Thank you both!

Nerd sniped myself... probably not perfect but I didn't hit any blockers like I thought I might.

@quinedot this looks really close to what I want !

and:
image

However, I think there's probably a simpler design here. Something like a [...] So that your hash lookup and downcasting are encapsulated away in the API.

Yes that's what I wanted to do next :slight_smile:

I do have something else I don't know how to do, now I'd like to have a struct able to listen to multiple events, something like this:

pub struct CountSomeEvents {
    counter: Arc<RefCell<u8>>,
}

impl HandlerTrait<FirstEvent, ...> for CountSomeEvents {
    handle(&self, _event: &FirstEvent) -> BoxFuture<...> {
        // increment self...counter
        todo!();
    }
}

impl HandlerTrait<SecondEvent, ...> for CountSomeEvents {
    handle(&self, _event: &SecondEvent) -> BoxFuture<...> {
        // increment self...counter
        todo!();
    }
}

fn somewhereelse() {
    let c = CoutSomeEvents::default();

    handlers.register::<FirstEvent, ...>(&c);
    handlers.register::<SecondEvent, ...>(&c);
}

Should I be able to do implement such a HandlerTrait to make this struct easy to implement?

I've tried a few things, eg:

    pub trait HandlerTrait<Event, Return> {
        fn handle(&self, event: &Event) -> BoxFuture<'_, Return>;
    }
    
    impl<Event, Return> Into<Box<Handler<Event, Return>>> for Box<dyn HandlerTrait<Event, Return>> {
        fn into(self) -> Box<Handler<Event, Return>> {
            todo!();
        }
    }

but couldn't figure out what the trait & Into<...> implementation should look like.

And, I was wondering if I could make the Handler a trait instead of a type alias, and have the Map::register() accept an implementation of this trait, I made some progress this way, I can register the handlers, but I can't get them back, downcast_*() always returns None :confused:

This is where I'm at : Rust Playground

I also tried to make the map accept a borrow of the handler to not have to clone it when I want to register a struct that implements multiple handlers but that only added more issues :sweat_smile: I thought it would make the Handlers API nicer to use but...

You weren't consistent in your Sync bounds on your casts. dyn Trait + Send and dyn Trait aren't the same type. If you stored a dyn Trait + Send you have to downcast to dyn Trait + Send if you want the downcast to succeed.

let a: Box<dyn Debug + Send> = Box::new("hello");
let b: Box<dyn Any + Send> = Box::new(a);
println!("{:?}", b.downcast_ref::<Box<dyn Debug>>()); // prints "None"
println!("{:?}", b.downcast_ref::<Box<dyn Debug + Send>>()); // prints "Some("hello")"

I fixed that by just moving the Send bound into Handler itself so you never have to specify it when downcasting, which makes it a bit easier to reason about I think.

You also weren't ever coercing the concrete handler type to a trait object in register. You were storing a Vec<Box<H>> in the Box<dyn Any> and not Vec<Box<dyn Handler<...>>>. That's an easy fix, just make sure you specify what type the boxed handler should have and the compiler will do the coercion.

pub fn register<Event, H>(&mut self, handler: H)
        where
            // Event: ?Sized + Any,
            Event: Any,
            // F: PinFut<In, Output>,
            H: Handler<Event, Output> + 'static,
        {
            let handler: Box<dyn Handler<Event, Output>> = Box::new(handler);
            if let Some(vec) = self.handlers_mut::<Event>() {
                vec.push(handler);
                dbg!("added");
            } else {
                let id = TypeId::of::<Event>();
                self.hm.insert(id, Box::new(vec![handler]));
                dbg!("first");
            }
        }

It can't do the coercion automatically because you're putting the box directly into the hashmap, which has a value type of Box<dyn Any + Send> so there's no way for it to know it should do a coercion in between the concrete type and the completely erased type.

Here's the updated playground

1 Like

This is awesome! thank you @semicoleon

It was my guess that the downcast type was wrong but couldn't figure out what it should have been. And I didn't notice the wrong coercion... It not something trivial for me. hope I'll learn to spot those issues with time :sweat_smile: