Generic over type and trait object with storage in a HashMap

Hi there,

in my actual use case I'm storing data of different types in a BTreeMap like this:

capabilities: BTreeMap<TypeId, RwLock<Box<dyn Any>>>;

fn add_capability<T>(capabaility: T)
    where
        T: Capability + 'static,
{
  capabilities
    .insert(TypeId::of::<T>(), RwLock::new(Box::new(capabaility)));
}

This is working fine for the required types like so:

struct Foo {}
impl Capability for Foo {}

add_capability(Foo {});

However, I'd like to be able to add some abstraction in form of a trait for Foo and only storing the trait object.When retrieving the data from the BTreeMap I only require access to the trait methods.

So I'd like something like this to work:

trait FooTrait {}
impl FooTrait for Foo {}

add_capabaility::<dyn FooTrait>(Foo {});

This obviously is not working as <dyn Foo> as not Sized. Working around this with add_capabaility::<Box<dyn FooTrait>>(Box::new(Foo {})) works, but requires Capability to be implemented also for Box<dyn FooTrait>. Also I'd like to hide this additional complexity with boxing and additional trait implementation for the Box variant from my API users.

Is there any neat way to achieve this?

Thanks in advance for any hints.

You are boxing in add_capability. Does it help if you make the bound on T like so: T: Capability + ?Sized + 'static,?

As far as I can tell you are stopped by the fact that Sized is a default bound on type parameters.

Potentially it could streamline the API by using your own trait:

trait Capability : std::any::Any {}

// Blanket impl so you users don't have to impl Capability.
//
impl Capability for T

   where T: Any + ?Sized + 'static

{}

This will keep the functionality of Any available but convey the intend of the BTreeMap more clearly.

Now it should be possible to enable other traits by declaring them as:

trait FooTrait : Capability {}

which also makes it more clear that FooTrait is a capability.

One thing that strikes me as odd is specifying the trait when calling add_capability. You are not really gaining anything over just passing Foo. You will still use as much memory and store the same bytes, but you tell the compiler to discard some type information. Even if you don't need it and you only need to use the trait methods, they would be available on on Foo anyways.

I'm not quite clear on how you want to use this, but know that on vanilla rust you can't crosscast traits. That is you cannot cast Any to FooTrait without the help of crates that enable cross casting.

Now you use type_id to index the BTreeMap. That is a bit shady, because now if you call add_capability again with another object and store it as dyn FooTrait again, that will overwrite the last one, which might be unintentional.

Disclaimer: the type system is complex and I just wrote this off the top of my head. Excuse me if you still hit some roadblock.

2 Likes

This problem can be avoided with a blanket impl

impl<T: ?Sized> Capability for Box<T> where T: Capability {
    // ...
}

Then, for a trait FooTrait: Capability you’ll have Box<dyn FooTrait>: Capability automatically.

1 Like

Thanks for your response and hints.

Well the thing is that my target "architecture" shall look like this:

  1. There is a crate that stores and manages the capabilities within the BTreeMap.
  2. A specific capability will be implementet in a another crate beeing able to register/add itself to the capability managing crate. The behaviour of this specific capability will be available through it's specific trait.
  3. Another crate can use the capability managing crate and can request a specific capability based on its trait to access its behaviours. Thus the actual capability is required to be stored with the traits type_id within the BTreeMap as the consumer / user of the capability should not need to care about the actual implementation/structure behind it, but the trait that represents it.

I hope this explanation does make any sense :wink:

This intended and will be handled in the final implementation to add a capability that only ever one dyn FooTrait can exist in the BTreeMap

Storing the dyn FooTrait as Box<dyn FooTrait> works fine and also allows the downcast from Any back to Box<dyn FooTrait> like this:

pub fn with_capability_ref<T, F, R>(&self, f: F) -> Result<R>
    where
        T: Capability + 'static,
        F: FnOnce(&T) -> R 
    {
        let capa = self.capabilities
            .get(&TypeId::of::<T>())
            .ok_or(
                BoxError::from(
                        GenericError::with_message("unable to access requested capability")
                )
            )?;

        Ok(
            f(capa.downcast_ref::<T>().unwrap())
        )
    }

The usage in the consuming crate might look like this:

features.with_capability_ref::<Box<dyn FooTrait>, _, _>(|foo| {
            foo.some_method();
        });

So basically I'd like to be able to omit the Box<dyn FooTrait> notation when calling the with_capability_ref and just use dyn FooTrait as type specifier if this is possible at all.

Thanks for your time and thoughts in advance.

I experimented a bit on the playground. It seems you are right. It's possible to downcast Box<dyn Any> to Box<dyn SomeTrait>. I thought that wasn't possible. So with that, the playground above compiles and works.

The only limit, I didn't think of it before is that you cannot pass unsized parameters, so you have to box Foo. However you can specify the dyn Trait type parameter without mentioning Box. There is no way you're going to be able to pass unsized params in current rust though. I think this is as close as we can get to what you are after.

1 Like

Just to avoid confusion: All you did is convert a Box<Box<dyn SomeTrait>> into a Box<dyn Any> and after putting that one into the map and getting back a reference to it from the map you use downcast_ref to turn a &dyn Any into a Option<&Box<dyn SomeTrait>> again.

The &dyn Any comes from calling downcast_ref on a &Box<dyn Any> via auto-deref, and the &Box<dyn SomeTrait> from unwrapping the Option is then coerced into a &dyn SomeTrait when you return it in your playground.

It is not possible to turn a Box<dyn SomeTrait> into a Box<dyn Any> or downcast a &dyn Any into a Option<&dyn SomeTrait>.

In case that isn’t clear yet, note that your self.capabilities.insert( TypeId::of::<T>(), RwLock::new(Box::new(capability)) ); call with capability: Box<T> has the effect of double-boxing everything, even for the case where T is sized. While this API has the advantage that the user does not need to write Box<dyn ...> in the type parameter of add_capability anymore, this is pretty much the only advantage you archieve over the original T: Sized version. In effect you just special-cased the original T: Sized to the case T == Box<T1> where now T1: ?Sized is of course okay, since Box<T1> will still be sized. On the other hand, with this function the caller now needs to box everything, even non-unsized types, which is a overhead in both performance and usability.

You are absolutely right. I am not saying this is a good design, but I have run into the need for this myself.

I just tried to help OP to see if what they wanted was possible. I did find it interesting to see that, even though it requires double boxing, you can downcast to a trait object which I didn't know was possible. I have a library in which I have a struct that wraps a Box<dyn Trait> in order to use Box<dyn Any> and I will review if I cannot get rid of the type all together. It double boxes as well. It's needed in order to store a heterogeneous collection over a trait that is generic. So the trait hides the concrete implementation, but it still has a type parameter I have to erase in order for it to be possible.

Performance is a very situational thing. In some code the performance cost might be neglegible (btw on my machine heap allocating seems quite negligible as far as I have measured, compared to thread sync for example). Certain code paths don't get invoked very much but maybe having a nicer API is preferable.