Why does downcasting not work for subtraits?

I am trying to create some sort of ECS as a learning for myself. I store Box<dyn Any> in a HashMap and downcast them when needed.

In short:

#[derive(Debug)]
struct Foo {
    bar: u8
}

fn main() {
    let foo = Box::new(Foo { bar: 0 }) as Box<dyn Any>;
    let x = foo.downcast_ref::<Foo>();

    println!("{:?}", foo);
    println!("{:?}", x);
}

// Output:
// foo: Any
// x: Some(Foo { bar: 0 })

Now, I want to add some standard behavior to my items, like debugging (so when debugging I don't see Any and serialization, so decided to make a subtrait:

trait Component : Any + Debug {}

impl Component for Foo {}

And when I try this, I get dynamically the correct Debug behavior:

let foo = Box::new(Foo { bar: 0 }) as Box<dyn Component>;
println!("foo: {:?}", foo);

// Output:
// foo: Foo { bar: 0 }

So far so good, but when I want to downcast I get this:

let x = foo.downcast_ref::<Foo>();

// ERROR: no method named `downcast_ref` found for type `std::boxed::Box<dyn Component>` in the current scope

What am I missing? The trait Component is a subtrait of Any and as such should have the downcast_ref method right?

You have to be explicit about which method you want. You can do this like this:

let x = Any::downcast_ref::<Foo>(&foo);

This happens when dealing with things in an impl dyn Any block, because they only operate on things that are explicitly an Any reference.

Thanks, I tried it and it compiles, however the output is None

let foo = Box::new(Foo { bar: 0 }) as Box<dyn Component>;
let x = Any::downcast_ref::<Foo>(&foo);
println!("x: {:?}", x);

// Output: x: None

This has to do with the fact that it can not by downcasted I think, or the type is wrong.

When trying:

let foo = Box::new(Foo { bar: 0 }) as Box<dyn Component>;
let unboxed = foo.as_ref();
let x = Any::downcast_ref::<Foo>(unboxed);
println!("x: {:?}", x);

I get:

   |
17 |     let x = Any::downcast_ref::<Foo>(unboxed);
   |                                      ^^^^^^^ expected trait `std::any::Any`, found trait `Component`
   |
   = note: expected type `&(dyn std::any::Any + 'static)`
              found type `&dyn Component`

But I don't understand. Component is a subtrait of Any.

Subtrait relationship isn't special in trait object. A trait object in Rust is simply a pair of a pointer to the concrete type and a pointer to the static table that contains function pointers of methods declared in the trait definition.

You may ask why those vtables don't contains subtrait's methods. Such inclusion will increase the size of vtable, but we don't see a net gain for it. Subtrait relationship is not a subtyping, it's more like a implementation dependency.

1 Like

Okay thanks, I understand now.

But then I do not know if my desired behavior is actually possible?

I want to store and retrieve arbitrary types which have certain requirements enforced by traits. I would say that it is not an uncommon use case when something like a Command pattern is used, of in my case an Entity Component architecture.

https://github.com/rust-lang/rust/pull/60900 would enable coercing Box<dyn Subtrait> to Box<dyn Any>, on which you can downcast to the original type.

For now, one way to achive this is to add a method to return &dyn Any: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=1d0dad7cedeae00568c280816318063d

2 Likes

Actually a dyn trait vtable does include the subtrait methods.

For instance:

use ::core::{any::Any, fmt::Debug};

trait MyAny : Any + 'static {
    /// trait method (!= an inherent method of dyn Any)
    fn downcast_ref_Foo (self: &'_ Self)
      -> Option<&'_ Foo>
    ;
}
impl MyAny for Foo
{
    fn downcast_ref_Foo (self: &'_ Self)
      -> Option<&'_ Foo>
    {
        Some(self)
    }
}

#[derive(Debug)]
struct Foo {
    bar: u8
}

trait Component : MyAny + Debug {}

impl Component for Foo {}

fn main ()
{
    let foo: Box<dyn Component> = Box::new(Foo { bar: 0 });
    let x: Option<&Foo> = foo.downcast_ref_Foo();

    println!("{:?}", foo);
    println!("{:?}", x);
}

works.


The issue lies with .downcast_ref::<T>() being an associated method / function of dyn Any + 'static (and dyn Any + Send + 'static as well as dyn Any + Send + Sync + 'static).

The main reason for that being that downcast_ref is generic and thus requires a static dispatch, which requires an arbitrary number of monomorphisations, which is obviously impossible to store within a fixed-size (v)table of function pointers.

Hence the solution suggested by @sinkuu, which can be made slightly more ergonomic by also adding ourselves inherent methods to dyn Component + 'static:

impl dyn Component + 'static {
    #[inline]
    fn downcast_ref<T : 'static> (self: &'_ Self)
      -> Option<&'_ T>
    {
        self.upcast_Any_ref().downcast_ref::<T>()
    }

    #[inline]
    fn downcast_mut<T : 'static> (self: &'_ mut Self)
      -> Option<&'_ mut T>
    {
        self.upcast_Any_mut().downcast_mut::<T>()
    }
}
3 Likes

Amazing everyone. Thanks for the help! I don't fully understand it though, but that means I have to dig deeper into understanding Rust.

Looking at the solution of @sinkuu, I could also make it more ergonomic by creating my own Component derive macro. As an exercise :wink:

So, I combined the solution of @sinkuu with a custom derive macro, something I haven't done before. The reason I did it is that I needed Serialization and by adding a Serialize type bound it was impossible to create an object, since trait objects don't allow generic type parameters.

I combined the typetag crate with custom derive macro:

#[proc_macro_derive(Component)]
pub fn derive_component(input: TokenStream) -> TokenStream {
    let ast = syn::parse(input).unwrap();
    impl_derive_component(&ast)
}

fn impl_derive_component(ast: &syn::DeriveInput) -> TokenStream {
    let name = &ast.ident;
    let gen = quote! {
        #[typetag::serde]
        impl ecs::Component for #name {
            fn as_any(&self) -> &dyn Any {
                self as &dyn Any
            }

            fn as_mut_any(&mut self) -> &mut dyn Any {
                self as &mut dyn Any
            }
        }
    };
    gen.into()
}

and this leads to the following usage:

#[derive(Debug, Serialize, Deserialize, Component)]
struct Foo {
    bar: u8
}

#[derive(Debug, Serialize, Deserialize, Component)]
struct Bar {
    foo: u8
}

fn main() {
    let foo = Foo { bar: 42 };
    let bar = Bar { foo: 22 };

    let foo_boxed: Box<dyn Component> = Box::new(foo);
    let bar_boxed: Box<dyn Component> = Box::new(bar);

    println!("{:?}", &foo_boxed);
    println!("{:?}", &bar_boxed);

    println!("{}", serde_json::to_string(&foo_boxed).unwrap());
    println!("{}", serde_json::to_string(&bar_boxed).unwrap());

    println!("{}", serde_yaml::to_string(&foo_boxed).unwrap());
    println!("{}", serde_yaml::to_string(&bar_boxed).unwrap());
}

Output:

Foo { bar: 42 }
Bar { foo: 22 }
{"type":"Foo","bar":42}
{"type":"Bar","foo":22}
---
type: Foo
bar: 42
---
type: Bar
foo: 22

Pretty happy with the result!

1 Like

Nice!!

A minor improvement, in case you want to support generic types (which ::typetage::serialize does, whereas ::typetag::deserialize does not), is to add the generic parameters to the impl (your macro currently ignores them):

extern crate proc_macro;
use ::proc_macro::TokenStream;
use ::quote::quote;
use ::syn::{
    DeriveInput,
    parse_macro_input,
};

#[proc_macro_derive(Component)] pub
fn impl_derive_component (
    input: TokenStream
) -> TokenStream
{
    let DeriveInput { ident, generics, .. } =
        parse_macro_input!(input)
    ;
    let krate = quote! {
        ::crate_name // the name of your frontend library crate
    };
    #[allow(bad_style)]
    let Any = quote! {
        // #krate /* if reexport of core: `#[doc(hidden)] pub use ::core;` in frontend crate */
        ::core::any::Any
    };
    let (impl_generics, ty_generics, where_clauses) =
        generics.split_for_impl()
    ;
    TokenStream::from(quote! {
        #[/* #krate:: */::typetag::serde] // if `#[doc(hidden)] pub use ::typetag;` in frontend crate
        impl #impl_generics #krate::ecs::Component
            for #name #ty_generics
        #where_clauses
        {
            #[inline]
            fn as_any (self: &'_ Self) -> &'_ dyn #Any
            {
                self
            }

            #[inline]
            fn as_mut_any (self: &'_ mut Self) -> &'_ mut dyn #Any
            {
                self
            }
        }
    })
}
2 Likes