Upcasting support for arbitrary types implementing a trait: Any vs MOPA


#1

I am porting a C++ half-edge data structure library which stores vertex, edge, and face properties in parallel arrays. It allows the user to attach arrays of user-defined properties, thus allowing it to be extensible with arbitrary data. That does mean that I don’t have a finite enumeration of vector types a priori.

Effectively, the graph needs to store a container with arrays of some user-specified type and allow the user to down-cast to the container of the user-specified type. The Any type seems perfect for this use case and it would store, eg, Vec Box<Container>> where trait Container: Any {...} and all the containers of user-specified types would implement Container (eg. impl Container for Vec<MyType>).

So here’s my failed attempt at testing this:

use std::any::Any;

trait Container: Any {}

struct Bar(u32);
impl Container for Bar {}

struct Baz(f32);
impl Container for Baz {}

fn exec(b: &Box<Container>) {
    let b_any = b as &Any;
    if let Some(container) = b_any.downcast_ref::<Bar>() {
        println!("u32: {}", container.0);
    } else if let Some(container) = b_any.downcast_ref::<Baz>() {
        println!("f32: {}", container.0);
    } else {
        println!("Unknown");
    }
}

fn main() {
    let mut vec: Vec<Box<Container>> = Vec::new();
    vec.push(Box::new(Bar(42)));
    vec.push(Box::new(Baz(1.0f32/0.0)));
    for val in vec.iter() {
        exec(val);
    }
}

Playground
The output is “Unknown” twice, which was unexpected.

But then I also ran into My Own Personal Any (MOPA) which is aiming for precisely my use-case but seems to solve a problem with Any that I don’t understand. For example, it mentions Box<Person + Any>, but I would be doing trait Person: Any and just using Box<Person>, much like what it does with its MOPA type and what I am doing above.

So, two questions:

  1. What’s wrong with my code.
  2. What’s MOPA actually trying to solve? (I can see it adds convenience functions to one’s trait using a macro – that part’s fine; but it defines its own Any for “technical reasons” and I don’t see why I can’t just do the same thing with a regular Any on my Container type above.)

How to downcast from trait object
#2

There are two problems:

  1. Your exec takes a &Box<Container>, so the original type is already forgotten. It’s only data and a bunch of method implementations now.
  2. Casting it to &Any will only allow you to downcast it to &Box<Container>, since that’s what you had.

Just depending on Any doesn’t give you the ability to downcast. You will have to implement your on downcast_ref for Container trait objects as well (notice the impl Any + 'static section), and this is what MOPA gives you. It does the tedious part for you.


#3

Conceptually, I thought this whole thing shouldn’t need any more boilerplate because:

  1. Container: Any means Box<Container> has access to get_type_id() in the vtable pointed to by its fat pointer. In other words, it bypasses type erasure – or rather, has decorated the trait object with type information.
  2. Thus I would imagine that downcast_ref() should somehow be able to be made automatically available for traits extending Any.

Could you offer more insights as to why downcast_ref() implementation doesn’t automatically exist for traits objects? Is it entirely because of the static constraint on Any and the impls? Box<T> doesn’t have any lifetime parameters, so it’s not like there is a lifetime check hazard.


#4

I think the main problem that causes this situation is that a generic trait method can’t be called on a trait object, since the compiler won’t know which version of the method it should call. This, in turn, forces us to implement downcast_ref<T>() on the trait object itself, but then it won’t be inherited by other traits, since they are different types (Container != Any).

Note that I’m not an expert on the inner workings of the compiler and I can’t say to what extent you can recover type information, but the type ID is just a number. You can’t make out anything useful from it, so it’s practically useless, unless you have something to compare it to. Your example has two layers of abstraction: Box<Bar/Baz> -> Box<Container> -> Any. Any can only “see” one layer down, since it has only the type ID from what it was before, which was a Box<Container>, but it doesn’t know that. It can only know that it had a certain ID, which isn’t any of the Baz or Bar IDs, so that won’t work.

Now, let’s say you didn’t cast the Box<Container> to Any. Then you would have the problem that MOPA solves, which is caused by the lack of trait object method inheritance. Traits objects are basically types that happen to have the same name as a trait and implements forwards the trait implementations from their original types. Kind of. The point is that a method that is implemented on a trait object isn’t a part of the trait with the same name. It’s not really intuitive, straight away, unless one keeps this in mind.


#5

Unless I am fundamentally mistaken (please correct me if I am), downcast_ref() should not be in the vtable – it’s not part of the trait definition. Only get_type_id() should be there, and it’s not generic. downcast_ref is only defined in the trait impl, so it seems like a free-form function with some convenient scoping.

The type ID is always enough to attempt to convert a trait object to a concrete type that you provide – one just checks against the type id of the concrete type. Here’s my example modified which demonstrates this:

Output

main():
  Bar|Baz
    (bar, baz): (TypeId { t: 2838179725187358675 }, TypeId { t: 4474829220185994047 })
  Box<Bar|Baz>
    (bar_box, baz_box): (TypeId { t: 15740178146275938547 }, TypeId { t: 12065486722540755553 })
    (*bar_box, *baz_box): (TypeId { t: 2838179725187358675 }, TypeId { t: 4474829220185994047 })
  Box<Container>
    (bar_boxc, baz_boxc): (TypeId { t: 6329136517252537689 }, TypeId { t: 6329136517252537689 })
    (*bar_boxc, *baz_boxc): (TypeId { t: 2838179725187358675 }, TypeId { t: 4474829220185994047 })
exec():
  Unknown with type id TypeId { t: 6329136517252537689 }
  Unknown with type id TypeId { t: 6329136517252537689 }

It demonstrates that one can get the Bar and Baz type ids out of a Box<Container> (see the main() function and the output under “Box<Container>”). Turns out, I need to dereference the Box to get the correct type id! But I tried to do the same within exec() and it didn’t quite work. For example, I tried let b_any = b as &&Any; and separately tried (**b_any).downcast_ref::<Bar>() but received compile errors; can someone help me with this? The desired of get_type_id() should be in the trait object’s vtable – I just need to know how to get it out.


#6

I think that it is more idiomatic to use enums instead of the Any trait.

https://play.rust-lang.org/?gist=8b79ed500a1b919462e0&version=nightly


#7

I think you are missing the point a bit. What I’m trying to explain is that you are hiding the original type by casting &Box<Container> to &Any in b as &Any. You can see from your own example that the ID in b_any is the ID of Box<Container>, because the vtable and data pointer is for the Box<Container> and not whatever it contains. What you have is a trait object, pointing to a trait object, pointing to Bar or Baz.

Take a look at this modified example. I have just added a check for Box<Container> and that’s what it finds.


#8

Enums only work for finite enumerations, which unfortunately don’t work in the use-case in my first post where not all enumerations are known.

Really sorry, I didn’t parse your sentence properly. However, I did at that time understand the point you are making, hence, what I said about me trying to dereference it properly with
let b_any = b as &&Any and (**b_any).downcast_ref::<Bar>(), but those gave me compiler errors. I also just tried let b_any = b.as_ref() as &(Container+'static) as &Any; which is perhaps cleaner, but of course doesn’t circumvent the error which was

error: non-scalar cast: `&Container + 'static` as `&core::any::Any + 'static`

I have so far not been able to get past this, and I was wondering whether you or anyone can help me cast it properly to access the right vtable/Any.


#9

We may also have misunderstood each other. Anyway, here comes the second part of the point. I don’t know of any safe way to cast Container to Any, even if Container: Any. My best guess is that we don’t know which vtable it should have if it was an Any, since all the type information, except the Container vtable is gone. We can, however, go the MOPA way and make Container work as an Any.


#10

Thanks for your patient answers. This is going against my mental model of how traits work and partially inconsistent with the examples – to your point, I am not able to invoke any Any-related methods using a &Box<Container> in exec(). But to my point, I am able to invoke Any-related methods within main() on a Box<Container>. So are the Any methods in the Container vtable or not? Since this is something more generic than Any, I’ll open a separate thread without this specific context.


Methods implemented on Base trait not carry over to Derived trait object
#11

Ha! Finally figured it out! I was partially comparing apples and oranges. The summary of what I learned is below – seems to lead to a possible bug or a lack of a feature in the compiler. The following works:

#![feature(get_type_id)]
use std::any::Any;

trait Derived: Any {}

struct Type;
impl Derived for Type {}

fn exec(b: &Box<Derived>) {
    println!("exec(): b.get_type_id() = {:?}", (**b).get_type_id());
    // cannot invoke Any::downcast_ref::<Type>() implemented on all `Any` on `Derived`
}

fn main() {
    let b = Type;
    println!("main(): b.get_type_id() = {:?}", b.get_type_id());
    let b_box: Box<Derived> = Box::new(b);
    println!("main(): b_box.get_type_id() = {:?}", (*b_box).get_type_id());
    exec(&b_box);
}

Playground

In other words, I can access get_type_id() within exec(). What I cannot do is call the Any::downcast_ref() which is implemented on the trait (though is not part of the trait definition). Somehow, the compiler seems to ignore that Derived includes Any and it seems to disallow any methods implemented on Any trait objects from being invoked by Derived trait objects. There’s no obvious reason I can see why it shouldn’t. Will open a separate thread. While this is less important for Any due to MOPA, I think it’s generally important for using trait objects of traits that extend other ones to inherit their abilities.

Yay!


#12

Nice to see that the pieces fell into place, at last :smile: The reason why downcast_ref isn’t available through Container is that traits and trait objects are two different things, as I mentioned before. Trait objects are unsized types with the same name as their corresponding traits, so it’s quite logical, in that sense, that the implementations on one trait object aren’t applied to an other. The problem I can see with letting implementations from derived traits to be carried over is that there may be naming collisions:

trait A;

impl A {
    fn foo() {...}
}

trait B;

impl B {
    fn foo() {...}
}

trait C: A + B; //Which `foo` should be applied to C?

#13

Aaaand better news. :smiley:
Given that Any and the standard library methods implemented on it are not modifiable, here is a variation based on @stebalien’s trick in the related post that allows one to cast to Any by including a conversion function in the subtrait vtable. Then all the helper methods are available.

use std::any::Any;

// AsAny trait + impl.
trait AsAny {
    fn as_any(&self) -> &Any;
}
impl<T: Any> AsAny for T {
    fn as_any(&self) -> &Any { self }
}

// Derived trait should include AsAny so that `as_any` is in its vtable.
trait Derived: Any + AsAny {}

// Concrete type implementing Derived.
struct Type(u32);
impl Derived for Type {}

// And now one can access `downcast_ref()` with the trait object though
// it still requires awkward double-dereferencing.
fn exec(b: &Box<Derived>) {
    if let Some(derived) = (**b).as_any().downcast_ref::<Type>() {
        println!("Type({})", derived.0);
    }
}

fn main() {
    let b_box: Box<Derived> = Box::new(Type(42));
    exec(&b_box);
}

Playground

The main reason I mention this is that with this trick, MOPA can dramatically simplify its implementation to use the standard library implementation of downcast_ref() and downcast_mut() instead of re-implementing its own from scratch using unsafe code and intermediate traits/objects. That would mean, though, that the *_unchecked variants would not be included since they are not in the standard library.


#15

Nice! That trick should be enough in almost all cases.