Downcast to Box<Trait>


#1

Hi,

to following code does not make any sense but it is a short example for my question (and it is unsafe but it is possible to add type checks to make it safe):

use std::any::*;

#[repr(C)]
#[allow(raw_pointer_derive)]
#[derive(Copy, Clone)]
pub struct TraitObject {
    pub data: *mut (),
    pub vtable: *mut (),
}

trait AnyData : Any {  }

impl AnyData {
    fn downcast<T>(self: Box<Self>) -> Box<T> {
        unsafe {
            let raw = Box::into_raw(self);
            let trait_object: TraitObject = ::std::mem::transmute(raw);
            Box::from_raw(::std::mem::transmute(trait_object.data))
        }
    }
}

trait Data : AnyData {
    fn get_data(&self) -> String;
    fn do_with_data(&self) {
        println!("do_with_data {}", self.get_data());
    }
}

struct Some;
impl Some {  }
impl AnyData for Some {  }
impl Data for Some {
    fn get_data(&self) -> String {
        String::from("some")
    }
}

fn main() {
    let a : Box<AnyData> = Box::new(Some);
    let a : Box<Data> = a.downcast::<Some>();
    a.do_with_data(); 
}         

What I would like to do is to cast directly to Box<Data> instead doing a downcast to Box<Some> followed by a upcast. e.g. somthing like this:

//...

impl AnyData {
    fn downcast_to_data(self: Box<Self>) -> Box<Data> {
        ...
    }  
}

//...

fn main() {
    let a : Box<AnyData> = Box::new(Some);
    let a : Box<Data> = a.downcast_to_data();
    a.do_with_data(); 
}    

Has anyone an idea how to do this?


#2

I think downcasting to a trait object can’t be done in general. Here’s why:

When you have a Box<Any>, it could be referring to an instance of literally any concrete type from any crate in the program. If we want to downcast to some specific trait object type, Box<Trait>, we need to decide whether our concrete type implements Trait, by consulting some sort of run-time metadata that Box<Any> carries about its concrete type. (Right now, it’s a vtable; but we’re trying to imagine all possible solutions, so imagine the metadata could be anything we want.)

The problem is that I can implement my own traits on any type at all. I can even provide implementations that apply to some open-ended set of types, like:

trait MyReaderMethods {
    fn parse_quoted_string(&mut self) -> std::io::Result<String>;
}

impl<R: BufReader> MyReaderMethods for R {
    ...
}

With these definitions in scope, any type that implements BufReader now also implements MyReaderMethods, even if the crate in which the type is defined has never heard of MyReaderMethods.

So, when we first produce a Box<Any> by conversion from a Box<T> for some concrete type T, it’s not possible at that point to construct a vtable which covers all the traits that T might implement. You’d have to scan all the trait implementations in the program, even in crates not yet linked to, to be sure to mention them in the metadata.

I guess one could construct these tables (and produce appropriately specialized impls for those tables to refer to) at final link time. But that presumes that one isn’t dynamically loading code into the program.

This is very much “reasoning from the implementation,” which is definitely a second-class technique. I’d love to see either 1) explanations of practical ways to make this work, or 2) explanations of why it can’t that rest on a firmer basis.


#3

Hm … I see the problem. The possibility to implement the own traits on any type makes it impossible to save the vtable (or other metadata) “in the object” similar to languages like C++.
Did I understand this correctly that the vtable is generated by the cast from Box<Type> to Box<Trait> and the vtables of Box<AnyData> and Box<Data> from my example are different?
Then it is clear that the concrete type has to be known to create the vtable for a Box<Data> trait object.

So, the only way would be something like this:

trait AnyData : Any { 
    unsafe fn get_data_vtable(&self) -> *mut ();
}

impl AnyData {
    fn downcast_to_data(self: Box<Self>) -> Box<Data> {
        unsafe {
            let meta = self.get_data_vtable();
            let mut trait_object: TraitObject = ::std::mem::transmute(self);
            trait_object.vtable = meta;
            Box::from_raw(::std::mem::transmute(trait_object))
        }
    }  
}
// ...
impl AnyData for Some { 
    unsafe fn get_data_vtable(&self) -> *mut () {
        let trait_object: TraitObject = ....?
        trait_object.vtable
    }
}  

…and no generic solution is possible…?


#4

That sounds right.

Rust’s trait coherence rules are a bit subtle, but I believe they ensure that the fact that ConcreteType implements Trait is known either when Concrete is defined, or when Trait is defined, but they don’t determine which. So the only practical time to put together a vtable for a trait object is when you actually make the trait object; if you’re permitted to make a trait object at all, you clearly know which implementation applies.

I would expect Box<AnyData> to refer to a vtable containing entries for Any's methods; and Box<Data> to refer to a vtable containing entries for both Any's methods and Data's methods. (Perhaps the Box<AnyData> vtable and the Box<Data> vtable could be combined into a single table, but that would be an optimization.)

Yes, because the vtable needs to contain pointers to the implementation’s code, and you can’t generate code for an implementation without knowing the concrete type (given that Rust implements generic functions by specialization).

The thing is, once you’ve decided that you have to implement AnyData, things can be much simpler and safer than the code you’ve written:

trait Data {
    fn yo(&self);
}

trait AnyData {
    fn downcast_to_data(&self) -> &Data;
}

fn yo_any_data(ad: &AnyData) {
    ad.downcast_to_data().yo();
}

impl<T: Data> AnyData for T {
    fn downcast_to_data(&self) -> &Data { self as &Data }
}

struct S;

impl Data for S {
    fn yo(&self) { println!("Data::yo for S"); }
}

fn main() {
    let ad : &AnyData = &S;
    yo_any_data(ad);
}

#5

Or even simpler… just implement Data? I suspect Any is just not relevant at all to whatever you’re trying to do.


#6

Hi,

after a few days of a lot of work I have time for rust again…

I’m just playing around with rust to learn how to use it or what is possible. A usecase could be something like this:

Imagine you would like to manage resources. All resources should be managed by a manager, who for example takes care that a resource is not loaded twice. Now there are subtypes of resources and someone who is doing something with the subtypes, but he does not know the exact resource type…something like this:

trait Resource { ...}

trait Image : Resource { ... }

struct JPEGImage { ... }
impl Resource for JPEGImage { ... }
impl Image for JPEGImage { ... }

struct ResourceManager {
    resources: HashMap<String, Arc<Mutex<Option<Box<Resource>>>>>,
}

struct DoesSomethingWithImages {
    images: LinkedList<Arc<Mutex<Option<Box<Image>>>>>,
}

And now at some point of the code I have a Resource and I know that it is a Image but do not know what exact type of Image it is and want to give it to DoesSomethingWithImages …
…do you know what I mean?


#7

I suggest adding a to_image method to Resource:

    fn to_image(self: Box<Self>) -> Option<Box<Image>> { None }

Then implement it in your concrete Image classes:

    fn to_image(self: Box<Self>) -> Option<Box<Image>> { Some(self) }

You’ll also want the reverse direction (minus the Option) because rustc can’t currently do generic trait upcasts.


#8

Hm, this would be a solution. But then all resource implementations have to be known and this makes it impossible to add more type without changing the Resource trait…


#9

I’m not sure if you’re still looking at this, but I think if you come at Rust; a composition-focussed language; with an inheritance-focussed mindset you’ll find it unfriendly.

I thought this was an interesting problem, since your Resource + Manager design is exactly what I would do in other languages, so I took a crack at it.

Here’s my result.

The main bits are:

  • Resource: the thing to manage
  • ResourceManager: how particular resources are managed
  • Manager: the thing that takes Resources and uses ResourceManagers to manage them

Calling code looks like this:

let mut manager = Manager {
    resources: HashMap::new()
};
    
let resource_1 = Image {
    name: "My Image".to_string(),
    path: "images/my_image.jpg".to_string()
};
    
let resource_2 = Document {
    name: "My Document".to_string(),
    contents: "This is a document with content.".to_string()
};
    
let _ = manager.manage(resource_1).unwrap();
let _ = manager.manage_with::<_, DefaultManager>(resource_2).unwrap();

Adding a completely new kind of Resource would look something like this:

struct MyNewResource {
    pub name: String,
    ...
}
impl Resource for MyNewResource {
    fn name(&self) -> String {
        self.name.clone()
    }
}

impl HasDefaultManager<MyNewResource, DefaultManager> for MyNewResource {}

//Or if there's a special Manager you want to use:
//impl HasDefaultManager<MyNewResource, MyNewResourceManager> for MyNewResource {}

With traits, you need to specify their concrete type as a generic parameter, otherwise the compiler doesn’t know what size they’re going to be. Thankfully, Rust is aggressive with its inference, so it doesn’t clutter things up too much this way. I see generics more as the caller making a promise to that method to tell it what it’s going to get.