Reflection in Rust

Hi everybody,

I've faced a challenge of serializing and deserializing of Box. The full description can be found here. It looks like the problem of deserializing Box cannot be solved without having reflection information (correct me if I'm wrong) : it's needed to match TypeID to the deserialization method for the type in runtime.

So I started thinking about how reflection could have been implemented in Rust. I realise that there are some limitations for languages that are compiled to native code but a lot of things can be made available. It seems that during compilation compiler can generate descriptive structures for all the types, traits, enums and functions. Ideally there could be also possibility of invoking functions and to have access to struct members. There should be no overhead in performance. The memory overhead will be equal to the summarised size of the descriptive structures, one instance per type, trait, enum or function. Of course, there should be possibility of turning generation of reflection information on and off because for embedded development even this memory overhead can be significant.

I'm very sorry for writing such obvious thoughts. But I couldn't find any up-to-date information about reflection status or plans. Does anyone know if some reflection implementations were discussed or maybe even some prototypes been made?

2 Likes

Do you know about the Any trait? It allows you to get the type of a value at runtime. So if you have a Box<Any> you can find out what the original type is.

I have to admit, I didn't think to hard about your problem and am just shooting in the dark/dusk. (As you didn't mention Any) Sorry if it doesn't help.

1 Like

Yes I know about Any trait. It helps solving problem of serialization of Box if SomeTrait : Any. But it doesn't help to deserialize Box. Even if you store TypeID to serialized data there is no way to get some associated with type deserialization function by the value of TypeID.

2 Likes

This sounds a bit like rust-lang/rfcs#668 Encodable trait objects. The gist of it is that although Rust can associate data to a type using vtable pointers (trait objects) as keys, they are transient and can't be sent from one executable to another portably. Therefore, a similar mechanism would have to be added for looking up data with a more persistent key.

3 Likes

Note that in C#, deserializing using reflection information lead to remote code execution: https://media.blackhat.com/bh-us-12/Briefings/Forshaw/BH_US_12_Forshaw_Are_You_My_Type_WP.pdf

Also, the large (non-embedded) C++ projects on which I've worked have all turned off C++'s RTTI, even though that's just a comparable id with a mostly-useless name -- much less than a full description. Monomorphization in particular -- which Rust also has -- tends to particularly inflate the size.

3 Likes

Thanks, Encodable looks very similar to what I need. Unfortunately topic looks not very popular and even doesn't contain important keywords like "polymorphic serialization". That explains why I haven't find it during search of possible solutions.

1 Like

As far as I remember C# has only runtime reflection. Rust already has some sort of compile time reflection (procedural macros) so efficient serialization code can be generated at compile-time. Actually serde does so. Reflection is needed only to deserialize polymorphic objects.

That's quite interesting for me. Did you measure the amount of memory that you've saved turning RTTI off? Do you have polymorphic serialization case? In our project we use cereal and uses RTTI to serialize and deserialize smart pointers.

1 Like

Turns out, there is a way to make this work on stable Rust, if you can make certain assumptions about vtable layouts etc. The technique is probably very questionable, but it was fun to try :​P It actually took me more time and effort to coerce it to work with Serde – turns out generic-heavy libraries are really hostile to dynamic serialization (serde#552).
https://github.com/Rufflewind/detrojt/blob/18c16623b5e75877273f1c6a329884a642b16f3d/src/lib.rs

1 Like

Building on @roSievers, you can get type information from a Type/Trait into a &str with std::intrinsics::type_name or TypeId but is there a way to use a TypeId as a parameter for a method such as downcast_ref() instead of the turbofish?

To try to illustrate that, currently we can do (playground link)

use std::any::*;

fn get_type_of<T: ?Sized + Any>(_s: &T) -> TypeId {
    TypeId::of::<T>()
}

fn get_downcast_ref<T: Any>(s: &Any) -> Option<&T> {
    s.downcast_ref::<T>()
}

fn main() {
    let x = "some string".to_string();

    let type_id = get_type_of(&x);
    println!("x's TypeId = {:?}", &type_id);
    println!("x downcast ref = {:?}", get_downcast_ref::<String>(&x));
}

but would it be possible to use a TypeId as a parameter to Any::downcast_ref()?

Something like

fn get_type_of<T: ?Sized + Any>(_s: &T) -> TypeId {
    TypeId::of::<T>()
}

let x = "some string".to_string();
let type_id = get_type_of(&x);
&x.downcast_ref(&type_id); // <- can't compile yet
1 Like

A function can only have one return type.
downcast_ref is a template function. The generic identifier <T> is substituted for type at compile time creating targeted functions.

You are right in terms of internals.

But @bgbahoue inquiry could be a missing link to making Any and TypeId more useful, from the user perspective.

1 Like

2 posts were split to a new topic: Reflection for business-type applications