How to do runtime polymorphism using Arc<Mutex<dyn SomeTrait>> in Rust?

help
1

Suppose I want to write a code that on runtime can receive different types of decoder that share the same interface, that is trait Decoder. I'd like to get the Arc<Mutex<dyn Decoder>> and downcast to my specific decoder. Something like this:

use std::sync::{Arc, Mutex};

trait Decoder {
    
}

struct SpecificDecoder1 {
    
}

impl Decoder for SpecificDecoder1 {
    
}

struct SpecificDecoder2 {
    
}

impl Decoder for SpecificDecoder2 {
    
}

fn main() {
    let decoder: Arc<Mutex<dyn Decoder>> = Arc::new(Mutex::new(SpecificDecoder1{}));
    
    if let Ok(specific_decoder_1) = decoder.downcast::<Mutex<SpecificDecoder1>>() {
        
    } else if let Ok(specific_decoder_2) = decoder.downcast::<Mutex<SpecificDecoder2>>() {
    
    } else {
        
    }
}

But downcast has this implementation only:

pub fn downcast<T>(self) -> Result<Arc<T>, Arc<dyn Any + 'static + Send + Sync>>

where
T: Any + Send + Sync + 'static,

it looks like T has to implement Any. I guess I have to do something like this:

impl Any for SpecificDecoder1 {
    fn type_id(&self) -> TypeId {
       //what to do here? TypeId has no constructors
    }
}

Also, is this the right way? On C++ I'd use std::shared_ptr and std::dynamic_pointer_cast<SpecificDecoder1> and etc. This is what I want to do.

2

If T: 'static, trait Any already implemented... and if it's not, it cannot be. Documentation. That's not what you're running into with Arc::downcast.

Arc::downcast is implemented on Arc<dyn Any + Send + Sync>, so before you can use it, you would have to coerce to one of those. (Note that this is a single, concrete type.) With enough added redirection and maybe trait bounds too, you could get there. But then you couldn't downcast directly into a SpecificDecoder1, you would be trying to downcast into what you started with: a Arc<Mutex<Box<dyn Decoder>>>. So it's not really what you want.

What you really want to downcast is your &mut dyn Decoder into a &mut SpecificDecoder1 (or maybe without the mut depending on your use case). So first get ahold of a dyn Decoder reference, and then coerce it to a dyn Any reference. (Sadly this step can be a pain, and I'm not really sure what the best approach is, but it can be done.)

Then you can use the Any methods to try and downcast your pointer.

There is the question of why you need to downcast to a specific decoder. Shouldn't the trait cover the functionality you need?

If you only use these after downcasting, consider just passing around an enum of the types you care about instead.

3

video decoders all have particularities so I cannot rely only on the common dyn Decoder trait. Sometimes I have to downcast to set some specific parameters. I uses to do this in C++ by using std::dynamic_pointer_cast.

Now that you mentioned, I don't know if an enum would be a better option or not, I think that using run-time polymorphism gives me more freedom for future modifications.

However, now that you mentioned the lifetime problem, it could also be a problem in the future. Or not. I don't understand exactly. When they say T: 'static, does it mean that if my type holds a reference, it will not work? For example:

struct MyStruct<'a> {
    slice: &'a[u8]
}

this cannot be made into Any to then be downcasted?

4

Traits are generally used when the user of the library can add new implementations. If all possible variants are known to you, it's possible to use an enum, and it'd be trivially extensible - just add a new variant, fix all compiler errors, and you're done.

5

Yeah, if you find the need to downcast, I would definitely try to either change it to use enums, or to add the functionality to the trait so that downcasting is unnecessary.

1 Like
6

Correct. Lifetimes are part of a type, but are erased by codegen. Code isn't monomorphized by lifetime (that would be some sort of combinatorial explosion in size, if it's even possible, I haven't given it much thought), so there's no way to downcast to a specific lifetime.

There was an accepted RFC to have TypeIds for non-static types, but even that was retracted. In part because it didn't differentiate by lifetime, but gave the impression that types with the same lifetime-erased TypeId were in fact the same type.

I can think of some potential (but ugly) workarounds...

trait Decoder {
    // override this in `impl Decoder for SpecificDecoder1`
    fn as_specific_decoder_1(&self) -> Option<&SpecificDecoder1> {
        None
    }
}
7

...however, this is, in fact, a manual reimplementation of enum matching.

8

Sure. Downcasting isn't?
Edit: Oh, you're not the OP :sweat_smile:

9

I'd say it's a nice middle-ground: I could imagine (although I have never encountered such a specific need myself) someone needing:

  • "extensibility", and thus needing to use dyn Trait;

  • downcasting, but within a library-fixed set of types (non extensible part of the API, we could say);

  • some of those library-fixed types, or the types provided by downstream users, may not be 'static.

In that case, a "manual downcasting method" such as the one suggested by @quinedot is actually a very apt (and non-hacky) solution.

That being said, I find the cases where extensibility is required to be so rare, that an enum will indeed most likely suffice.

For the sake of completeness, since going from dyn Trait to an enum of fixed Trait implementors can come with some ergonomic hindrance, I'll mention the following helper crate to palliate that:

1 Like
10

I just remembered why I didn't choose enum. My code has a lot of pieces that work on specific targets. So there would be a decoder just for android, one for desktop and android (ffmpeg), other for iOS, as well as other components like rtsp clients specific for each device.

Doing conditional compilation to include the enum variants for each target would get ugly. Runtime downcasting would be much more elegant here

11

what if I'm making a library that specifies trait Decoder, gives a SpecificDecoder1, but wants people to be able to make their own decoders like SpecificDecoder2?

For example, I'd create let specific_decoder_2: Arc<Mutex<dyn Decoder>> = Arc::new(Mutex::new(SpecificDecoder2::new())) and then pass to the library, which expects a Arc<Mutex<dyn Decoder>>.

However, since the decoder is shared between me and the library, I'd like to downcast the decoder to SpecificDecoder1 to make some specific changes. This would not work with enums since I can't add my new SpecificDecoder2 to the Decoder enum.

Of course the library could expect Arc<Any + Send + Sync> instead of Arc<Mutex<dyn Decoder>> . It's not very very bad but it does not enforce dyn Decoder and does not work for non 'static types.

12

what about https://github.com/marcianx/downcast-rs?

It looks like it solves the problem, except for non static things. Am I right? Or I'm missing something?

13

I haven't used it, but it looks like it.

14

This is what the extensibility we were talking about refers to. So you do need a dyn Trait.

For this, you do seem to need downcasting.

So, if you can get away with something such as

then indeed, go for it.

But since you mentioned wanting to support implementors such as:

without requiring 'a = 'static, then you are ticking all three of the bullets I mentioned:

and thus a manual downcasting method would be warranted, as @quinedot mentioned. I'll detail it here nonetheless:

trait Decoder {
    /* Current trait contents … */

    // Added.
    fn try_downcast_SpecificDecoder_ref (self: &'_ Self)
      -> Option<&'_ SpecificDecoder>
    {
        // Default impl: cannot be downcasted to that type.
        None
    }
}

together with:

impl Decoder for SpecificDecoder {
    /* … */

    fn try_downcast_SpecificDecoder_ref (self: &'_ SpecificDecoder)
      -> Option<&'_ SpecificDecoder>
    {
        Some(self)
    }
}

That way you can later on, write:

fn stuff_with_dyn_Decoder (decoder: &'_ dyn Decoder)
{
    match decoder.try_downcast_SpecificDecoder_ref() {
        | Some(specific_decoder) => { /* special code */ },
        | None => { /* fallback code */ },
    }
}

while still being able to impl Decoder for MyStruct<'_> (i.e., no 'static requirement!), and also allowing extensibility, i.e., allowing downstream crates to provide their own implementors (impl Decoder for SpecificDecoder2 { … }).

15

thanks, this made me think.

I'll have the Arc<Mutex<dyn Decoder>> be shared between threads, so gives that it needs to be sent to a thread, I think it's not even possible to have SpecificDecoder1<'a>, it should be 'static.

So, I guess it's better for me to stick with downcast_rs and just assume MyDecoder: Decoder can never have lifetime parameters. But what if I want to be a little more permissive and enable MyDecoder to have lifetime parameters but use downcasting only for MyDecoder<'static>? Maybe there are utilities for a MyDecoder that holds a reference.

But it looks like that the moment I make MyDecoder<'a>, Any will not be implemented for it. Or will it but just for the 'static case?

16

I don't know about downcast_rs, but it should be possible to feature what you mention:

trait Decoder {
    /* … */

    fn upcast_Any_ref (self: &'_ Self)
      -> &'_ dyn Any
    where
        Self : 'static,
    ;
}

although maybe this requires trivial_bounds…

EDIT: It doesn't :slightly_smiling_face:

17

Do you need to downcast? I have a Message trait that has about a dozen impls. The trait has a process() method that calls back to a method of the handler specific to the message type.

I deserialize to a msg: Box<dyn Message> and invoke msg.process(&handler).

impl Message for FooMsg {
    fn  process(&self, handler: &Handler, ...) -> Result<()> {
        handler.process_foo_msg(&self, ...)
    }
}

The handler now has the specific message type.

pub fn process_foo_msg(&self, msg: &FooMsg, ...)

This double dispatch has an extra call, but the work done in the handler is enough that the overhead isn't a problem. The only annoying part is the ... in the process() method. It has to have the union of the arguments needed by any of the message specific handlers. In my case there are only 3 of them.

18

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.