Rust deserialization of a dynamic trait object

Brief summary

I'm currently writing a very basic dynamic protocol section in my current hobby project. The protocol trait should be turned into a dynamic trait object and I have some issues with serde.

What I want in Java

import java.io.Serializable;

// Request.java
public interface Request extends Serializable {
    public String version();
}

// Response.java
public interface Response extends Serializable {
    public String version();
}

// Protocol.java
public interface Protocol {
    public String version();
    public Response respond(Request request);
}

This is very basic, but i think you get the idea. The important thing here is the Serializable part.

What I have in Rust

use semver::Version;
use serde::{Deserialize, Serialize};

pub trait RequestTrait<'a>: Serialize + Deserialize<'a> + RequestBehaviour + Send + Unpin {}
pub trait ResponseTrait<'a>: Serialize + Deserialize<'a> + Send + Unpin {}


pub trait RequestBehaviour {
    fn peer_id(&self) -> String;
}

pub trait Protocol<'a>: Send
{
    fn protocol_identifier(&self) -> &'static str;
    fn protocol_version(&self) -> Version;
    fn respond(&self, request: Box<dyn RequestTrait>) -> Box<dyn ResponseTrait>;
}

When I try to compile, i get this error:

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
error[E0038]: the trait `RequestTrait` cannot be made into an object
   --> src/main.rs:16:36
    |
16  |     fn respond(&self, request: Box<dyn RequestTrait>) -> Box<dyn ResponseTrait>;
    |                                    ^^^^^^^^^^^^^^^^ `RequestTrait` cannot be made into an object
    |
note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
   --> src/main.rs:4:41
    |
4   | pub trait RequestTrait<'a>: Serialize + Deserialize<'a> + RequestBehaviour + Send + Unpin {}
    |           ------------                  ^^^^^^^^^^^^^^^ ...because it requires `Self: Sized`
    |           |
    |           this trait cannot be made into an object...
    |
   ::: /playground/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.136/src/ser/mod.rs:247:8
    |
247 |     fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    |        ^^^^^^^^^ ...because method `serialize` has generic type parameters
    = help: consider moving `serialize` to another trait

error[E0038]: the trait `ResponseTrait` cannot be made into an object
   --> src/main.rs:16:58
    |
16  |     fn respond(&self, request: Box<dyn RequestTrait>) -> Box<dyn ResponseTrait>;
    |                                                          ^^^^^^^^^^^^^^^^^^^^^^ `ResponseTrait` cannot be made into an object
    |
note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
   --> src/main.rs:5:42
    |
5   | pub trait ResponseTrait<'a>: Serialize + Deserialize<'a> + Send + Unpin {}
    |           -------------                  ^^^^^^^^^^^^^^^ ...because it requires `Self: Sized`
    |           |
    |           this trait cannot be made into an object...
    |
   ::: /playground/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.136/src/ser/mod.rs:247:8
    |
247 |     fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    |        ^^^^^^^^^ ...because method `serialize` has generic type parameters
    = help: consider moving `serialize` to another trait

For more information about this error, try `rustc --explain E0038`.
error: could not compile `playground` due to 2 previous errors

What I've tried

I tried moving the Request and Response types into trait types, I also tried erased_serde and I considered Typetag but the repository is archived and it seems to not work with generic impls and I need that.

I fully acknowledge, that this is a hard problem, but it seems to be a rust problem, since the language doesn't support reflection (which is a good thing, don't get me wrong) but i need a solution.

Pointing at tarpc

I think there is a solution, because tarpc seems to have solved the problem somehow, but I have massive issues comprehending the solution. It involves this interesting pin_project stuff, but I have issues understanding basically all of it. [pin_project in tarpc]

Comparison to other languages

I've considered doing this in Go, but i decided against it, because of performance. I hope the decision in favor of Rust is a huge problem here.

The tarpc link you have posted works because they don't use trait objects (they use generics). It's unrelated to pin-project. Have you considered not using trait objects?

1 Like

I have, but as far as i understood, the functionality i want is only available through trait objects. This is, because i want to create a Vec of protocols, that are again different implementations of the trait.
Something like this:

pub struct SwarmBuilder<'a> {
    transports: Vec<Arc<dyn Transporter>>,
    protocols: Vec<Arc<dyn Protocol<'a, dyn RequestTrait<'a>, dyn ResponseTrait<'a>>>>,
    muxers: Vec<Arc<dyn Multiplexer>>,
}

Yes, I know that this is a little different implementation of protocol but basically the same.

If there is a solution without dynamic trait objects, please tell me :slight_smile:

Well, if you make your traits such that serde does not appear in them, then it will work. For example, you could provide a byte array or an IO resource and have the protocol call serde itself.

Unrelated, but you do not want any lifetimes in those traits. I guarantee you that using a lifetime in them is wrong. In this particular case, the fix would be to use DeserializeOwned rather than Deserialize<'a>. (though the real fix is to not include serde in the traits at all)

1 Like

You can find a trait-object compatible Serialize trait in the erased_serde crate, however there's no equivalent for Deserialize. I don't think this is possible in Java either.

I replaced Deserialize with DeserailzeOwned, it still does not work. Playground

I'm starting to consider your solution though...

Yes, of course. That was just how you remove the lifetime. It doesn't fix the problem that serde is not designed to work with trait objects.

1 Like

I should also point out that the Serialize interfaces in Java are not at all comparable to the serde traits. The Java Serialize interface is tied to a particular data format, and cannot be used with other stuff (e.g. JSON). This is not the case for serde, whose Serialize trait will work with all formats simultaneously.

Anyway, I definitely think that you should drop the trait objects for responses and requests. Make them into some specific concrete type (a byte array, or some more fancy type with lists of headers and such). The protocol can figure out on its own how to convert back and forth to its own protocol-specific types internally.

1 Like

One thing I'll note is that it's just straight-up impossible to deserialize a truly arbitrary type, even in Java. (Well, I suppose you could have them send you a JAR that you load into your JVM and then use that, but that's such a bad idea that I'll just pretend it doesn't exist.)

What I generally see is that there's a list of concrete types specified in an attribute, or something like that. Even with reflection, one doesn't want to allow deserializing any type available (see examples in c# - BinaryFormatter deserialise malicious code? - Stack Overflow).

So that suggests to me that in Rust, you could just have an enum of all the concrete types you want to support, and deserialize that. Then after deserializing you can turn that into the boxed trait object of the active variant if you want. And your trait could have a way to turn itself back into the enum again, if needed.

2 Likes

Perhaps we should consider the tower crate as a case-study since it does something similar. It defines a Service trait. If you look at its definition, then its rather complicated because Rust doesn't really support async functions in traits, but you should understand it as the following trait:

pub trait Service<Request> {
    type Response;
    type Error;
    
    async fn call(&mut self, req: Request) -> Result<Self::Response, Self::Error>;
}

So here, the request type is not fixed, however it is a generic rather than a trait object. This has the implication that to store a vector of services, you would need to write something like

protocols: Vec<Arc<dyn Service<InsertTheRequestTypeThatTheyAllUseHere>>>

This is different from your Java version, which requires that a protocol can accept any request type that implements the interface, whereas with Rust, the request type is known.

A similar story applies to the response type, however it is an associated type instead of a generic. They also need to be specified in the trait object if you want a vector of them. The difference is that you can implement both Service<RequestType1> and Service<RequestType2> for the same struct, but associated types do not allow you to choose multiple types like this.


All in all, the above seems like a strictly better design than the Java interface. The reason is that in Java, your protocols have to try and downcast the requests and throw some sort of exception if the request is a class that the protocol doesn't understand. In Rust, this kind of type mismatch would be caught at compile time. The same applies to the response, which the caller also has to downcast to use it.

1 Like

That is indeed a bad idea. Last time I heard of it, that idea was called the "log4j vulnerability".

8 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.