Creating a new type from a list of traits

I am trying to create a type that is nothing more than a series of trait bounds - basically, I want to say: "Anything that is suitable for a hashmap and can be serialized by Serde"

I am trying this and and it's clearly not correct

type Payload = ?Sized + Deserialize + Serialize + Hash + Eq + PartialEq

I've been able to get something similar by defining a T on the functions that need it, but then I am doing a lot of copy and paste:

    pub fn submit_transaction<T> (
        &mut self,
        data: T,
    ) -> &mut Self where T: ?Sized + Deserialize + Serialize + Hash + Eq + PartialEq {}

I'd like to define a Type so I don't have to define the bounds on every function

Multiple things:

  • A trait is not a type. (So type aliases can't denote traits.)
  • ?Sized types can't be passed by-value.
  • Eq implies PartialEq so + PartialEq is redundant.
  • Sized is implied in generics, so you have to repeat it everywhere.

Apart from that, you can define and blanket-impl a trait which requires the specified bounds:

trait Payload: Deserialize + Serialize + Hash + Eq {}
impl<T: ?Sized + Deserialize + Serialize + Hash + Eq> Payload for T {}

pub fn submit_transaction<T: ?Sized + Payload> (
    &mut self,
    data: &T,
) -> &mut Self
11 Likes

This is much simplified with trait aliases, an open RFC: 1733-trait-alias - The Rust RFC Book

You forgot the T: ?Sized bound in the impl, making Payload only be implemented for T: Sized.

1 Like

Good catch, fixed.

1 Like

Extremely helpful. Thanks!

Ok, so revisiting this issue, here is what I have:

trait Payload<'a>: Deserialize<'a> + Serialize + Hash + Eq {}

impl<'a,T: ?Sized + Deserialize<'a> + Serialize + Hash + Eq> Payload for T {}

    pub fn submit_transaction<'a, T: ?Sized + Payload<'a>, CallBack: 'static + FnOnce(Transaction)>(
        &mut self,
        payload: T,
        on_done: CallBack,
    ) -> Transaction {
        let trans = Transaction::new(
            TransactionType::Create,
            self.submitter,
            payload,
            &self.keypair,
        );
        self.trans_observers
            .insert(trans.clone(), Box::new(on_done));
        trans
    }}

Having been required by the compiler to give a lifetime to everything, I think it's just simpler to link to the repo I'm working on. My current error is:

christianb@christianb-mac blockchain % cargo build                                      
   Compiling pyrsia_blockchain_network v0.1.0 (/Users/christianb/dev/jfrog/pyrsia/src/blockchain)
error[E0038]: the trait `transaction::Payload` cannot be made into an object
   --> src/blockchain/src/structures/transaction.rs:46:14
    |
46  |     payload: dyn Payload<'a>,
    |              ^^^^^^^^^^^^^^^ `transaction::Payload` cannot be made into an object
    |
note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
    |
   ::: src/blockchain/src/structures/transaction.rs:29:7
    |
29  | trait Payload<'a>: Deserialize<'a> + Serialize + Hash + Eq {}
    |       ------- this trait cannot be made into an object...

error[E0038]: the trait `transaction::Payload` cannot be made into an object
   --> src/blockchain/src/structures/transaction.rs:92:14
    |
92  |     payload: dyn Payload<'a>,
    |              ^^^^^^^^^^^^^^^ `transaction::Payload` cannot be made into an object
    |
note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
    |
   ::: src/blockchain/src/structures/transaction.rs:29:7
    |
29  | trait Payload<'a>: Deserialize<'a> + Serialize + Hash + Eq {}
    |       ------- this trait cannot be made into an object...

For more information about this error, try `rustc --explain E0038`.
error: could not compile `pyrsia_blockchain_network` due to 2 previous errors

The error does make sense, in so far as the error message goes. The struggle I am having is that I need something as simple as it is in almost any other language, where in I can use a Map<String,Object>, List<Thing> or just Thing and it knows what to do, because ultimately those who use this code I am working on need to just be able to shove in data that's JSON friendly.

For anyone who cares, the project is called Pyrsia, but my particular bit of work (for the above) is here, and we're actively seeking developers.

You might be able to remove the lifetime on everything by replacing Deserialize<'a> with DeserializeOwned. Unfortunately, the Payload trait cannot be made object safe, since Deserialize::deserialize() returns an owned Self, which is incompatible with using an unsized dyn Payload.

My question is, do you need to store the payload in its original type in the Transaction, instead of serializing it as a Vec<u8> when you receive it, and deserializing it when you give it out again? That would be the simplest solution to the problem.

Could you implement serde on Box<dyn Payload> (it would have to be manually, not derived), or add where Self: Sized on Payload?

Well, there is no way around object safety because it's pretty much physically impossible to do dynamic dispatch on yet-unknown types without a JIT mechanism that is built into the language. The classic alternative for "true" vtable-based dynamic dispatch for a closed set of types (i.e. avoiding generics) is an enum. But that is exactly what Value representations of serialization formats are. I.e., if you want your transaction to store JSON, then you should likely not insist on storing dyn Payloads – convert everything to serde_json::Value instead, at the first point where you decide you don't need the static type anymore.

The alternative would be, of course, to always care about static types, and make Transaction and PartialTransaction generic over the payload.

1 Like

You have two options here:

  1. Use impl Payload everywhere.

  2. If you have to use dyn Payload, checkout erased_serde - Rust and replace Payload's super trait Ser/De with traits provided in this crate.

That would actually be ideal! If they could do:

let payload = HashMap::from([
    ("Mercury", 0.4),
    ("Venus", 0.7),
    ("Earth", 1.0),
    ("Mars", 1.5),
])
submit_transaction(payload, |trans| {
    println!("transactions {} settled", trans.id())
}

struct Thing {
   first : String,
   second: String
   age : u8
}

let my_thing = Thing{
  first : "Christian",
  last: "Bongiorno",
  age: 10
}
submit_transaction(my_thing, |trans| {
    println!("transactions {} settled", trans.id())
}

let some_other_thing : Thing = get_transaction(thing_id).unwrap();
let some_map : HashMap<&str,f32> = get_transaction(map_id).unwrap();

If this sort of syntax could be used, that would be amazing!

What syntax? Apart from some trivial errors (missing semicolons and closing parentheses), there's nothing in your code that couldn't be made work today.

Well, that's what I am trying to get to, I just don't know how. Hence why I tried to define a payload as "anything that can be serialized to JSON and can be used as a key in a hashmap"

If you wanted to define a function and struct to support this, how would you do it? I mean, my idea as quoted above was just a stab since it was the loosest requirements I could think of

I've been looking at your codebase a bit further. My original idea was to directly do something like this:

pub struct Transaction {
    type_id: TransactionType,
    submitter: Address,
    timestamp: u64,
    payload: Vec<u8>,
    nonce: u128,
    // Adds a salt to harden
    hash: HashDigest,
    signature: TransactionSignature,
}

impl Transaction {
    pub fn new<T: ?Sized + Serialize>(
        type_id: TransactionType,
        submitter: Address,
        payload: &T,
        ed25519_keypair: &Keypair,
    ) -> Result<Self, bincode::Error> {
        let partial_transaction = PartialTransaction {
            type_id,
            submitter,
            timestamp: SystemTime::now()
                .duration_since(UNIX_EPOCH)
                .unwrap()
                .as_secs(),
            payload: bincode::serialize(payload)?,
            nonce: rand::thread_rng().gen::<u128>(),
        };
        Ok(partial_transaction
            .convert_to_transaction(ed25519_keypair)
            .unwrap())
    }

    pub fn payload<'a, T: Deserialize<'a>>(&'a self) -> Result<T, bincode::Error> {
        bincode::deserialize(&self.payload)
    }
}

However, this might not be what you want. It looks like the storing and retrieving the exact binary data of a transaction payload is very important in some cases (e.g., in the GENESIS_BLOCK), so it would be a bad idea to only allow access through bincode's representation. So I'd suggest either letting users handle bincode themselves, or keeping the current methods unchanged and adding separate helper methods:

impl Transaction {
    pub fn new_with_payload_value<T: ?Sized + Serialize>(
        type_id: TransactionType,
        submitter: Address,
        payload: &T,
        ed25519_keypair: &identity::ed25519::Keypair,
    ) -> Result<Self, bincode::Error> {
        Ok(Transaction::new(
            type_id,
            submitter,
            bincode::serialize(payload)?,
            ed25519_keypair,
        ))
    }

    pub fn payload_value<'a, T: Deserialize<'a>>(&'a self) -> Result<T, bincode::Error> {
        bincode::deserialize(&self.payload)
    }
}

(These methods are relative to the codebase before your changes to Transaction.)

I think this is where the issue is: the "almost any other languages" you're talking about are garbage collected, so Thing is actually a GcPointerTo<Thing>, and where you can pass an Object to a serializer and it does something useful, it is doing something along the lines of:

if object implements ISerialize {
  object.serialize(output);
} else ... built in types

In Rust, like any language that isn't already implicitly adding a pointer, you need to indirect the reference to an unknown typed value through &, Box, etc., and you don't need the Object, because traits can be added to built in types, unlike interfaces.

I would say:

  • Add a Box<dyn Payload> around the value to move it into the Transaction so it can take ownership of the passed in values
  • Swap Deserialize for DeserializeOwned, as you want the fetched values to outlive the transaction

and you should be close to done, as far as I can tell.

Rust makes it a bit too easy to go down the hard path, when the equivalent to what you would do in "almost any other language" is normally also quite simple, just not immediately obvious from what you've already been using.

Tldr: avoid lifetimes.

The problem is, DeserializeOwned isn't object safe and can't really be made object safe, since its associated deserialize() function returns a Result<Self, ...>. You could use the blanket-impl-on-supertrait trick, but that would only give you a Box<dyn Any>, which wouldn't be much more useful. (Also, you wouldn't be able to get the original type back from a Vec<u8> in any case.) That's why I've recommended just serializing it on the spot and trusting the user to deserialize it into the same type.

1 Like

That is exactly what my previous piece of advice was about – namely, converting the value to JSON right when it comes in. You would do that by accepting a regular generic type (no dyn Payload) and storing a serde_json::Value (effectively switching to dynamic typing for the internal representation).

Deserialize vs DeserializeOwned is about removing the lifetime on the returned value, nothing else? That's separate to boxing values to be serialized.

Indeed, DeserializeOwned is just for<'a> Deserialize<'a>. But both of them require Self: Sized, meaning that you can't add them to Payload, since it's being used here as Box<dyn Payload>.

1 Like