Serde json - flatten manually

Hi, i am trying to implement custom serde serializer. I am using a Node struct with common fields for each structs. In a language like go this would be struct composition but here I tried serde flatten and it worked fine. However I don't understand how to do this manually.

impl Serialize for DBUser<'_> {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error> where
    S: Serializer {
    //tuple struct
    let us = self.0;
    let mut state = serializer.serialize_field(3)?;
    state.serialize_field("User.name", &us.name)?;
    state.serialize_field("User.email", &us.email)?;
    state.serialize_field("User.hash", &us.hash)?;
//This line does not work, with a long error 
    us.node.serialize(state);
    state.end()
}
}

Error given by
the trait api::config::_IMPL_DESERIALIZE_FOR_Dgraph::_serde::Serializer is not implemented for <S as pi::config::_IMPL_DESERIALIZE_FOR_Dgraph::_serde::Serializer>::SerializeStruct
This would be otherwise implemented as
struct DBUser {
#[serde(flatten)]
node : Node
}

Is there any way to do this with existing methods in serde?

Of course there is. #[derive] macros are not magic – they merely generate boilerplate code that you would have to (and could) write by hand otherwise.

The reason for the error is that serialize_field() returns a Result<(), Self::Error>, as clearly visible in the documentation. I don't understand what you are trying to do by serializing anything into a (), which just isn't a serializer.

You could instead inspect the code generated by Serde using cargo-expand and mimic that, although it's probably better to just stick with #[serde(flatten)].

Hi,

thank you for your reply. The problem is that I am essentially creating a wrapper struct in order to serialize with other tags (User.name instead of name) since the database requires that. So the struct already has #[serde(flatten)] but since I need to use other keys I have to write it manually. Essentially, the struct is

struct DBUser<'a>(&'a User);

The line I wrote is wrong,

    us.node.serialize(state);

Is just pseudocode but that is roughly what I am trying to do, call the node serializer using the same serializer instance I am given in the user so that they can create fields at the same level.

Is there any way to serialize the Node using the same serializer or some method in it? To do roughly what flatten does?

In go, I would use two meta tags instead, roughly like
struct User {
Name: string json:"name", data:"User.name"
}
and specify which to use at which scenario. Thanks :smile:

I don't know the low-level details of how serialization works in Go. Given the above definition of DBUser, what do you want the resulting JSON to look like?

Hi,
roughly like (given a node struct definition)

struct Node {
    pub uid: String,
}
struct User {
   pub name: String,
   pub email: String,
   pub hash: String,
   #[serde(flatten)]
   pub node: Node
}
serde_json::to_str::<DBUser>
{
    "User.name": "name",
    "User.hash": "hash",
    "User.email": "email",
    "uid": "0x1"
}

Whereas, for a regular user(not the wrapper), I get

serde_json::to_str::<User>
{
    "name": "name"
    ...
    "uid": "0x1"
}

thanks to flatten.

That's why I am looking to call the node serializer at the same level as the user, since that would automatically 'flatten' it.

You can't context-sensitively rename fields like that in Serde. However, the manual approach can still work, but you have to spell out the fields one-by-one and you have to use the serializer state correctly.

1 Like

Thanks for your help H2CO3. So the take-away is that serde does not support this without just CTRL-C-ing all the code? Since, as far as I could see from cargo expand serde::private::ser::FlatMapSerializer is, well, private! So the #[serde(flatten)] cannot be manually implemented.

I was thinking just writing a separate trait for populating a StructSerializer and reusing that instead, so it would instead be (which imo would be more convenient)

        ser.serialize_field("User.name", &self.0.name)?;
        ser.serialize_field("User.email", &self.0.email)?;
        ser.serialize_field("User.hash", &self.0.hash)?;
        Populator::populate(&self.0.node, &ser);

I'd actually step back and ask why you need to rename fields like that. Usually, when you find yourself fighting the language, the frameworks, and the idioms thereof, this indicates that you are doing something that you shouldn't in fact be doing. Context-sensitive field names sounds like something potentially very confusing – maybe this is a representation issue and there's a better way to prepare your values for storage in a database.

1 Like

Hi,

the reason is that I/we am using a database(dgraph) where predicates are specified as User.name (by choice, as using "name" would cause major problems down the road), however we intend to send data as "name", to offer a simple API to the front-end. Another alternative is essentially making an identical struct that is sent to the user, but in this way we ensure the specified data is in one position.

The reason I am fighting the frameworks is just that this was a breeze in Go and it was how we planned to do it.

It works perfectly to do something like this, which is what I hoped to do from the beginning.

impl FieldPop for Node {
    fn populate_fields<S>(&self, val : &mut S) ->  Result<(), S::Error>
        where S : SerializeStruct {
        val.serialize_field("uid", self.uid())?;
        Ok(())
    }
}

It boils down about different representations to the front-end and back-end and therefore not exposing the database logic. Otherwise you can just create separate distinct representations of the data, but if the only difference is the keys and otherwise identical I do not see the point!

I just want to advertise that there is also serde_with::with_prefix!, which might help you. With that it is quite easy to add a prefix to all fields in a struct. However, that won't work easily if node is part of the User struct. If you can change the types to something like this it would be easiest. Otherwise, you likely still need to write a lot of glue code, which is likely worse than putting rename attribute on the fields.

struct Inner {
   pub name: String,
   pub email: String,
   pub hash: String,
}

serde_with::with_prefix!(prefix_user "User.");
struct User {
   #[serde(with = "prefix_user")]
   pub user: Inner,
   #[serde(flatten)]
   pub node: Node
}
2 Likes

Have you tried #[serde(rename="User.name")]?

That unconditionally renames the field, not conditionally, what OP wants to do.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.