Custom serde deserializer/visitor to flexibly parse with wrapper or without

help
1

I am currently trying to simplify for downstream developers an API client I writing. The REST API I am consuming from has a root key "response" and then basically the entity fields there after.

For example, the struct below might be what I want to deserialize from the payload.

#[derive(Clone, Debug, Default, Deserialize, Serialize)]
pub struct Example {
   pub field1: String
   pub field2: String
   pub field3: u32
}

However in the naive case I have to do something like:

#[derive(Clone, Debug, Default, Deserialize, Serialize)]
pub struct Single<T> {
   pub response: T 
}

and

-> Response<Single<Example>> might be what comes back from the client.

I don't want consumers of the client having to deal with the "response" root key represented by the "Single" struct.

Exploring this I landed on the solution talked about here - Add a #[serde(root = "<something>")] for deserializing Structs under alternate root names · Issue #1345 · serde-rs/serde · GitHub. It uses a custom deserializer and visitor to automagically remove the "response" root key and now I can just do Response<Example> on my client returns.

Now this technique works fine, but I now want to extend a solution to handle the case of Example being wrapped OR not wrapped with a response root key - as this Example object is used sans wrapper in other places of the API.

I'm trying to work out how this is possible. One idea I had was in the WrapperVistor from - Add a #[serde(root = "<something>")] for deserializing Structs under alternate root names · Issue #1345 · serde-rs/serde · GitHub - if the wrapper key is missing I can deserialize the Example object directly. However the serde::de::MapAccess type is stateful (it remembers it was iterated over) and there appears to be no way to reset it to the beginning. Likewise there doesn't seem to be a way to backtrack or reset a deserializer. Another option was the deserialize to a json::Value and use get to see if the "response" key is present and take the appropriate action, but I haven't got this working yet.

I'm fairly new to rust so I feel like there are other options I'm not aware of. Would love to get some knowledgable guidance. Am I just fighting the canonical way to do this is rust?

2

You could just flatten the response field:

That way, Single<T> will be serialized as the plain T.

3 Likes
3

If I understand correctly, this works for the JSON but not the struct itself.

You still need to do result.response.charge in rust with flatten but the JSON serialized goes from

{
   "response": {
      "field1": "field1value",
      "field2": "field2value",
      "field3": "field3value"
    }
}

to

{
   "field1": "field1value",
   "field2": "field2value",
   "field3": "field3value"
}

I want to be able to deserialize the former but in the rust client result be able to access it like

result.field1
result.field2
result.field3
4

Yes. The proposed solution only changes the JSON representation of your data structure.

Now I think I understand what you are trying to achieve, but I don't know how to solve it with a custom deserializer since, as you mentioned, the deserializer is consumed.

Another approach would be to deserialize both options (nested under response and not nested), and then only expose the unwrapped field through your API: playground.

5

Thanks for your answer. I did persist and I ended up stumbling on some clues that lead me to working out a way to do this. I'm not entirely clear on why this works but it seems to for me. See below:

#[derive(Clone, Debug, Default, Deserialize, Serialize)]
#[serde(remote = "Self")]
pub struct Example {
   pub field1: String
   pub field2: String
   pub field3: u32
}

#[derive(Debug, Deserialize)]
pub struct Single<T> {
    pub response: T
}

impl<'de> Deserialize<'de> for Example {
   fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
        #[derive(Deserialize)]
        #[serde(untagged)]
        enum Helper {
            Contained(Single<Example>),
            #[serde(with = "Example")]
            Flat(Example),
        }

        Ok(match Helper::deserialize(deserializer)? {
            Helper::Contained(single) => single.response,
            Helper::Flat(this) => this
        })
   }
}

If I understand correctly this creates a function to deserialize Example off to the side that doesn't quite have the same signature as the normal deserialize. This allows me to create an explicit deserialize that attempts the untagged enum match and then calls the generated deserialize as required.

This handles both the response with the container "response" root and raw response.

The one downside seems to be that it gives pretty unhelpful deserialisation error messages if you make a mistake or get a funky payload.

1 Like
6

That's interesting! The juggling to get around a stack overflow is intriguing.

Have you managed to get serialization to work with this?

7

If this is an HTTP API client, and every endpoint looks like

{
  "response": {
    the actual data ...
  }
}

then why don't you just factor the whole Single<T> out of the very core of the mechanism for sending the request and receiving the response? If e.g. you write a method

fn send<R: Request>(&self, req: R) -> Result<R::Response, HttpError>

then you could implement it in a way that it always deserializes into a Single<R>, and then simply returns Ok(wrapper.response).

8

I haven't tried serialization - I don't really need it for the client - only deserialization. I expect there is some kind of analogous implementation though.

9

This is a good point. While the response is not always wrapped (the API is inconsistent - not mine so I cannot fix it) - I can identify which endpoints wrap the response and which one does not. So I could probably implement two such send methods and use the appropriate one for a selected endpoint.

I will try that next. Thanks for the suggestion.

10

So following your train of thought it was it was simply this that I did in the end, which I use around the generic send and in the resource specific function for single response endpoints that are wrapped. I know I could probably have done it close to your way but I got this working pretty quick and then didn't mind it. It isolates the end user of my client from having to think about the wrapper part - which is what I wanted.

pub fn unpack_contained<T: 'static>(container_response: Response<Single<T>>) -> Response<T> {
    Box::pin(container_response.map(|pb| pb.map(|single| single.response)))
}

Note that the first map is this one from here as what is returned form my lower level API client is Response defined by pub type Response<T> = BoxFuture<'static, Result<T, Error>>;

Thanks for your help!