Serde: When implementing deserialize, is it possible to consume the first value from the deserializer before moving onwards?

help
1

I am writing a custom serializer and deserializer for a few data formats, one is which has a few values before a json value, so like XXX{}. In a serializer I have an idea on how to implement this, however for deserialize, I dont know if serde allows you to pass in something like a mutable reference to a deserializer and like pop one or a few values off it and be able to read those values, and pass the rest into another deserializer without having the impliment the whole Visitor pattern.

2

as far as I know, serde deserializer does not support this feature.

but you can use an intermediate value that can represent the serde data model (e.g. serde_json::Value if you are only interested in json, or serde_value::Value if you want to support more) as a "buffer".

the trick is, this intermediate value implements Deserialize and Deserializer at the same time.

this is similar to the decode_buffer() in musli, except it is builtin feature for musli, while for serde, it's more of a workardound.

3

I was told:
impl Deserializer for &mut MyFormat, then you can do u32::deserialize(&mut MyFormat)
would get me the first u32 and it would remove it from the deserializer, is this true?

4

yes, it can, but to be pedantic, it depends on how your deserializer is implemented.

the notion of "remove (a value) from the deserializer" only applies when you are deserializing a sequence type, or when the deserializer has stream-like behavior.

in a more general context, when you deserialize a value (of type u32, in your example) from a deserializer, it can either succeed or fail, but you have no idea what happens to the internal state of the deserializer.

but since you are the owner of MyFormat, you are in control of its behavior. you need to at least have some positional information in the deserializer in order to make sense the "remove it from the deserializer" semantic.

do you mind providing more details of the data format?

5

The data format is:

XXX{}

where XXX is the tag and {} and whatever is inside it is json, and I will use serde json to handle the parts after XXX.

What would you recommend for this?

6

This is still not enough information.
Is the format nested?

XXX{"foo": YYY{"bar": 123}}

Does the data occur in lists?

[
    XXX{"foo": "Bar"},
    YYY{"bar": 12.23}
]

If not, and what you say is truly the whole format (XXX{}) I would spare any hassle and just use a preprocessor to strip the "tag" and then delegate to serde_json.

7

as pointed out by @conqp , it depends on if the tag is recursively nested.

if the part following the tag is pure json and no nested tags, you can simply skip the tag at construction.

for example, if the deserializer is constructed from a &str (or &[u8] if it's a binary format), you can simply skip the initial tag bytes and wrap the remaining input in a json deserializer:

something like:

struct MyDeserializer<'de> {
    inner: serde_json::Deserializer<StrRead<'de>>,
}
impl<'de> MyDeserailizer<'de> {
    fn from_str(input: &str) -> Self {
        let inner = serde_json::Deserializer::from_str(&input[TAG_LEN..]);
        Self { inner }
    }
}