Building a Custom Field Deserializer in Serde

I have a JSON structure which can be null which I'd like to deserialize using Default::default, here's a playground example:

use serde; // 1.0.101
use serde_json; // 1.0.40

use serde::Serialize;
use serde::Deserialize;

use std::collections::HashMap;

#[derive(Serialize, Deserialize)]
pub struct Wrapper {
    #[serde(default)]
    pub dictionary: HashMap<String, String>,
}

static JSON: &'static str = r###"
{
    "dictionary": null
}
"###;

fn main() {
    let _w: Wrapper = serde_json::from_str(&JSON).unwrap();
}

This fails:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("invalid type: null, expected a map", line: 3, column: 22)', src/libcore/result.rs:999:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

I'm not sure why this is not the default functionality, to convert null to Default::default just as if the field was not present.

I have tried following Serde's deserialize_with field attribute documentation, but I'm not sure as to what the actual function signature needs to be. The docs read:

fn<'de, D>(D) -> Result<T, D::Error> where D: Deserializer<'de>

However, this doesn't make sense as T isn't mentioned anywhere and simply taking a Deserializer would not make it possible to deserialize a value.

I'm hoping to make a function which is generic over T: Default that will make it easy to implement generic deserialization logic so that if a field is not present or its value is null. it will return the default value for that type.

If you want a field that can handle null or a JSON object, wrap it in an Option, i.e. dictionary: Option<HashMap<String, String>>.

Yes, I'm aware that this is the "right" way to do things, but I'd like to present a hash map no matter what, it being empty if the field was null. This gets rid of a whole layer of unwrapping, which isn't super necessary.

The default attr is used when the field is not present at all, rather than present, but with a missing/null value, as you've found.

I think you probably could make progress with a separate type for the field value, and a default impl on that to use when the value is null.

But persisting with the deserialize_with attr is the right way, perhaps following the "either string or struct" example: https://serde.rs/string-or-struct.html

:tada:

I was able to get something working pretty quickly, and it's available and running on the playground!

use serde; // 1.0.101
use serde::de;
use serde::de::Deserialize;
use serde::de::Deserializer;
use serde::de::MapAccess;
use serde::de::Visitor;
use serde::de::value::MapAccessDeserializer;

use serde_json; // 1.0.40

use std::collections::HashMap;

use std::default::Default;

use std::fmt;

use std::marker::PhantomData;

/// property value is literal `null`
static IS_NULL: &'static str = r###"
{
    "duck": null
}
"###;

/// property is missing altogether
static IS_MISSING: &'static str = "{}";

/// property is the right thing
static IS_PRESENT: &'static str = r###"
{
    "duck": {}
}
"###;

/// Deserialize a value or, if it is not present or null, return `Default::default()` for that type.
pub fn null_or_default<'de, T, D>(deserializer: D) -> Result<T, D::Error>
where
    D: Deserializer<'de>,
    T: Deserialize<'de> + Default,
{
    struct NullOrDefault<T>(PhantomData<fn() -> T>);

    impl<'de, T> Visitor<'de> for NullOrDefault<T>
    where
        T: Deserialize<'de> + Default,
    {
        type Value = T;

        fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
            formatter.write_str("null or value")
        }

        fn visit_map<A>(self, map: A) -> Result<Self::Value, A::Error>
        where
            A: MapAccess<'de>,
        {
            Deserialize::deserialize(MapAccessDeserializer::new(map))
        }

        fn visit_unit<E>(self) -> Result<Self::Value, E>
        where
            E: de::Error,
        {
            Ok(Default::default())
        }
    }

    deserializer.deserialize_any(NullOrDefault(PhantomData))
}

/// A duck with a dict.
#[derive(serde::Deserialize)]
pub struct Duck {
    #[serde(default, deserialize_with = "null_or_default")]
    pub duck: HashMap<String, String>,
}

fn main() {
    let _: Duck = serde_json::from_str(&IS_NULL).expect("unable to deserialize null");
    let _: Duck = serde_json::from_str(&IS_MISSING).expect("unable to deserialize missing");
    let _: Duck = serde_json::from_str(&IS_PRESENT).expect("unable to deserialize present");
}

The only annoyance that I've run into thus far is that I need to implement visit_X for every type of possible value that the Visitor trait defines, which is a lot of methods.

Also, I've been thinking about this: it's really weird to me that #[serde(default)] covers absent fields but not null fields. As in the example, trying to deserialize {} works fine, but { "duck": null } does not. Perhaps I'm missing some detail, but I'd imagine that generally if you want Default::default for the first case, you'd also want it for the second, I can't think of a circumstance in which you'd not want Default::default where a field is null.

Perhaps I'm missing something. I'd be willing to submit a PR to Serde, but I'm not sure if I'm just missing things.

I would recommend using Option's Deserialize impl for deserializing nullable input.

fn null_default<'de, D, T>(deserializer: D) -> Result<T, D::Error>
where
    D: Deserializer<'de>,
    T: Deserialize<'de> + Default,
{
    let option = Option::deserialize(deserializer)?;
    Ok(option.unwrap_or_default())
}

Playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=2337cb6fdc88197b28ae19adff01c975

2 Likes

Perfect, greatly simplifies things :+1: