Proc_macro_attribute to add a `serde(deserialize_with)` to every field in a struct

[balajisivaraman]

Hey All,

I apologize in advance if this is a repeated question. I searched Google, DuckDuckGo, and this very site to come up with an answer for an issue I'm facing, but couldn't find one that existed already. Hence I'm posting it here in the hope that someone will clarify.

Problem Statement:
We have a struct that we are deserializing from an unstable data source where any field can be null at random in the JSON. We do not want to mark every field as Option in the struct as that would be quite cumbersome. The serde(default) attribute does not help us here as that only deals with missing fields and not fields that are present in the JSON with null as the value.

We came across serde_with::rust::default_on_null - Rust that solves the problem for us. And it works when I add it on a single field in my Struct. But we want the attribute to be set on all fields of our struct. And we certainly don't want it done by hand.

Enter procedural macros!

We created a simple attribute macro whose code is the following:

#[proc_macro_attribute]
pub fn deserialize_null_values_as_default(
    _attr: proc_macro::TokenStream,
    input: proc_macro::TokenStream,
) -> proc_macro::TokenStream {
    let ast = parse_macro_input!(input as DeriveInput);
    let gen = match modify_ast_to_add_deserialize_attr(ast) {
        Ok(g) => g,
        Err(e) => return e.to_compile_error().into(),
    };
    gen.into_token_stream().into()
}

fn modify_ast_to_add_deserialize_attr(ast: DeriveInput) -> Result<impl ToTokens> {
    let new_data: Data = match ast.data {
        syn::Data::Struct(DataStruct {
            struct_token,
            fields,
            semi_token,
        }) => match fields {
            Fields::Named(named) => {
                modify_fields_to_add_deserialize_attr(struct_token, semi_token, named)?
            }
            fields => {
                return Err(syn::Error::new(
                    fields.span(),
                    "Deserialize Null Values as Default requires named fields",
                ))
            }
        },
        data => data,
    };
    Ok(DeriveInput {
        attrs: ast.attrs,
        vis: ast.vis,
        ident: ast.ident,
        generics: ast.generics,
        data: new_data,
    })
}

fn modify_fields_to_add_deserialize_attr(
    struct_token: syn::token::Struct,
    semi_token: Option<syn::token::Semi>,
    fields: FieldsNamed,
) -> Result<Data> {
    let mut fields = fields;
    let attr = create_field_attribute();
    fields
        .named
        .iter_mut()
        .for_each(|f| f.attrs.push(attr.clone()));
    Ok(Data::Struct(DataStruct {
        struct_token,
        fields: Fields::Named(fields),
        semi_token,
    }))
}

fn create_field_attribute() -> Attribute {
    let stream = quote!((deserialize_with = "serde_with::rust::default_on_null::deserialize"));
    let mut segments: Punctuated<PathSegment, Token![::]> = Punctuated::new();
    segments.push(PathSegment {
        ident: Ident::new("serde", Span::call_site()),
        arguments: PathArguments::None,
    });
    Attribute {
        pound_token: syn::token::Pound::default(),
        style: syn::AttrStyle::Outer,
        bracket_token: syn::token::Bracket::default(),
        path: Path {
            leading_colon: None,
            segments,
        },
        tokens: stream,
    }
}

When expanded, this does exactly what is intended. It adds the serialize_with attribute to every one of our struct fields.

The problem is that the attribute is having no effect at run time. If I try deserializing a JSON with null for one of the keys, I'm getting JsonParseError(Error("invalid type: null, expected i64", line: 7, column: 30)) indicating that Serde deserialization is behaving as if the attribute doesn't exist at all.

I have tried multiple ways of generating the attribute token stream, using quote, using the structs provided in syn by hand. Nothing seems to work.

At this point, I'm not sure if it is possible to use an attribute macro to add field level attributes like this.

Any help would be greatly appreciated. Thanks in advance.

[steffahn]

Just to double-check, can you als give an example of how you call the macro?

(I think it would need to be #[deserialize_null_values_as_default] followed by the #[derive(Deserialize, …)], i. e. in particular the order should be important.)

[balajisivaraman]

@steffahn, Oh my! Thank you so much for pointing that out.

Indeed, I was adding #[deserialize_null_values_as_default] as the last line before the struct definition. The moment I moved it before the #[derive(Deserialize, …)], things started working.

And I can clearly see why since you mentioned the order matters also. The derive(Deserialize) derives an implementation of Deserialize for my struct, which relies on the field level attributes. And so the attribute macro needs to be expanded before the derive macro.

I feel like an idiot for not spotting this and wasting around 4 hours of my time. But thank you so much!