Serde - trailing characters error with custom map Visitor from_str

I'm trying to set up a minimal reprex for a bigger question that I have about serde_json and I stumbled upon the error below. Could anyone explain what am I doing wrong?

Background:
I want to store my sample json data that I use in the reprex as a string slice so that I can deserialize the data multiple times from the same content. As far as I know, this can be achieved by using serde_json::from_str(&str). On the other hand, serde_json::from_value() moves the value so I cannot access it once it has been called.

The problem/question:
I'm implementing a custom Deserialize for my data, as you can see in the code below. In this Deserialize, I am only interested in getting to the content of the "geometries" object, everything else is ignored.
However, when I try to deserialize from_str, I get an error of "trailing characters" at the colon after "geometries" in the data.
On the other hand, when I deserialize from_value, everything works.

Something must be wrong with my Deserialize or Visitor implementation, but I'm not finding what it is.

#![allow(dead_code,unused_variables,unused_must_use)]
use std::collections::HashMap;
use std::fmt;

use serde::de::{Error, MapAccess, Visitor};
use serde::{Deserialize, Deserializer};
use serde_json::json;

#[derive(Deserialize)]
struct Geometry {
    type_geom: String,
    boundary: Vec<i32>,
}

struct TargetStruct {
    version: String,
    geometries: HashMap<String, Geometry>,
}

struct TargetStructVisitor;

impl<'de> Visitor<'de> for TargetStructVisitor {
    type Value = ();

    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
        write!(formatter, "a valid file")
    }

    fn visit_map<A>(self, mut map: A) -> Result<(), A::Error>
    where
        A: MapAccess<'de>,
    {
        while let Ok(Some(key)) = map.next_key::<String>() {
            println!("{:#?}", key);
            if key == "geometries" {
                println!("--> yaaay!");
                // do something here
            } else {
                println!("--> do nothing")
            }
        }
        Ok(())
    }

    fn visit_str<E>(self, v: &str) -> Result<(), E>
    where
        E: Error,
    {
        println!("str: {:#?}", v);
        Ok(())
    }
}

impl<'de> Deserialize<'de> for TargetStruct {
    fn deserialize<D>(deserializer: D) -> Result<TargetStruct, D::Error>
    where
        D: Deserializer<'de>,
    {
        deserializer.deserialize_map(TargetStructVisitor);
        Ok(TargetStruct {
            version: "".to_string(),
            geometries: Default::default(),
        })
    }
}

fn main() {
    static DATA: &str = r##"
        {
            "version": "1",
            "geometries": {
                "id1": {"type_geom": "Polygon", "boundary": [1,2,3,4]},
                "id2": {"type_geom": "Polygon", "boundary": [1,2,4]}
            }
        }
        "##;

    // This fails with a "trailing characters" error, at the colon after "geometries"
    let cm: TargetStruct = serde_json::from_str(DATA).unwrap();

    let j = json!({
        "version": "1",
        "geometries": {
            "id1": {"type_geom": "Polygon", "boundary": [1,2,3,4]},
            "id2": {"type_geom": "Polygon", "boundary": [1,2,4]}
        }
    });
    // This succeeds
    let cm: TargetStruct = serde_json::from_value(j).unwrap();
    // This also succeeds
    let cm: serde_json::Value = serde_json::from_str(DATA).unwrap();
}

(Playground)

Output:

"version"
--> do nothing

Errors:

   Compiling playground v0.0.1 (/playground)
    Finished dev [unoptimized + debuginfo] target(s) in 9.37s
     Running `target/debug/playground`
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("trailing characters", line: 3, column: 22)', src/main.rs:79:55
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

When iterating over the values in a MapAccess you always need to pair the next_key with a next_value. Or you use the combined next_entry function. If you only call next_key then the key part (i.e., "version") and the next token to process is :. This token and the value is only processed if you call next_value. If you call next_key again, then it notices the : token, doesn't know how to process it and fails with the "trailing characters" error.

It is best if you know the expected type for next_value, but in case you don't there is always the option of using IgnoredAny:

map.next_value::<serde::de::IgnoredAny>()?;

It is probably a good idea to propagate any deserialization errors you get from next_key instead of silently dropping them. You achieve that by changing the while to something like

while let Some(key) = map.next_key::<String>()? {

You can reuse serde_json::Value too. There is no specific function for it, but you can directly write Type::deserialize(&value).

2 Likes

Fantastic! Thank you for the super clear explanation! :slightly_smiling_face:

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.