Having a problem with serde

help
1

Hello!

I'm having a problem deserializing some data I got from my spotify usage data. I'm using a struct like so

#[derive(Insertable, Default, AsChangeset, Deserialize, Clone, Debug)]
#[serde(default)]
#[diesel(table_name = crate::schema::songrecords)]
pub struct NewSongRecord {
    #[serde(with = "my_date_format")]
    ts: DateTime<Utc>,
    username: Option<String>,
    platform : Option<String>,
    ms_played: i32,
    conn_country : Option<String>,
    #[serde(alias = "user_agent_decrypted" )]
    user_agent : String,
    shuffle : Option<bool>,
    offline: Option<bool>,
    incognito_mode : Option<bool>,
    #[serde(alias = "master_metadata_track_name")]
    track_name: Option<String>,
    #[serde(alias = "master_metadata_album_artist_name")]
    artist_name: Option<String>,
    #[serde(alias = "master_metadata_album_album_name")]
    album_name: Option<String>,
    spotify_track_uri: Option<String>,
    reason_start: Option<Reason>,
    reason_end: Option<Reason>,
    id_addr_decrypted: Option<String>,
}

To deserialize data that looks like this:

  {
    "ts": "2016-11-01T11:30:31Z",
    "username": "11140152173",
    "platform": "Android OS 5.1.1 API 22 (samsung, GT-I9100)",
    "ms_played": 7490,
    "conn_country": "AR",
    "ip_addr_decrypted": null,
    "user_agent_decrypted": null,
    "master_metadata_track_name": "The Robots - 2009 Remaster",
    "master_metadata_album_artist_name": "Kraftwerk",
    "master_metadata_album_album_name": "The Man-Machine",
    "spotify_track_uri": "spotify:track:5eqZWYQ5tbIehx00NeKXz7",
    "episode_name": null,
    "episode_show_name": null,
    "spotify_episode_uri": null,
    "reason_start": "trackdone",
    "reason_end": null,
    "shuffle": true,
    "skipped": null,
    "offline": false,
    "offline_timestamp": 0,
    "incognito_mode": false
  },

This is my main function

fn main() {
    let conn = &mut establish_postgres_connection();
    let filestring = std::fs::read_to_string("full_str_redux.json").unwrap();
    let datas: Vec<NewSongRecord> = serde_json::from_str(&filestring).unwrap();
    let loaded_songs = diesel::insert_into(schema::songrecords::table)
        .values(&datas)
        .execute(conn)
        .unwrap();
    println!("Length of load {}", loaded_songs);
}

When I run my program I get the following error
thread 'main' panicked at src/main.rs:22:71:
called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 240198, column: 32)
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

I think the problem might be that serde can assign the string "null" to the None variant but since in the file null is not quoted, it's having a problem with that. Is that so? If so, is there any way to fix that besides find and replace?

Thank you!

2

Try:

    //           vvvvvvv      v
    user_agent : Option<String>,
5 Likes
3

The serde_path_to_error crate (by the author of serde) is very helpful for getting useful error messages when you encounter these types of errors.

You can use it like so:

fn main() {
    let jd = &mut serde_json::Deserializer::from_str(DATA);
    let result: Result<NewSongRecord , _> = serde_path_to_error::deserialize(jd);
    match result {
        Ok(ok) => (),
        Err(err) => {
           let path = err.path().to_string();
           println!("{:?} {}", path, err);
        }
    }
}

And you would have gotten some confirmation that the problem was indeed with the user_agent field.

"user_agent_decrypted" user_agent_decrypted: invalid type: null, expected a string at line 9 column 32
4

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.