Deserializing chrono::NaiveDateTime => "invalid characters"

For some reason, parsing NaiveDateTime from CSV doesn't work, although all necessary stuff is there. The code in the Playground.

Serde/CSV/chrono can't parse a single line of CSV, no matter how you try to fix it.

Tried:

  • adding quotes. I had datetime without quotes, it works fine without quotes in Pandas/Polars, but here no matter whether they are or aren't here, it fails.
  • tried removing the fractional part of seconds
  • already removed all other fields that could have caused confusion
  • checked that serde, derive and chrono feature "serde" are on.

Still date&time can't be parted into NaiveDateTime.

Cargo.toml:

[package]
name = "mycrate"
version = "0.1.0"
edition = "2021"

[dependencies]
csv = "1.1"
serde = { version = "1", features = ["derive"] }
chrono = {version = "0.4", features = ["serde"] }

main.rs:

use chrono::naive::NaiveDateTime;
use std::{error::Error, fs::File};
use csv::Reader;
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug)]
struct OrderRow {
    dt: NaiveDateTime,
}


fn main() {
    let bts = "id,dt
1,\"2021-01-18 08:32:45.123\"
2,\"2021-01-18 10:55:12.456\"
".as_bytes();
    
    
    for row in Reader::from_reader(bts).deserialize() {
        let r: OrderRow = row.unwrap();
        println!("{:?}", r);
    }
}

Output:

Result::unwrap()` on an `Err` value:
Error(Deserialize { pos: Some(Position { byte: 6, line: 2, record: 1 }),
err: DeserializeError { field: None, kind: Message("input contains invalid characters") } })

I have no idea what else characters can be invalid here. Please, help!

The datetime strings separate date and time by a space, while chrono expects it to be a T e.g. "2021-01-18T08:32:45.123".

3 Likes

Oh, that makes sense. Is there a way to provide a custom format string for deserialization?

There's NaiveDateTime::parse_from_str. The example even uses your format.

1 Like

You can provide an alternative deserialize function to serde

#[derive(Serialize, Deserialize, Debug)]
struct OrderRow {
    #[serde(deserialize_with = "custom_deserialize")]
    dt: NaiveDateTime,
}

fn custom_deserialize<'de, D>(deserializer: D) -> Result<NaiveDateTime, D::Error>
where
    D: serde::Deserializer<'de>,
{
    struct CustomVisitor;

    impl<'de> serde::de::Visitor<'de> for CustomVisitor {
        type Value = NaiveDateTime;
        fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result {
            formatter.write_str("a datetime in the format %Y-%m-%d %H:%M:%S%.f")
        }

        fn visit_str<E>(self, value: &str) -> Result<Self::Value, E>
        where
            E: serde::de::Error,
        {
            NaiveDateTime::parse_from_str(value, "%Y-%m-%d %H:%M:%S%.f").map_err(E::custom)
        }
    }

    deserializer.deserialize_str(CustomVisitor)
}

This is mostly copied from chrono's source.

If you need to also serialize it, you can also write a serializer function, which is simpler.

4 Likes

Yes, I saw this. I meant a "serde(with...)" macro. Here's what I implemented:

    #[serde(with="parse_time")]
    dt: NaiveDateTime
...

pub mod parse_time {
    use serde::{Deserialize, Serializer, Deserializer, Serialize};
	use chrono::naive::NaiveDateTime;

	pub fn serialize<S>(
		dt: &NaiveDateTime,
		serializer: S,
	) -> Result<S::Ok, S::Error>
	where
		S: Serializer,
	{
		dt.format("%Y-%m-%d %H:%M:%S%.f").to_string().serialize(serializer)
	}

	pub fn deserialize<'de, D>(
		deserializer: D,
	) -> Result<NaiveDateTime, D::Error>
	where
		D: Deserializer<'de>,
	{

		let t = String::deserialize(deserializer)?;
        // it doesn't try to handle the error, just unwraps
        let d = NaiveDateTime::parse_from_str(&t, "%Y-%m-%d %H:%M:%S%.f").unwrap();
		Ok(d)
	}
}

This allocates a string while serializing and deserializing. Here's serialize written properly:

pub fn serialize<S>(time: &NaiveDateTime, serializer: S) -> Result<S::Ok, S::Error>
where
    S: serde::Serializer,
{
    serializer.collect_str(&time.format("%Y-%m-%d %H:%M:%S%.f"))
}
2 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.