Reuse memory in parsing ndjson with a known (at runtime) schema

help
1

Hi,

Is there a crate or mechanism over which we can parse njson without the allocations associated with serde_json?

Specifically, given

let data = r#"
{"a":"hello", "b": 1}
{"a":"world"}
{"a":null, "b": 3}
"#;

we could go about in serde_json and iterate over every row and convert each to Value::object. However, for a large number of lines, there is an optimization over which we pre-allocate something like vec![Value; 10] and read over the ndjson every 10 lines, re-using the pre-allocated vector of values.

Is there something like that available as a crate or API?

2

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.