Serializing to ndjson with serde

Is it possible to implement serde's Serialize trait to be able to serialize to newline delimited json (ndjson)?

I'm looking at a collection of APIs where sometimes the body might be json and other times may be ndjson, and I know for each API which one it will be.

My original intention was to have a general send() fn that accepts a body, B, that implements Serialize and provide a struct with a Serialize impl that would serialize to ndjson. From what I can see from researching serde and serde_json however, I don't think my original intention will work because it doesn't seem possible to write \n to the underlying io::Write of serde_json's Serializer, inside of the Serialize impl.

I was then thinking I could perhaps implement a Serializer to write ndjson, but I don't think this is possible because it's not possible to generalize how to write ndjson for any impl of Serialize.

This has led me to thinking that the B parameter to the general send() fn might need to be something like &[u8], and serialization to &[u8] needs to be handled in specialized API fns, before calling into send(), where I know whether to serialize the input to each API fn as either json or ndjson.

Does this sound like a reasonable approach? I'm pretty new to Rust!

1 Like

Could you do something like this?

fn serialize_to<W: Write, T: ?Sized + Serialize>(mut writer: W, value: &T) -> Result<()> {
    serde_json::to_writer(&mut writer, value)?;
    writer.write_all(b'\n')
}

serde_json::to_writer will write the value to a single line, and you can then append a newline to the end of the writer to make it ndjson compliant.

4 Likes

Thanks! Yes, this is essentially what I've ended up doing; introduced a Body trait with a write fn that controls how it should be written to a buffer

pub trait Body {
    fn write(&self, bytes: &mut BytesMut) -> Result<()>;
}

Then have json and ndjson implementations that can write to BytesMut, either by just serializing an inner Serialize impl in the case of the former, or by serializing a vec of inner Serialize impls, each followed by \n in the case of the latter.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.