Serializing to ndjson with serde

Is it possible to implement serde's Serialize trait to be able to serialize to newline delimited json (ndjson)?

I'm looking at a collection of APIs where sometimes the body might be json and other times may be ndjson, and I know for each API which one it will be.

My original intention was to have a general send() fn that accepts a body, B, that implements Serialize and provide a struct with a Serialize impl that would serialize to ndjson. From what I can see from researching serde and serde_json however, I don't think my original intention will work because it doesn't seem possible to write \n to the underlying io::Write of serde_json's Serializer, inside of the Serialize impl.

I was then thinking I could perhaps implement a Serializer to write ndjson, but I don't think this is possible because it's not possible to generalize how to write ndjson for any impl of Serialize.

This has led me to thinking that the B parameter to the general send() fn might need to be something like &[u8], and serialization to &[u8] needs to be handled in specialized API fns, before calling into send(), where I know whether to serialize the input to each API fn as either json or ndjson.

Does this sound like a reasonable approach? I'm pretty new to Rust!

Could you do something like this?

fn serialize_to<W: Write, T: ?Sized + Serialize>(mut writer: W, value: &T) -> Result<()> {
    serde_json::to_writer(&mut writer, value)?;
    writer.write_all(b'\n')
}

serde_json::to_writer will write the value to a single line, and you can then append a newline to the end of the writer to make it ndjson compliant.

2 Likes

Thanks! Yes, this is essentially what I've ended up doing; introduced a Body trait with a write fn that controls how it should be written to a buffer

pub trait Body {
    fn write(&self, bytes: &mut BytesMut) -> Result<()>;
}

Then have json and ndjson implementations that can write to BytesMut, either by just serializing an inner Serialize impl in the case of the former, or by serializing a vec of inner Serialize impls, each followed by \n in the case of the latter.