Serde serializer for columnar format

I'm trying to write a Serde serializer that can serialize a large Vec of nested structs into a columnar format such as Apache Arrow. Since my structs already have Serde support, in theory this frees me from needing to deal with every field manually. My idea is to keep a stack of "active" fields (by surrounding my value.serialize in serialize_field with a push/pop) and use that to determine which output "bucket" to serialize each value to. Then at the end I can turn the paths into flat keys like "". Simplified example below.

  1. Is Serde the wrong tool for this job?
  2. Is there a better approach for serializing to columnar formats with Serde than maintaining a stack of field names? Doing a push/pop for every field of every struct seems suboptimal.
  3. Is there some clever, fast way to index the "buckets"? I'm not sure about the performance implications of using a Vec<&str> as a hash-map key.
struct Serializer {                      
    buckets: HashMap<Vec<&str>, Vec<u8>>,                      
    stack: Vec<&str>,       

impl Serializer {                                 
    fn writer(&mut self) -> &mut Vec<u8> {         
        if !self.buckets.contains_key(&self.stack) {           
            self.buckets.insert(self.stack.clone(), Vec::new());

impl<'a> ser::Serializer for &'a mut Serializer {
    fn serialize_i8(self, v: i8) -> Result<()> {

impl<'a> ser::SerializeStruct for &'a mut Serializer {                        
    fn serialize_field<T>(&mut self, key: &'static str, value: &T) -> Result<()> where T: ?Sized + Serialize {
        value.serialize(&mut **self)?;

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.