Writing a Serde Deserializer for length prefixed arrays

help
1

I'm working on a Serde Deserializer for the LCM format. LCM has a type specification language which is similar to C, and a code generator that produces bindings and serialization code for various languages.

In LCM, the size of a variable length array is given by an integer field declared elsewhere in the struct. Multiple arrays can share the same length, like so:

struct example {
    int32_t num_readings;
    float temperatures[num_readings];
    float pressures[num_readings];
}

... which may result in a generated Rust struct such as

struct Example {
    num_readings: i32,
    temperatures: Vec<f32>,
    pressures: Vec<f32>,
}

When serialized, each field/element is written to a buffer sequentially; so arrays are length-prefixed, but the length does not necessarily appear immediately before the array.

How would I go about capturing these semantics in serde? I've already implemented Serializer, and I've mostly implemented Deserializer. I've written a small struct which implements SeqAccess. Everything works, except I'm not sure how to initialize that struct with the value of (in this example) num_readings.

I feel like I should be able to something with field attributes, but I'm not sure what.

2 Likes
3

Have you found a solution?
I have been looking into solving a very similar problem and I haven't seen that this is possible so far.

4

No, I haven't managed to, although I haven't spent much time on it lately. The last time I worked on it, I started to wonder whether it even made sense to use Serde for LCM. Even with a Serializer and a Deserializer for LCM, you wouldn't be able to have an arbitrary struct implement Serialize and Deserialize and expect it to work, because the LCM protocol needs additional information - most notably, it needs a 64 bit "fingerprint" for each type, which is determinedy by the lcm-gen code generator.

So I think the right approach in my case is to do the same thing as the other language bindings - generate serialization/deserialization functions for each type. It would be easy to add an optional flag to put #[derive(Serialize, Deserialize)] on each struct.

2 Likes
5

I still haven't found time to come back to this project, but when I do, I think I'll draw inspiration from Dan Burkert's protocol buffers crate, prost. He came to a similar conclusion - that protobufs need additional information in order to serialize/deserialize, so using serde isn't a good fit.