Serde Deserializing with Context

help
1

I am trying to serialize (not a problem) and deserialize a nested struct using serde . The problem I'm facing is that one of the fields needs some context to create itself. For example

#[derive(Serialize, Deserialize)]
struct Foo {
    x: u32,  // (de-)serializable
    #[serde(skip)]
    y: SomeType, // not deserializable because no Default implementation.
}

impl Foo {
   fn new(ctx: SomeContext) -> Self  {
      Foo {
        x: 0,
        y: ctx.make_some_type(0),
      }
   }
}

#[derive(Serialize, Deserialize)]
struct Bar {
    z: Foo,
}

I've thought of a couple of solutions to work around this and wanted to get people's opinions on which is better, and to see if there's any other way I've not thought of (most likely!).

1. "Mirror" Structs

We can create mirror structs which only have the serializable parts, deserialize this, then convert to the proper structs when we have the context available.

#[derive(Serialize)]
struct Foo {
    x: u32,
    #[serde(skip)]
    y: SomeType, 
}


impl Foo {
   fn new(ctx: SomeContext) -> Self  {
      Foo {
        x: 0,
        y: ctx.make_some_type(0),
      }
   }
}

#[derive(Serialize)]
struct Bar {
    z: Foo,
}

// This is deserializable because it has only the non-contexty fields ...
#[derive(Deserialize)]
struct FooDeserializer {
    x: u32,
}

// ... but we need to add a method to convert it to the real type ...
impl FooDeserializer {
    fn to_foo(self, ctx: SomeContext) -> Foo {
        Foo {
            y: ctx.make_some_type(self.x),
            x: self.x,
        }
    }
}

/// ... and it's infectious; we have to do the same all the way up the hierarchy.
#[derive(Deserialize)]
struct BarDeserializer {
    z: FooDeserializer,
}

impl BarDeserializer {
    fn to_bar(self, ctx: SomeContext) -> Bar {
        Bar {
            z: self.z.to_foo(ctx),
        }
    }
}

fn main() {
    let ctx = SomeContext::new();
    let bar: BarDeserializer = get_deserialized();
    let bar: Bar = bar.to_bar(ctx);
}

The advantage of this pattern is it makes full use of the type system and when we have a Foo we know everything is initialized properly. The downside is clearly it's very verbose, and it's infectious. Any types which use our structs must also follow this pattern (I'm not writing a library so perhaps this isn't such a down-side). We have to keep them in sync but I think that's not such an issue since we always need to construct an actual Foo and Bar at the end of the day and the compiler will check we're providing all of the fields.

2. "Lazy" Initialization

The second solution is to make the contexty fields Options, initially deserialize as None, and the create them later when we have the context.


/// Can be deserialized fine since [`Option`] implements [`Default`]
#[derive(Serialize, Deserialize)]
struct Foo {
    x: u32,
    #[serde(skip)]
    y: Option<SomeType>, 
}

/// But we still need a method call to add the contexty field ...
impl Foo {
   fn with_ctx(self, ctx: SomeContext) -> Self  {
      Self {
        y: ctx.make_some_type(self.x),
        x: self.x,
      }
   }
}

#[derive(Serialize)]
struct Bar {
    z: Foo,
}

/// ... and it's still infectious.
impl Bar {
    fn with_ctx(self, ctx: SomeContext) -> Bar {
        Bar {
            z: self.z.with_ctx(ctx),
        }
    }
}

fn main() {
    let ctx = SomeContext::new();
    let bar: Bar = get_deserialized(); // contexty fields are `None` here
    let bar: Bar = bar.with_ctx(ctx);  // now they're set
}

The advantage of this approach is that we don't need the very verbose mirroring of the struct hierarchy. However we still have the nested method calls to add the context-using fields. The disadvantage is that we cannot tell purely from the types whether we have the context initialized fields even if in principle we know that we always will at the point of use.

3. Use DeserializeSeed

I saw this trait in serde and had a quick go at implementing it but a) it was not clear to me exactly how to use it and if it was suitable for my needs and b) it rapidly got even more verbose than even my solution #1 and seemed to require a deep knowledge of serde, which I do not posses.

Please let me know your thoughts.

Thanks!

2

I personally don't mind the verbosity of 1. With DeserializeSeed you should be able to deserialize Bar with a visitor that has access to ctx, but any Deserialize implementation will be quite verbose compared to the derive macros.

3

I would prefer method 1, it's easy to understand and implement. and some days earlier, I was informed of the #[serde(from = "FromType")] attribute, which can reduce the verbosity if you can use it in your case.

1 Like
4

Another option you could try to reduce the mirror struct verbosity is using generics to pass down the actual type. With associated types this parameter can be a bag of related types if you have a bunch of different "mirrors" that need context, and you can default the parameter so normal use isn't any different.

Potentially, you could even use a library like functor_derive — Rust library // Lib.rs to implement the mapping with context through the types recursively, though it would need enough trait hackery that it might not be worth it.

1 Like
5

You also have the tried and true, though mildly sketchy "throw your context in a thread-local"