How to serialize type-level information? E.g. to serialize generic type parameters of a struct

Context

My crate sosecrets-rs has a type named Secret<T, MEC: typenum::Unsigned, EC, typenum::Unsigned> and it uses the typenum crate to enable compile time counting of the number of exposure of the secret and if EC is greater than MEC, the program would not compile.

Here is an example of how one can use it:

use sosecrets_rs::{
  prelude::*,
  traits::ExposeSecret,
};
use typenum::U2;

// Define a secret with a maximum exposure count of 2
let secret = Secret::<_, U2>::new("my_secret_value".to_string());

// Expose the secret and perform some operations with the exposed value; secret has been exposed once: `EC` = 1, `MEC` = 2;
let (next_secret, exposed_value) = secret.expose_secret(|exposed_secret| {
    // `exposed_secret` is only 'available' from the next line -------
    assert_eq!(&*exposed_secret.as_str(), "my_secret_value"); //     ^
    // Perform operations with the exposed value                     |
    // ...                                                           v
    // to this line... -----------------------------------------------
});

Problem

I am looking at how I am able to serialize Secret<T, MEC, EC> such that it is something like this:

{
    "secret": "encrypted_secret_string_or_bytes",
    "MEC": 5
}

Such that it can be deserialized into Secret<Bytes, typenum::consts::U5, typenum::consts::U0>.

I have researched the following:

Stackoverflow
Serde-attrs-bound

If you are fine with the code which tries to deserialize to Secret<Bytes, U5, U0> and rejects data which has e.g. "MEC": 4 you should be able to do that reasonably easy: basically deserialize input to a helper structure SerializedSecret {secret: String, MEC: u32}, error out if MEC field happens to not match MEC::to_u32() and proceed with decrypting secret and deserializing result into actual type. If you want to deserialize to Secret<Bytes, _, U0> and get _ at runtime then no, you cannot do that, compiler has to know types at compile time.

1 Like

Delegating Serialize/Deserialize to a helper struct is easy, but required a lot of code. But the upside is that it's possible to make the wrong value of MEC a kind of "parse error" in serde.

BTW, wouldn't it be possible for users of your library to copy/clone the secret by serializing it, defeating the purpose?

1 Like

I tried actually writing the code and it appears that you cannot really implement serde’s Deserialize on Secret: you can write something like this

extern crate serde;
extern crate serde_derive;
extern crate typenum;

use serde_derive::Serialize;
use serde_derive::Deserialize;
use serde::Deserializer;
use serde::de;

struct Secret<T, MEC: typenum::Unsigned, EC: typenum::Unsigned> {
    data: T,
    mec: MEC,
    ec: EC,
}

#[derive(Serialize, Deserialize)]
struct SerializedSecret {
    secret: String,
    mec: u32,
}

pub trait DeserializeSecret {
  fn deserialize_secret(encripted_secret: String) -> Self;
}

impl<'de, T, MEC, EC> serde::Deserialize<'de> for Secret<T, MEC, EC>
    where T: DeserializeSecret,
          MEC: typenum::Unsigned,
          EC: typenum::Unsigned,
{
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        let ssecret = SerializedSecret::deserialize(deserializer)?;
        if ssecret.mec != MEC::to_u32() {
            Err(de::Error::invalid_type(de::Unexpected::Unsigned(
                ssecret.mec.into()), &"different MEC value"))?
        }
        Ok(Secret {
            data: T::deserialize_secret(ssecret.secret),
            mec: Default::default(),
            ec: Default::default(),
        })
    }
}

, but you may see one problem here: if encrypted secret is encrypted where is deserialize_secret going to get key from? Unless you are fine with using some kind of global (or thread-local) variable this is not going to work. You can, however, do everything you need by discarding Deserialize implementation on Secret, making deserialize standalone function which accepts key: I think you should do the following here:

Agree that all secrets get serialized into some fixed format (e.g. JSON) before being encrypted and that deserialize() accepts key as its second argument. This way new deserialize function will do the following:

  1. Deserialize to SerializedSecret using provided deserializer.
  2. Check MEC.
  3. Decrypt string.
  4. Create new deserializer for the selected format.
  5. Deserialize decrypted string from to T using that deserializer.

Note that second stage deserializer will be capable of leaking secret data, so fixed format with a second stage deserializer written by you is probably the best choice (example below, using JSON deserializer in place of handwritten). Or, at least, have marker trait SecureDeserializer and require second stage deserializer implement both Deserialize and it.

extern crate serde;
extern crate serde_derive;
extern crate serde_json;
extern crate typenum;

use serde_derive::Serialize;
use serde_derive::Deserialize;
use serde::Deserializer;
use serde::de;

struct Secret<T, MEC: typenum::Unsigned, EC: typenum::Unsigned> {
    data: T,
    mec: MEC,
    ec: EC,
}

#[derive(Serialize, Deserialize)]
struct SerializedSecret {
    secret: String,
    mec: u32,
}

struct Key;

fn decrypt(encrypted_data: String, key: Key) -> Option<String> {
    Some(encrypted_data)
}

fn deserialize<'de, D, T, MEC, EC>(deserializer: D, key: Key)
    -> Result<Secret<T, MEC, EC>, serde_json::Error>
    where
        D: Deserializer<'de>,
        T: de::DeserializeOwned,
        MEC: typenum::Unsigned,
        EC: typenum::Unsigned,
{
    let ssecret: SerializedSecret = de::Deserialize::deserialize(deserializer)
        .map_err(|error| de::Error::custom(error))?;
    if ssecret.mec != MEC::to_u32() {
        Err(de::Error::invalid_type(de::Unexpected::Unsigned(
            ssecret.mec.into()), &"different MEC value"))?
    }
    let decrypted_secret = decrypt(ssecret.secret, key)
        .ok_or_else(|| <serde_json::Error as de::Error>::invalid_value(
            de::Unexpected::Other("invalid string"),
            &"string, which can be decrypted"))?;
    let data = serde_json::from_reader(decrypted_secret.as_bytes())?;
    Ok(Secret {
        data,
        mec: Default::default(),
        ec: Default::default(),
    })
}

This is a beautiful implementation. How would you deal with encrypting the T in Secret<T, MEC, EC> assuming both parties, callers of Serialize and Deserialize, have unfettered access to the encryption key (which may or may not be shared)?

I don't think symmetric encryption will work. You could use a non-pub static in your library to define the key, but if the library is open-source, nothing prevents users from just copying the key into their own code.

I am thinking about to do this like so:

(in pseudocode)

#![cfg(feature = "serializable-secret")]

...

Secret<T: SerializableSecret, MEC, EC>.serialize_secret<EC: EncryptionKeyTrait>(ec: EC) -> SerializableSecret<T: SerializableSecret, MEC, EC>

Then impl serde::Serialize for SerializableSecret<_, _, _>.

The advantages are:

  1. Feature-gated.
  2. Even if feature is enabled, user of the library can choose to only impl SerializableSecret for some-of-their-T (library will impl for common Ts in std).

What do you guys think of this?

At a meta level, remember that (non-lifetime) generics are monomorphized. Thus it will never be possible to deserialize into a type that your application hasn't already monomorphized.

Hi, thank you for your response but would you mind elaborating further on this?