How to deserialize a base64-encoded string into a generic type?

[avsaase]

I also posted this question on r/rust but I figured this is a better place to ask questions like this

I have a struct that I need to deserialize from JSON. The struct has an optional generic field that arrives as a base64 encoded string. I'm trying to create a custom deserializer that both handles the base64 decoding and the deserialization into a concrete type.

My current attempt based on this issue is:

use base64::Engine;
use serde::Deserialize;
use serde::Deserializer;
use serde::Serialize;

#[derive(Serialize, Deserialize)]
pub struct Message<T> {
    #[serde(deserialize_with = "from_base64")]
    pub data: Option<T>,
}

fn from_base64<'a, D, T>(deserializer: D) -> Result<T, D::Error>
where
    D: Deserializer<'a>,
    T: Deserialize<'a>,
{
    use serde::de::Error;
    String::deserialize(deserializer).and_then(|string| {
        base64::engine::general_purpose::STANDARD
            .decode(&string)
            .map_err(|e| Error::custom(e.to_string()))
            .and_then(|bytes| {
                serde_json::from_slice(&bytes).map_err(|e| Error::custom(e.to_string()))
            })
    })
}

This doesn't work like I want for two reasons:

1. It gives this error:

error[E0277]: the trait bound `T: Deserialize<'_>` is not satisfied
  --> src/main.rs:6:21
   |
6  | #[derive(Serialize, Deserialize)]
   |                     ^^^^^^^^^^^ the trait `Deserialize<'_>` is not implemented for `T`, which is required by `std::option::Option<T>: Deserialize<'_>`
   |
   = note: required for `std::option::Option<T>` to implement `Deserialize<'_>`

The compiler suggests to add a Deserialize trait bound on the generic parameters. If possible I would like to avoid doing that because this type is used in other types which requires adding the trait bound to all of those types as well.

2. The inner data field is deserialized specifically from JSON and not a generic transport format. Is there a way to keep it generic over the data format?

---

After posting the question on Reddit I realized that the trait bound error originates from the T: Deserialize<'a> trait bound on from_base64. This trait bound is needed to call serde_json::from_slice but I actually don't want to call this function at all in from_base64 because I want to keep it generic over de data format in case I switch to another format like msgpack.

I rewrote from_base64 like this but I don't know how to finish the function.

fn from_base64<'a, T, D>(deserializer: D) -> Result<Option<T>, D::Error>
where
    D: Deserializer<'a>,
{
    use serde::de::Error;

    let Some(base64) = Option::<String>::deserialize(deserializer)? else {
        return Ok(None);
    };

    let bytes = base64::engine::general_purpose::STANDARD
        .decode(&base64)
        .map_err(|e| Error::custom(e.to_string()))?;

    // How to continue?
}

[drewtato]

The trait bound you actually want is T: for<'a> Deserialize<'a>. Serde provides an equivalent trait called DeserializeOwned, so it can be written as T: DeserializeOwned.

use base64::Engine;
use serde::de::DeserializeOwned;
use serde::Deserialize;
use serde::Deserializer;
use serde::Serialize;

#[derive(Serialize, Deserialize)]
pub struct Message<T> {
    #[serde(
        deserialize_with = "from_base64",
        bound(deserialize = "T: DeserializeOwned")
    )]
    pub data: Option<T>,
}

fn from_base64<'a, D, T>(deserializer: D) -> Result<T, D::Error>
where
    D: Deserializer<'a>,
    T: DeserializeOwned,
{
    use serde::de::Error;
    let string = String::deserialize(deserializer)?;
    let bytes = base64::engine::general_purpose::STANDARD
        .decode(&string)
        .map_err(Error::custom)?;
    serde_json::from_slice(&bytes).map_err(Error::custom)
}

I also cleaned up your function, but should still be functionally identical.

As for using another deserializer for the base64: That's a bit more complex, and probably requires you to make another trait for "deserializer that works on slices".

[drewtato]

I went ahead and fixed the Option problem, and implemented Serialize. This is much simpler if you create a wrapper type that you can use in the functions, and even simpler if you stick the wrapper type directly into the struct.

Rust Playground

The only difference between to_base64 and to_base64_opt is that opt will create an actual null when serializing None instead of a base64 encoded string containing null. This is irrelevant if you have skip_serializing_if = "Option::is_none".

The only difference between from_base64 and from_base64_opt is that opt will deserialize null as None instead of erroring. This is irrelevant if your data doesn't have null, for example, if your data is created by serializing the same struct with skip_serializing_if = "Option::is_none".