Serialize a Vec<u8> to JSON as base64

[AndreKR]

I'm serializing a struct to JSON by passing it to Rouille's Response::json() method, which calls

let data = serde_json::to_string(content).unwrap();

on it. I don't have control over this line, as it comes from the Rouille crate.

In my struct I have a Vec<u8> (containing a JPEG image):

pub struct Status {
	name: String,
	#[serde(skip_deserializing)]
	pub map: Option<Vec<u8>>,
}

By default the struct is encoded as {"name": "Foo", "map": [255, 216, 255, 224, 0, ...]}.

This is a bit problematic to deal with in my client application, so I'd like to encode it as base64, a relatively common standard when packing binary data into a JSON.

I tried the crate serde_bytes like this:

pub struct Status {
	name: String,
	#[serde(skip_deserializing, with="serde_bytes")]
	pub map: Vec<u8>,
}

but that seems to be doing nothing. (I removed the Option just in case that causes the issue.)

I guess I'm looking for something like #[serde(with="base64")]? Is there a crate that provides it?

[alice]

I'm not aware of a crate that does it, but defining a custom module for it is pretty easy.

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize)]
pub struct Status {
	name: String,
	#[serde(with="base64")]
	pub map: Vec<u8>,
}

mod base64 {
    use serde::{Serialize, Deserialize};
    use serde::{Deserializer, Serializer};

    pub fn serialize<S: Serializer>(v: &Vec<u8>, s: S) -> Result<S::Ok, S::Error> {
        let base64 = base64::encode(v);
        String::serialize(&base64, s)
    }
    
    pub fn deserialize<'de, D: Deserializer<'de>>(d: D) -> Result<Vec<u8>, D::Error> {
        let base64 = String::deserialize(d)?;
        base64::decode(base64.as_bytes())
            .map_err(|e| serde::de::Error::custom(e))
    }
}

[AndreKR]

Works nicely.

Different (related) question... is there any way to keep the Option? I.e. specify the with="base64" for the value inside the Option?

If not, using an empty Vec to represent "not there" works alright.

[alice]

Sure, just adjust the types.

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize)]
pub struct Status {
	name: String,
	#[serde(with="base64")]
	pub map: Option<Vec<u8>>,
}

mod base64 {
    use serde::{Serialize, Deserialize};
    use serde::{Deserializer, Serializer};

    pub fn serialize<S: Serializer>(v: &Option<Vec<u8>>, s: S) -> Result<S::Ok, S::Error> {
        let base64 = match v {
            Some(v) => Some(base64::encode(v)),
            None => None,
        };
        <Option<String>>::serialize(&base64, s)
    }
    
    pub fn deserialize<'de, D: Deserializer<'de>>(d: D) -> Result<Option<Vec<u8>>, D::Error> {
        let base64 = <Option<String>>::deserialize(d)?;
        match base64 {
            Some(v) => {
                base64::decode(v.as_bytes())
                    .map(|v| Some(v))
                    .map_err(|e| serde::de::Error::custom(e))
            },
            None => Ok(None),
        }
    }
}

[steffahn]

This should be a way to avoid unnecessary extra allocation of an intermediate string (for both serialization and deserialization).

use serde::{de::Error, Deserialize, Deserializer, Serialize, Serializer};
use std::str;

#[derive(Serialize, Deserialize, Debug)]
pub struct Status {
    name: String,
    pub map: Option<Base64>,
}

#[derive(Debug)]
pub struct Base64(Vec<u8>);
impl Serialize for Base64 {
    fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
        serializer.collect_str(&base64::display::Base64Display::with_config(
            &self.0,
            base64::STANDARD,
        ))
    }
}

impl<'de> Deserialize<'de> for Base64 {
    fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
        struct Vis;
        impl serde::de::Visitor<'_> for Vis {
            type Value = Base64;

            fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result {
                formatter.write_str("a base64 string")
            }

            fn visit_str<E: Error>(self, v: &str) -> Result<Self::Value, E> {
                base64::decode(v).map(Base64).map_err(Error::custom)
            }
        }
        deserializer.deserialize_str(Vis)
    }
}

you can adapt this into a #[serde(with=…)] based solution if you want the field to still be a Vec. (You can also just use this Base64 struct in a #[serde(with=…)] based solution.)