What purpose does the crate `bincode` serve in binary serialization that `serde` does not?

[thor314]

I've been looking at the bincode crate, which one of my upstream dependencies uses. It seems to me to be a less featureful version of the serde crate, and I'm unsure why one would use it. Here's an example, the rust playground doesn't support bincode.

# In toml: 

[dependencies]
bincode = "1.3.3"
serde = "1.0.136"
serde_json = "1.0.79"

#main.rs
use std::mem::size_of_val;

fn main() {
    let target = Some("hello_world".to_string());
    let encoded_b: Vec<u8> = bincode::serialize(&target).unwrap();
    let encoded_s = serde_json::to_string(&target).unwrap();

    // bytes will be harder to read:
    dbg!(&encoded_b, &encoded_s);
    // these are the same size, 24 bytes:
    dbg!(size_of_val(&encoded_b), size_of_val(&encoded_s));

    let target_b: Option<String> = bincode::deserialize(&encoded_b).unwrap();
    let target_s: Option<String> = serde_json::from_str(&encoded_s).unwrap();

    dbg!(target, target_b, target_s);
}

[Michael-F-Bryan]

JSON is an inefficient format, especially when you need to pass around large amounts of binary data like numbers. If you know you don't need a self-describing format and care more about performance (e.g. because you are implementing message passing in Firefox/Servo) then bincode is a much more appropriate format to use.

[LegionMammal978]

bincode is a format that works on all types that implement serde traits; that is, bincode is not competing with serde, but instead JSON. In this case, bincode results in 20 bytes of data, whereas serde_json takes only 13 bytes for this string. The main advantage of bincode over JSON is that bincode is much faster to encode and decode for large enough data structures, and it can be more easily streamed.

Also, the 24 bytes you get from size_of_val() doesn't measure the actual length of the data. For that, you should just use encoded.len(). Instead, size_of_val() is measuring the size of the Vec itself, which is effectively a (ptr, len, capacity) triple.

[H2CO3]

You are confusing serde with serde-json. Serde is not a JSON serialization library. It is a generic, i.e. format-agnostic and data structure-agnostic serialization library. Here's what a traditional JSON library looks like:

+-------------------+
| Dynamically-typed |
| JSON value tree:  |    +--------------+    +----------------------+
| null, boolean,    | -> | JSON encoder | -> | JSON string          |
| numbers, string,  |    | (black box)  |    | {"foo": [42, "bar"]} |
| array, object     |    +--------------+    +----------------------+
+-------------------+

In contrast, here's what Serde wants to be:

                             +-------------------------------------------------+                              Nope, still
        Not Serde            |                         Serde                   |        Also not Serde         not Serde
+-----------------------+    |+-----------------------+    +------------------+|    +-------------------+    +----------+
| Strongly-typed custom |    || Generic value tree    |    |                  ||    | Data Format       |    |          |
| data structure,       | -> || (optionals, booleans, | -> | Serializer trait || -> | (impl Serializer) | -> | Whatever |
| impl Serialize        |    || strings, sequences, … |    |                  ||    |                   |    |          |
+-----------------------+    |+-----------------------+    +------------------+|    +-------------------+    +----------+
                             +-------------------------------------------------+

Serde doesn't know how to serialize your own structs and enums and it also doesn't know what to serialize them to. Serde has no support for MyAwesomeCustomType and it doesn't have support for JSON or YAML or Bincode or XML or BSON or MessagePack or ProtoBuf or ASN.1.

Serde is merely an abstraction layer that provides a unified interface (Serialize and Deserialize for data structures, and Serializer and Deserializer for data formats) and a simple tree-like type system which in turn allows arbitrary data structures and serialization formats to communicate. (It also provides derive macros for implementing Serialize and Deserialize on your own data types.) It doesn't do any actual serialization or deserialization itself.

The actual serialization and deserialization happens by data formats, which are implemented in separate crates. There is serde-json for JSON, of course, but the other formats have their own crates, too. Bincode is one of them.

Bincode is not a less featureful version of serde, just like a drill bit is not a less featureful version of a drill. It is part of the drill-as-a-whole-equipment. The drill bit is useless in itself, because it needs an electric motor to turn it, but the motor is useless without the drill bit, because it can't do the actual drilling without the bit. Serde is the motor of the drill, and JSON and Bincode are two differently-shaped and differently-sized drill bits.

Serde can't do serialization in itself. Nor can bincode, nor can serde-json, nor can your own data structures by themselves. The data structures tell serde what to serialize (i.e. the tree of sequences, maps, and primitives, which are universally-recognized data types), and then serde passes on this knowledge to the particular serialization format, which then knows how to serialize it. Neither counts as complete serialization on its own, and all three parts of the system must work together in order to have useful serialized data come out at the end.