Rust MessagePack and Serde 1.0

announcements
1

The primary focus of this release is upgrading with Serde up to the first stable 1.0 version. This affects two serialization crates: rmp-serde (a.k.a rmps) and rmpv (MessagePack Value type) which received the following features... well, actually one, but significant, feature.

Zero-copy deserialization

Of course one of the most hot feature of Serde 1.0 is the ability to deserialize into types that borrow from the input data. This ideally fits MessagePack pattern allowing to deserialize bytes with absolute zero-copying.

For example the serialization part (using Python):

>>> s = msgpack.dumps(['John', 'Smith', 42])
>>> s
'\x93\xa4John\xa5Smith*'
>>> ', '.join(map(lambda x: '{:#2x}'.format(x), array.array('B', s)))
'0x93, 0xa4, 0x4a, 0x6f, 0x68, 0x6e, 0xa5, 0x53, 0x6d, 0x69, 0x74, 0x68, 0x2a'

And the deserialization part from Rust:

let buf = [0x93, 0xa4, 0x4a, 0x6f, 0x68, 0x6e, 0xa5, 0x53, 0x6d, 0x69, 0x74, 0x68, 0x2a];

#[derive(Debug, PartialEq, Deserialize)]
struct Person<'a> {
    name: &'a str,
    surname: &'a str,
    age: u8,
}

assert_eq!(Person { name: "John", surname: "Smith", age: 42 }, rmps::from_slice(&buf[..]).unwrap());

And of course zero-copy deserialization into ValueRef type and from it has also landed:

For example the same bytes into ValueRef:

let buf = [0x93, 0xa4, 0x4a, 0x6f, 0x68, 0x6e, 0xa5, 0x53, 0x6d, 0x69, 0x74, 0x68, 0x2a];

assert_eq!(ValueRef::Array(vec![ValueRef::from("John"), ValueRef::from("Smith"), ValueRef::from(42)]),
    rmps::from_slice(&buf[..]).unwrap());

And the ability to unpack ValueRef into a Deserialize type.

#[derive(Debug, PartialEq, Deserialize)]
struct Person<'a> {
    name: &'a str,
    age: u8
}

assert_eq!(Person { name: "John", age: 42 },
    deserialize_from(ValueRef::Array(vec![ValueRef::from("John"), ValueRef::from(42)])).unwrap());

What's next

As there are no unstable dependent crates left, it's time to stabilize ourselves. First of all, the core crate - rmp - is fully documented and almost stable - I don't think there will be breaking changes. Speaking of other two crates - rmps and rmpv - they need some love, review, documentation and, of course, production-experience (which is in progress with my next I/O crate, but it's completely different history...).

The repository is on github.

21 Likes
2

This is huge news! We pack / unpack millions of messages a minute through different parts of our data pipeline so I can't wait to get this in production.

8 Likes