How does zero-copy deserialization work?

A little introduction, So that we are all on the same page.

From Wikipedia:

" Zero-copy " describes computer operations in which the CPU does not perform the task of copying data from one memory area to another or in which unnecessary data copies are avoided.

What kind of types, That can be deserialization without copying every bits. hence zero-copy ?

And what are those methods to get reference of underlying data (bytes) ?
I know few:

  • Reference or pointer
  • *::from_raw_parts
  • mem::transmute
  • pointer::cast

Is there more procedures for zero-copy deserialization ?

This helped me understand zero-copy deserialization. It assumes you know the basics of the language.

https://serde.rs/lifetimes.html

I’m sure the other people here that know more can be more helpful.

1 Like

But serde represent very simple datatype, for example &str or &[u8]

Is there any way to deserialization a struct without cloning ?

For example:

struct Foo { x: u8, y: u16, }

let bytes = [0_u8; 16];
// How to get `&Foo` from `bytes` ?
let foo: &Foo = ??? ;

Transmute could be an option ? For example:

use std::mem::transmute;
#[repr(C)]
struct Foo { x: u8, y: u16, }

fn main() {
    let mut bytes = [0; 3];
    let foo = unsafe { transmute::<&mut [u8; 3], &mut Foo>(&mut bytes) };
    foo.x = 123;
    foo.y = 456;
    println!("{:?}", bytes); // [123, 0, 201]
}

But It does not respect Endianness or Convert from bytes (&[u8])...

Writing your own serde deserializer would allow you to decide what endianness to use when making these conversions. I did that myself once and it’s surprisingly easy, but I used bincode in the end.

If you need to convert endianness, then you cannot use zero-copy.

7 Likes

Well, for example if you're dealing with contiguous 32-bit big-endian integers you can zero-copy into a &[MyInt], where MyInt is a repr(transparent) wrapper around [u8; 4] with a getter that calls u32::from_be_bytes.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.