Zero copy deserialization works for slice of slices but segfaults for slice of structs

[panicfarm]

In the first part of the code below, I successfully zero-copy serialize and deserialize &[&[u64]] to/from a byte array.

I try to do the same for a slice of structs &[Out] that is very similar to &[&[u64]] but it segfaults.

This is probably because Out contains a pointer (a memory address) that I don't control, that gets serialized to sl_u8
Is there a way to encode the correct pointer addresses in the byte array that point to the correct place within the same byte array sl_u8? I started doing pointer arithmetic ptr.offset() during the deserialization instead, but I wonder if I can do this better.

#[derive(Debug, Eq, PartialEq)]
#[repr(C)]
pub struct Out<'a> {
    indx: &'a [u64],
}

fn main() {
    //Slice of slices &[&[u64]] works
    let s1: &[u64] = &[3, 7, 8];
    let s2: &[u64] = &[8, 87843];
    let s3: &[u64] = &[439];

    let sl: &[&[u64]] = &[s1, s2, s3];

    //serialization
    let ptr = sl as *const _ as *const u8;
    let len = sl.len() * std::mem::size_of::<&[u64]>();
    let sl_u8 = unsafe { std::slice::from_raw_parts(ptr, len) };
    println!("byte array {:?}", sl_u8);

    //zero copy deserialization
    let ptr = sl_u8 as *const _ as *const &[u64];
    let len = sl_u8.len() / std::mem::size_of::<&[Out]>();
    let deser_slice = unsafe { std::slice::from_raw_parts(ptr, len) };

    assert_eq!(deser_slice, sl);
    println!("Success. Deserialized slice {:?}", deser_slice);

    //Slice of structs &[Out] that is very similar to &[&[u64]] segfaults
    //probably because Out contains a pointer (a memory address) that I don't control, that gets serialized to sl_u8
    //Is there a way to do this?
    let indx: &[u64] = &[3, 7];
    let sl_out = &[Out { indx }];

    //serialization
    let ptr = sl_out as *const _ as *const u8;
    let len = std::mem::size_of::<Out>() * sl_out.len();
    let sl_u8 = unsafe { std::slice::from_raw_parts(ptr, len) };
    println!("byte array {:?}", sl_u8);

    //zero copy deserialization (segfaults)
    let ptr = sl_u8 as *const _ as *const &[Out];
    let len = sl_u8.len() / std::mem::size_of::<Out>();
    let deser_slice = unsafe { &std::slice::from_raw_parts(ptr, len) }; //segfault here

    println!("Success. Deserialized slice {:?}", deser_slice);
}

(Playground)

Output:

byte array [8, 128, 43, 149, 133, 85, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 32, 128, 43, 149, 133, 85, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 64, 128, 43, 149, 133, 85, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]
Success. Deserialized slice [[3, 7, 8], [8, 87843], [439]]
byte array [48, 128, 43, 149, 133, 85, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0]

Errors:

   Compiling playground v0.0.1 (/playground)
    Finished dev [unoptimized + debuginfo] target(s) in 0.66s
     Running `target/debug/playground`
timeout: the monitored command dumped core
/playground/tools/entrypoint.sh: line 11:     7 Segmentation fault      timeout --signal=KILL ${timeout} "$@"

[mbrubeck]

The elements of sl_out have type Out rather than &[Out], so you should Out when doing size calculations or constructing pointers to elements:

//serialization
let ptr = sl_out as *const _ as *const u8;
let len = std::mem::size_of::<Out>() * sl_out.len();
let sl_u8 = unsafe { std::slice::from_raw_parts(ptr, len) };
println!("byte array {:?}", sl_u8);

//zero copy deserialization
let ptr = sl_u8 as *const _ as *const Out;
let len = sl_u8.len() / std::mem::size_of::<Out>();
let deser_slice = unsafe { &std::slice::from_raw_parts(ptr, len) }; //segfault here

Playground

[panicfarm]

Thank you, I corrected it (stupid mistake), but it still segfaults, because in this case they are the same:

[src/main.rs:45] std::mem::size_of::<Out>() = 16
[src/main.rs:45] std::mem::size_of::<&[Out]>() = 16

[mbrubeck]

I pasted the wrong playground link above; here's the corrected link: Playground.

Make sure you have corrected the pointer type as well, so that you are not trying to deserialize to &[&[Out]].

When working with raw pointer casts and from_raw_parts, it can also be useful to annotate more types than strictly necessary, so the compiler will catch this problem for you:

let deser_slice: &[Out] = ...

[panicfarm]

Awesome, thanks! That did it.

When working with raw pointer casts and from_raw_parts , it can also be useful to annotate more types than strictly necessary, so the compiler will catch this problem for you:

That's a very good advice, it's easy to get confused like I was