Read_unaligned of packed vector in a struct crashes program

I'm very new to rust and have run into this situation. The program listed below, crashes due to:
"free(): double free detected in tcache 2". I've tried this reading from primitives inside the
struct and that does not cause the issue.

Any ideas?

/**
 * We want to store the data obtained from a disk file. The file consists
 * of a bunch of bytes whose structure is defined in an external spec.
 * This struct is a small example of what's in that spec. The spec does
 * not include any padding between the tag and a_vec.
 */
#[repr(packed)]
struct S{
    tag : u8,
    a_vec : Vec<u8>
}

/**
 * in this example we build the input from an array but in the real
 * program, it would come from a disk file.
 * 
 */
fn main() {
    let a_array : [u8;11] = [0x5, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3A];
    let a_vec = a_array.to_vec();
    process(&a_vec);
}

/**
 * After loading up a byte vector (e.g. from a disk file) here we want
 * to have our way with it.  In particular, we want to put it into a struct
 * along with it's first byte value, its length, and the first word of it.
 * 
 * @v : a vector containing the bytes we want to process
 */
fn process(v : &Vec<u8>){
    let my_s = S{
        tag : v[0],                             // the first byte is a tag
        a_vec : get_bytes(&v, 1, v.len()-1)     // the next bunch of bytes is data
    };

    //  this works because the first byte is aligned
    println!("tag = {:x?}",my_s.tag);

    //  but the vector is not so we have to fetch it using
    //  read_unaligned.

    //  This println fails due to "reference to packed field is unaligned"
    //      println!("doing something with a_vec {:x?}", my_s.a_vec);
    //  so we try this instead. But when the program ends we get:
    //
    //      free(): double free detected in tcache 2
    //
    let unaligned = std::ptr::addr_of!(my_s.a_vec);
    let v = unsafe { std::ptr::read_unaligned(unaligned)};
    println!("doing something with a_vec {:x?}", v);

}


/**
 * Get a subset of a vector of bytes and return it as a vector of bytes
 * 
 * @buf : is a vector containing a bunch of bytes which could have come from a file
 * @start_index : first byte of the subset
 * @length : how many bytes to get from start_index
 */
fn get_bytes (buff : &Vec<u8>, start_index : usize, length : usize) -> Vec<u8>{
    let mut rtn : Vec<u8> = Vec::new();
    let mut 
    i : usize = 0;
    while i < length {
        rtn.push(buff[start_index+i]);
        i += 1;
    }
    rtn
}

(Playground)

Output:

tag = 5
doing something with a_vec [31, 32, 33, 34, 35, 36, 37, 38, 39, 3a]

Errors:

   Compiling playground v0.0.1 (/playground)
    Finished dev [unoptimized + debuginfo] target(s) in 5.32s
     Running `target/debug/playground`
free(): double free detected in tcache 2
timeout: the monitored command dumped core
/playground/tools/entrypoint.sh: line 11:     8 Aborted                 timeout --signal=KILL ${timeout} "$@"

I am not that confident in writing correct unsafe code myself, so my opinion might be wrong.
Looking at the documentation of read_unaligned it is mentioned that

Like read , read_unaligned creates a bitwise copy of T , regardless of whether T is Copy . If T is not Copy , using both the returned value and the value at *src can violate memory safety.

To me it seems that the bitwise copy of the Vec gets dropped and later the original as well, thus creating the problem you are seeing.

2 Likes

That is exactly what the "double free" in the error message means.

Thank you for your very timely response.

I'm still struggling, however (the problem with not entirely grasping rust terminology).

Why would this issue not arise when doing the same thing with a primitive such as a u16 (I have tried this same code replacing the vector with a u16 and the program did not crash)? In other words, what is the difference between a Vec and a u16 within this context?

I'm relatively certain (and embarrassed) that I have missed something really fundamental here.

A primitive doesn't have any associated resources that would need to be managed; it is not Drop. A Vec, however, has to deallocate its heap buffer upon Dropping.

Thank you. I appreciate your help. And maybe this is a new topic but, here goes anyway... (I have looked around for the answer to this question and have not hit upon a solution):

How should one fetch the contents of an unaligned vector from inside a packed struct? Is that even possible?

One thing that might help is to use

#[repr(packed)]
struct S{
    tag : u8,
    a_vec : ManuallyDrop<Vec<u8>>
}

Because if you real_unaligned the ManuallyDrop<Vec<u8>> you don't have to remember to forget it to avoid the double-drop.

(Basically, you switch the "if you get it wrong" from double-frees -- which are really bad -- to just memory leaks.)

1 Like

I don't know what I'm missing here.. having a Vec<u8> inside a packed struct seems like nonsense to me. In what sense are you reading the vec value from a file? How are those pointers valid? :slightly_smiling_face:

4 Likes

@scottmcm: thanks for that suggestion, I'll give it a try.

@bluss: As a "learning exercise" I decided to write a program to read (and eventually write) Java .class files. You can see the specs for the format here:

https://docs.oracle.com/javase/specs/jvms/se17/html/jvms-4.html

As an example the spec says:

attribute_info {
    u2 attribute_name_index;
    u4 attribute_length;
    u1 info[attribute_length];
}

is how the class file stores attributes for methods and fields. I modeled it as:

#[repr(packed)]
pub struct AttributeInfo {
    pub attribute_name_index: u16,
    pub attribute_length: u32,
    pub attribute_info:Vec<u8>
}

I wasn't expecting to do anything earth shattering but it seemed like a "fun" exercise.

I think bluss is right here -- a Vec is not layout-compatible with that at all. A vector is a pointer and two lengths, with the actual data behind the point. Whereas that structure from the spec has the data inline.

So since you can't directly read the Vec from the file anyway, there's no need to have packed on the struct holding it.

2 Likes

Thank you all for your kind help and suggestions.

I clearly need to revisit my approach.

I had naively thought I could build the exact file layout as a bunch of structs, populate them with what I wanted to be in the class file and then just (just!!!! -- ha ha) write it out to the file.

I think this topic is closed now. I hope I didn't waste too much of your time.

Cheers!

I'd give it one more try without packed structs. (But with the right serializer you can basically do it, just not exactly that way).

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.