Partial dealloc

help
#1

Hello,

I am trying to do a particular optimization, where I deallocate part of a block of memory, and give the ownership of the rest to something like a Vec:

struct MyStruct {
    // the length is also contained in here in the first bytes
    ptr: NonNull<u8>,
}

impl Into<Vec<u8>> for MyStruct {
    fn into(self) -> Vec<u8> {
        unsafe {
            let length = *(self.ptr.as_ptr() as *mut usize);
            let ptr = self.ptr.as_ptr().add(size_of::<usize>());
            // somehow only deallocate the part that contains length
            Vec::from_raw_parts(ptr, length, length)
        }
    }
}

I tried using something like

let layout = alloc::Layout::from_size_align_unchecked(
    size_of::<usize>(),
    align_of::<usize>(),
);
alloc::dealloc(self.ptr.as_ptr(), layout);

But miri complained that the Layout was incorrect, with something like incorrect layout on deallocation: alloc3805 has size 29 and alignment 8, but gave size 8 and alignment 8. So if miri is right, is there something in the alloc API that can achieve this ? I am guessing not, since I didn't find anything like this in std, but who knows...

#2

You can't partially dellocate an allocation. You'll need to copy over the bytes to safely construct the Vec

#3

Isn't this realloc? (Provided that the layouts match, which is not obvious from your description. If they don't, then I don't think you can re-use memory for different types like that.)

#4

Yeah, I am afraid that the Layouts would be in conflict here.
And even if they weren't, realloc does not guarantee to reuse memory, right ? I think in this case where the part I want to throw is at the beginning, it would likely not do what I want :frowning:

#5

I think that is true, yes. However, for shrinking an allocation, I'd expect it to reuse the memory. However, you said you would use this as an optimization only – but are you actually relying on this property for correctness as well?

#6

No, no, this is simply me trying to overoptimize something :slightly_smiling_face:

#7

Hmm, that's a shame... I wonder if this is simply a limitation, or if there is something deeply dangerous in this ?

#8

In this case, if you are worried about the number of allocations being too high, you might be interested in an arena. There are a couple of crates for this purpose, for example, bumpalo.

1 Like
#9

While many allocators allow you to shrink things, no allocator that I'm aware of allows you to shrink an allocation from the "beginning" because the starting address is usually how the allocator keeps track of memory internally (and Rust's allocator API also assumes this). Similarly, you can't grow an allocation "backwards" either. It's not dangerous per se, it's merely impossible.

3 Likes
#10

Ralloc (the Redox OS memory allocator) allows arbitrary partial deallocation.

5 Likes