Why extra drop while deallocation

While reading The Rustonomicon, I came across following code snippet for deallocation of vector

impl<T> Drop for Vec<T> {
    fn drop(&mut self) {
        if self.cap != 0 {
            while let Some(_) = self.pop() { }
            let layout = Layout::array::<T>(self.cap).unwrap();
            unsafe {
                alloc::dealloc(self.ptr.as_ptr() as *mut u8, layout);
            }
        }
    }
}

I want to understand why there is a need to drop individual value using ptr::write or ptr::drop_in_place even though we are about to deallocate the complete layout we allocated (like in C). And is this required for every collectible like hashmaps that every entry should be dropped in ptr::write or ptr::drop_in_place before calling dealloc.

With something like a Vec<String>, there are actually multiple levels of allocations to talk about.

First, a String is just a struct with 1x pointer field and 2x usizes (the len and capacity). Because the String contains a pointer to some heap memory, we need to make sure the characters it contains is free'd when a String goes out of scope.

Next, there is the contiguous chunk of memory (self.ptr) that Vec<String> allocates to hold the Strings (remember, a String is just a pointer and 2x usize fields).

If we only free'd self.ptr in Vec<T>'s Drop implementation then we'd free the space all those Strings took up without also freeing the characters (a memory leak!).

By removing each String from the collection in a while let Some(_) = self.pop() {} loop, we can make sure the String's destructor gets called and the heap allocation containing the string's characters is free'd like normal. They could have also used ptr::drop_in_place() if they wanted, but that requires a couple extra lines of unsafe and the std authors probably felt the performance gains weren't worth the extra unsafe.

You'll find a similar story in collections like HashMap, too.

1 Like

Mere untyped deallocation of a chunk of memory would 't run the destructors of the collection's items automatically.

1 Like

so that means dealloc just dissolves the memory layout we assigned like in your example, it would dissolve the array of string fat pointers leaving behind underlying heap stored character array ?

Yes, because the allocator is just a memory allocator. It doesn't know about what other semantics every type has.

There's potentially a lot more to be done in a destructor that has nothing to do with memory management; e.g. a database transaction object might want to auto-commit or auto-rollback upon being dropped, a mutex guard should unlock the mutex in drop(), and so on. This is none of the business of the memory allocator, but it would be very bad if a transaction didn't commit or a mutex didn't unlock just because you put it in a Vec.

2 Likes

This makes sense now. Also the above implementation says in start that we can use needs_drop to check whether it needs dropping or not. So if my data type is returning false then also do I need to call drop_in_place before deallocator ?

Technically you never need to drop things, as leaking memory is safe[1].

But the documentation is really helpful here:

needs_drop is an optimization hint. If it returns false, then that T is guaranteed to have trivial drop glue i.e. dropping it will have no effect and can be safely skipped. However, this should only be done if iterating over multiple objects to drop them has a cost that can be skipped; if it's a fixed number of drop of objects, then drop_in_place should just be called.


  1. Caveats about Pin notwithstanding, but you acknowledged those when you created the pin. ↩ī¸Ž

2 Likes

This is a good thing to point out because LLVM is often smart enough to see when a function (e.g. dropping a !needs_drop type) does nothing and optimise it out (godbolt).

For example, dropping a Vec<u32> generates the following machine code:

example::doesnt_need_drop:
        mov     rax, qword ptr [rdi + 8]
        test    rax, rax
        je      .LBB1_2
        mov     ecx, 4
        mul     rcx
        test    rax, rax
        je      .LBB1_2
        mov     rdi, qword ptr [rdi]
        mov     edx, 4
        mov     rsi, rax
        jmp     qword ptr [rip + __rust_dealloc@GOTPCREL]
.LBB1_2:
        ret

Which is roughly the equivalent of

fn drop(v: Vec<u32>) {
  let ptr = v.as_mut_ptr();
  let capacity = v.capacity()

  if !ptr.is_null() {
    if v.capacity() != 0 {
      let item_size = 4;
      let byte_size =  capacity * item_size;
      unsafe {
        let layout = Layout::from_size_align_unchecked(byte_size, item_size);
        std::alloc::dealloc(ptr, layout); 
      }
    }
  }
}

Even though we know from the source code that there is a while Some(_) = self.pop() {} loop that pops and drops each item, LLVM saw that dropping an u32 does nothing and removed the loop.

2 Likes

Ohh that's quite interesting backend level detail :grinning:.

It is true in general that modern compilers, LLVM included, optimize aggressively. You shouldn't usually perform micro-optimizations by hand; aim for code readability as your first goal, and optimize when required.

7 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.