Does allocating without using actually take up memory?

I wrote a tool that runs in multiple modes, some requiring a buffer, others do not. The buffer is created before mode-specific logics, meaning that the buffer is created whether it is used or not. The buffer is created using vec![0u8; buf_len], which calls "allocate zeroed" in the background. On x86_64-pc-windows-msvc, I observed that when running in modes that don't use a buffer, the taskmgr shows that the memory usage is less than the size of the buffer, i.e. the memory allocated for the buffer is not actually being used. Is this behavior consistent across all platforms that can use std? Do I need to add logic that determines whether a buffer is needed and thus whether it is created? I think this may be an OS internal level optimization, so the behavior is not guaranteed.

Optimizations like that could come from multiple places

  • The compiler / LLVM
  • Or like you said, the OS (might give you a fake page and then upon fault when you try to read or write it, actually find the memory for you... or OOM you)

Neither are guarantees. Whether it's worth it to add your own logic depends on your use case (maybe an actual allocation wouldn't matter in practice) and how comfortable you are relying on non-guaranteed optimizations (if it would matter).

2 Likes

vec![0u8; size] is special in that it will allocate memory initialized to zero (on Windows, it's calling HeapAlloc with HEAP_ZERO_MEMORY flag set).

Windows when dealing with large allocations may not actually commit the memory pages until you actually access them.

As mentioned above, neither of those are guarantees. Rust may decide to not special-case allocations for elements whose representation is all zero, and Windows at any point can decide to change its allocator behavior.

4 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.