Box<T> vs `std::alloc::{alloc, dealloc}` (pad to align?)

help
1

Just to make sure I'm not misunderstanding any of it.

Is there any difference in how the Box<T>'s #[rustc_box]:

pub fn new(x: T) -> Self {
    #[rustc_box]
    Box::new(x)
}

and std::alloc::alloc handle allocations, assuming the default A: Allocator = Global?

Can you recreate a Box<T> on your own with a call to alloc in new() and dealloc in Drop? Or is the Box further enhanced/optimized in the background? In particular, I might have seen a few cases in which the Layout, after an extend, was pad_to_align()-ed. Yet his didn't seem to be (always?) needed for any compile-time known T. What's going on there?

Context: looking into manual memory management, for fun and profit learning.

2

If your question can be understood as "can I make a pointer that is also a valid Box from scratch this way", then https://doc.rust-lang.org/std/boxed/index.html#memory-layout should answer that: yes.

Are you sure you saw this in Box, not perhaps Rc? Box always uses exactly the layout of the contained value. It has no reason not to.

3

Must have missed that one, thanks.

Quite possible. Slightly off-topic (Box/alloc-wise), yet still: not sure I'm fully aware of when is it necessary to pad_to_align() a given Layout about to be allocated or not, if ever. When manually constructing a tuple or a struct with extend()? What would happen if were to try and dealloc a wrong sized Layout from a given pointer, later on: memory leak, undefined behaviour, both?

4

pad_to_align() is not, as far as I know and the allocator documentation says, an operation that is relevant to allocation. Types and values in the Rust language have the rule that “The size of a value is always a multiple of its alignment.” pad_to_align() makes a Layout compliant with that rule. So, it's relevant to building up a layout from parts — like a struct from its fields — and making sure that the result matches what you'd get from, say, a #[repr(C)] struct definition.

If you get the Layout from a type or value with Layout::new() or Layout::for_value(), then unless you modify it first, pad_to_align() will do nothing because all types’ layouts are already padded to their aligment.

If you call the global dealloc() with possibly a different layout than alloc(), you have violated its safety requirements, i.e. you have written unsound code:

Safety

The caller must ensure:

  • ptr is a block of memory currently allocated via this allocator and,
  • layout is the same layout that was used to allocate that block of memory.

Now, any trait implementation may promise more than the trait, so if you happen to know the specific allocator in use doesn’t care, it could be fine — but that would be an abnormal situation. Ordinary Rust code performing allocation must use the same layout for deallocation as for allocation.

4 Likes
5

You can't really duplicate a Box<T> in userspace code. That's why the rustc_box attribute exists. There are several issues I know of:

  • Box<T> is assumed to be a unique pointer to its allocation, in the same way as &mut T is. This may or may not be true in the future (there is talk about removing that guarantee), but it can't be enabled for userspace smart pointers.
  • There is a bunch of general smart pointer magic, like unsizing and using Box<Self> as method receivers. Some of that may be stabilized with RFC 3621, which is accepted.
  • Box<T> is marked #[fundamental] which allows to write blanket userspace trait impls which wouldn't be valid for any other type. I don't think there is any current plan to make this feature available in user code (see issue 29635).
  • Box<T> uses the dropck eyepatch #[may_dangle], which affects the borrow checker in subtle ways. I'm not aware of current plans to stabilize it.
  • Box<T> employs some compiler magic, which makes it act like "a local variable on a heap". Specifically, you can move the contents out of the Box without dropping the Box itself, and later put something back in (or not, and just drop an empty box). That's related to the DerefMove and &move ideas. No other type uses that magic, inclduing those in the stdlib.

But specifically to your question, none of that magic is related to the interaction with the global allocator. You can't fully recreate Box, but you can write something sufficiently Box-like, even if nonergonomic. Since the allocator API is not stable, crates for custom arena allocators often declare their own Box, Vec and String types.

1 Like
6

You might have seen that in Rc, where it's doing it so that it matches the internal-implementation-detail RcInner<T> type.

It's only necessary because the current Rc implementation wants to get &RcInner<T>s internally, which would be illegal if the allocated size didn't match what Rust expects for the type.

There's actually in-progress work (Make `Rc<T>::deref` and `Arc<T>::deref` zero-cost by EFanZh · Pull Request #132553 · rust-lang/rust · GitHub) to change the implementation to no longer do that, and that change as a result also removes the pad_to_align -- it means that if all it needs for the allocation is (size: 17, align: 4), that's what it'll ask for. And that's fine since it never tries to access that whole thing as one rust type; it changes the internals to stay in pointer-land until it's accessing one or the other internal sub-type Rust types.

(Now, the allocator probably still rounds up the request, but that's its business. If it doesn't need to, great.)

1 Like