Version of AtomicPtr that can handle Unsized boxed Types

1

I am trying to consolidate my memory allocations but for that I need unsized types (aka FatPointers).
Because they are unsized I need to Box them. Is there a Version of Box that stores the size at the allocation so that when I convert the Box into a raw Pointer I get a regular slim pointer that can be use with Atomic CAS and later restore the Box using the size stored at the allocation?

2

The simplest version is to use a Box<Box<dyn Foo>>. Otherwise you could use a boxed enum (if a finite set of types is used) or even use the more advanced Thin Trait Objects pattern.

It'd be really nice if platforms exposed atomic instructions for an i128, but that doesn't sound possible:

https://stackoverflow.com/questions/18177622/how-to-atomically-add-and-fetch-a-128-bit-number-in-c

3

The Problem of boxing twice is that I am not achieving the goal of consolidating my allocations.
I don't even think that we would need double pointer sized atomics as long as we store the size with the box (which the global allocator probably does internally anyways).

4

That's essentially the approach taken with thin trait objects. You move the vtable (which includes pointers to trait object methods, drop code, and the object's std::alloc::Layout) into the allocation, then "forget" the original allocation's size.

I don't think you can normally tap into the global allocator. Maybe you could switch the global allocator to the allocator from jemallocator and use their usable_size() function?

5

That's an implementation detail of the C malloc(3) based allocators. They need to store the size of the allocation since the malloc(3) API doesn't provide such information. The Rust's GlobalAlloc API guarantees that the size of allocation is passed to the deallocation so it's possible to write an allocator without store them.

6

don't think you can normally tap into the global allocator.

I would be even fine storing the size again.

That's essentially the approach taken with thin trait objects.

I just skimmed over it, but I am not sure if it fit my UseCase. Because not only have I to erase the type information but I also have to restore it again. I could switch everything over to vtables but I am not really ready doing that either, just because I want to temporarily store it in a LockFree Data Structure.

7

Could you use something like this for inspiration?

#[repr(C)]
struct VTable {
  layout: Layout,
  type_id: TypeId, // for downcasting back to the original type
}

#[repr(C)]
struct Repr<T> {
  vtable: VTable,
  data: T,
}

fn unsize<T: 'static>(data: T) -> *mut VTable {
  let vtable = VTable { 
    layout: Layout::new::<T>(),
    type_id: TypeId::of::<T>(),
  };
  let boxed = Box::new(Repr { vtable, data });
  // Safety: This cast is only valid when Repr's first field is a 
  // VTable and both are #[repr(C)]
  Box::into_raw(boxed).cast()
}

/// # Safety
///
/// This will leave `ptr` invalid afterwards.
unsafe fn downcast<T>(ptr: *mut VTable) -> T {
  assert_eq!(TypeId::of::<T>(), (*ptr).type_id, "Type mismatch");
  let boxed_repr: *mut Repr<T> = ptr.cast();
  let repr = Box::from_raw(boxed);
  repr.data
}
8

T itself is ?Sized which makes it complicated. I technicality don't use a dyn Trait. I just use a FnOnce (even though technically this is a Trait as well) as the last member of T which makes the entire struct ?Sized.

9

Well at some point you go from an unnameable Sized type, F to a dyn FnOnce(). Instead of relying on Rust's natural DST coercion, could you manually coerce to a thin trait object there?

10

There are a couple of crates for storing thin pointers to trait objects with the vtable pointer stored in the same allocation as the value. I haven't tried either of these, so I don't know how well they work in practice:

1 Like
11

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.