Type-erasing pointers to T: ?Sized

help
1

TL;DR is it possible to round-trip a T: ?Sized pointer *const T -> *const [()] -> *const T without using unstable features?

I'm trying to solve a situation where I'm trying to type-erase Arc. The issue I have is that for T: ?Sized, I can't store the pointer from Arc::into_raw in a type-erased form.

  1. Is it possible to cast a potentially-fat pointer into a type-erased fat pointer (like *const [()]) without hitting UB and without specialization or the ptr_metadata feature?
  2. If yes, what should that type-erased pointer type be? Is using something like *const [()] safe, if I transmute_copy it later to a pointer that's potentially not fat (but always the same type the pointer was created from)?
  3. If this can only be done with ptr_metadata, is this feature likely to get stabilized within the next 1, 3, 5 years?
2

What type are you specifically trying to convert to? Arc<dyn Trait> works just fine, and so does *const dyn Trait.

3

A MaybeUninit is guaranteed to retain the value of its bytes exactly, so it can be used to roundtrip anything. Using that strategy, you could do it like this: playground.

4 Likes
4

I'm writing stuff that's generic over T, so it's not guaranteed to be unsized.

5

I'm not sure I understand the problem, but what's wrong with *const T, where T: ?Sized. That way it would be generic.

1 Like
6

I'm trying to write a type-erased type where the user won't know what T is, so I can't store it as *const T.
The user won't interact with T directly either.

7

It's not unusual to type-erase *const T into *const () with T: Sized when passing a pointer via a route like generic code → non-generic code → generic code, where you know that both generic parts will be generic over the same type. It's especially common in C FFI, but it can also be useful in pure Rust code, e.g., I know that Tokio does this in some places internally.

I don't think its a stretch to want the same for unsized T.

8

But dyn Trait is always unsized. Isn't that what you mean by type erasure? At least that's what is primarily used for type erasure in Rust. The usual pattern is

fn erase<T: Trait>(value: T) -> Box<dyn Trait> {
    Box::new(value) as _
}

and you can replace the box with any other pointer-like type, as appropriate.

9

Yes, dyn Trait is always unsized.
Consider this pair of functions:

// There
fn erase<T: ?Sized>(ptr: *const T) -> ErasedPtr { ... }
// And back
unsafe fn restore<T: ?Sized>(ptr: ErasedPtr) -> *const T { ... }

We use erase to store ErasedPtr in a struct that doesn't have T as a generic parameter, but at some point it passes it to code that will know it's T and want to get the pointer back. T can be anything, including u8, dyn Trait, or [U].

The T here is not always unsized, and we need to be able to write the implementation so that it works for both sized and unsized pointers. @alice already gave an example using MaybeUninit that should work correcly in both cases as far as I know.

10

Okay, so you are not doing type erasure. The returned pointer doesn't, apparently, contain any knowledge about what type it points to (and so you can't dynamically dispatch on it), therefore you will use unsafe to supply the type upon downcasting (which is not guaranteed to be correct). Is that what you are trying to do?

11

That's how interpreted it.

I think type erasure is a quite reasonable name. It's just implemented manually rather than using the features provided by the language.

3 Likes
12

In short, you cannot go from *mut dyn Trait to *mut [()], because those have different "pointee kinds". There are a few solutions which can allow you to do type erasure of this kind:

  • Use ptr_metadata to restrict to a known set of pointee kinds.
  • Use a custom trait implemented for<T: Sized> and any unsized types which you want to support.
    • This is the approach that my erasable crate takes, though it only supports types which can erase to a thin pointer (*mut ()).
      • It's designed for use where erasing to a thin pointer is desired, e.g. when doing FFI with APIs carrying void*.
      • I could add support to erasing to a fat pointer, but haven't had a use case. Adapting to ptr_metadata will require a breaking update to the crate family anyway, so I'm not super motivated to extend the crates until then.
    • Would work well if you only need to support a single unsized kind (e.g. slice tails).
  • Transmute to an untyped storage blob (e.g. MaybeUninit<(*mut (), *mut ())>) to hold erased pointers.
    • Relies on formally undecided but reasonably likely memory model details.
    • More restrictive than other options; effectively an implicit union between any pointer type.
      • E.g. can't get the pointer address without knowing the proper pointer type to access as.
    • Would panic if there's ever a pointer which has size or alignment greater than 2×usize.
    • Using *mut [()] as your storage would be unsound; the provenance (roughly, pointer validity) of the vtable pointer for dyn Trait would be unsound.
    • Using *mut dyn Trait might be sound (unclear) but is at best a safety footgun (it's considered a safety requirement that such a pointer references a valid vtable).

If a motivated developer "adopts" the ptr_metadata API and pushes towards stabilization, I could see it happening on around a year timeline. I don't really see it stabilizing anytime soon without such "adoption," since it's relatively higher effort for lower impact than other development avenues.

13

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.