Why doesn't `Option` support `dyn Trait`?

That's not true; the act of calling mem::zeroed<&T>() (for example) is enough to invoke instant UB, even if the result is never used. The docs for ManuallyDrop even mention this:

That's precisely because MaybeUninit is designed to make it harder to invoke this particular kind of UB. Unfortunately it doesn't work in this case because MaybeUninit<T> doesn't have T: ?Sized. I'm not sure why not; maybe it's just an oversight.

No, there is a difference between uninitialized data and data that has been dropped. Even drop cannot zero the bits of a &T.

1 Like

Okay, so zeroing was a bad idea (according to all the theory, although a practical example of how this could go wrong in this case would make it clearer). But if I used ManuallyDrop::new(mem::uninitialized()), would that be okay?

UB is just shorthand for breaking the contract with the optimizer, so what could it optimize that would make this go wrong? I have to initialize (or not initialize) some memory that I will never access. No code touches that memory after it is written (or not written).

Edit: So, I've found the rule in the Rustonomicon that says I shouldn't produce any value which isn't valid. It all seems a bit woolly around ManuallyDrop, though, since it can be post-drop and an invalid value and still be moved around. So maybe it just needs someone to bless this particular use of ManuallyDrop. Otherwise really I need MaybeUninit to support ?Sized, but since it is a union, and unions don't support ?Sized (according to an error message) I'm stuck. So that's all pretty disappointing. I can't even use ptr::write, as I need something to reserve space in the struct, i.e. do size/alignment.

The answer is niche filling size optimizations.

It's easiest to demonstrate with the guaranteed non-null optimization of Option<&T>. This isn't represented as (bool, MaybeUninit<&T>), it's roughly represented as union { 0usize, &T }.

If someone does Option<jimuazu::Opt<&T>>, the compiler is perfectly allowed to say "hey, in the &T part of Opt<&T>, 0x00000000 is an invalid bit pattern that can never occur. Let's say that setting the &T to that is how we represent Option::None, so we don't need to make Option<jimuazu::Opt<&T>> any bigger than jimuazu::Opt<&T>."

This is incorrect. A dropped ManuallyDrop<T> is not an invalid value. It is a "logically moved from" value. The raw bits are still valid for type T. Drop::drop is still not allowed to put your value into an invalid state, just an unsafe one (i.e. one where safety invariants (of the type itself, not of its members!) are not met).


I'm pretty sure what you're trying to do isn't possible on the stack. But you're using an indirection, so you can theoretically do better by dropping down to manually allocating your own indirection. It would be annoyingly difficult and require re-implementing basically all of Rc, RefCell, and Option manually using just raw allocation, raw pointers, and UnsafeCell, but it would be possible. (And then you still wouldn't be able to coerce Actor<Struct> to Actor<dyn Trait> anyway because CoerceUnsized is a compiler-internal trait (and probably stuck that way).)

4 Likes

Thanks for explaining it. Okay, I see that part now.

I think it could be an invalid value, because the Rustonomicon says that a Box that is dangling is an invalid value. But I see what you are saying. It will be valid bit patterns for those types.

I'm a lot further along than you imagine. I have a type that's effectively a custom Rc of a base structure, a discriminant and a union (but not using union), with one branch of the union being unsized. I have solved the CoerceUnsized problem, because if you can get it in the form of a Box<something> you can do the coercion, and then turn it to a raw pointer to make the Rc. So I think I can do this, without UB (although I will double-check the Rustonomicon list). It's unfortunate about Opt, though, but I was only implementing that to test some of the ideas.

If I understand correctly, there are two different cases of "invalid values" (described here). In short, null pointer as &T is insta-UB, since references being non-null is a validity invariant; Box being dangling is UB only if it is accessed, since it pointing to valid memory is a safety invariant.

2 Likes

The maybe-uninit crate uses exactly the same approach that I did (an uninitialised ManuallyDrop instance), and so is also unsound behind a safe interface.

The maybe-uninit crate is a backport of std::mem::MaybeUninit for older Rust versions.

It is exactly the same implementation as std::mem::MaybeUninit.

It uses a union.

Edit: Nevermind, the linked scr in docs.rs confused me. I didn’t notice that clicking [src] on this page links to the standard library type but that one is only used with a certain compilation flag.

Edit2: But the README also says that the API is unsound before Rust 1.35 where that implementation is used.

1 Like

Well you can kind of see how it happened and understand the author's dilemma, but it is still a huge footgun if someone wraps an Option around it, and it is a dependency of crates such as crossbeam. The unsoundness will only appear when testing against earlier Rust versions.

The list is certainly quite short and as far as I can tell crossbeam doesn’t have the maybe-uninit-dependency anymore on master, so that’ll go down to maybe-uninit being almost entirely unused in the near future.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.