Where did Unique<T> go?

[jbe]

The Rustonomicon mentions a type (constructor) Unique<T> on the page about PhantomData. It's also often mentioned and/or referenced, e.g. here.

It looks like it was once part of std as an unstable feature, but it disappeared. My questions:

• Where did it go?
• Is it still idiomatic to use it, or should I manually use a combination of *const T and PhantomData<T> where needed?
• Any other important facts to know about it?

[Cyborus]

@jbe

Where did it go?

From that first link you gave:

.. standard library made a utility for itself called Unique<T> ..

It's logically internal to std cause, while technically pub, it's perma-unstable behind ptr_internals and is intended to be an internal utility.

@jbe

Is it still idiomatic to use it, or should I manually use a combination of *const T and PhantomData<T> where needed?

Yeah, either use *const T/NonNull<T> + PhantomData<T>, or define it yourself:

pub struct Unique<T> {
    ptr: NonNull<T>,
    __marker_owning: PhantomData<T>,
}

@jbe

Any other important facts to know about it?

Since it owns the data it points to (hence "unique"), it's sorta like an unsafe, non-dropping Box. (in fact, it's used to implement Box)

[quinedot]

@jbe

Any other important facts to know about it?

I suggest reading all the source code comments.

[quinedot]

If you want to read a few hundred comments of unstable history and changing feelings on what should be stabilized or not, there's

• Tracking issue 27730
• Stabilize NonNull
• Some convo about Unique vs NonNull as relates to Box
• Probably other issues linked off those

But only if you're bored (I don't think you'll pick up any extra important facts).

[steffahn]

@jbe

It looks like it was once part of std as an unstable feature, but it disappeared.

It still is part of std as an unstable feature, but its documentation is hidden.

#![feature(ptr_internals)]
use std::ptr::Unique;

(playground)

You can see it here

Unique in std::ptr - Rust

[quinedot]

This issue though, is very interesting!

Gankra summarizes:

[If you use *const T or *mut T you can silence "unused T" warnings, but don't get owning semantics.] In either case you get no warning to use PhantomData . So dropck is fundamentally very easy to get wrong while also being incredibly obscure .

However this unsafety has been temporarily resolved by the fact that the non-parametric dropck rfc moved to safe defaults, where the presence of a generic argument implies “owns T”. And there’s no way to sneak in interesting lifetimes without being generic over them!

[...]

Now “owns T” only matters if you use the unsafe eyepatch, which is a great place to teach the user “hey if you do this, you should add a bunch of “owns” annotations.

("Eyepatch" is #[may_dangle].)

Niko replies,

I find this logic persuasive. It'd be nice if we could have it documented somewhere very central.

And naturally... that never happened . The Nomicon here is technically incorrect given that their example implementation of Vec doesn't feature #[may_dangle]. (However, the actual Vec does.)

At least, if that issue is taken to be normative. (Or maybe it's implied by the RFC, I'd have to revisit it. The attribute is still unstable and the tracking issue still open.)

[steffahn]

@quinedot

This issue though, is very interesting!

Technically a pull request though, I suppose?

@quinedot

Now “owns T” only matters if you use the unsafe eyepatch, which is a great place to teach the user “hey if you do this, you should add a bunch of “owns” annotations.

Now that's interesting; if I understand that correctly, this might explain my personal experience that PhantomData doesn't matter at all in practice for drop check, even though some places in the documentation/reference/nomicon (I don't remember where exactly) claim it does. Naturally I haven't ever been using may_dangle when experimenting with drop check! (Since it's unstable.)

[quinedot]

Oops, yeah, s/issue/PR/g.

Also, a correction: it's the nonparametic RFC (1238) that made things safer (just like Gankra said). Additionally, #[may_dangle] wasn't intended to be stabilized, but is rather a placeholder for some further, more complete solution to the whole situation.

I guess I'd personally fall into wanting a more official statement before considering only-need-own-semantics-with-#[may_dangle] a guarantee. (Given the perma-unstable status of #[may_dangle], and "we'll figure the rest out later" nature of both RFCs, it all just seems too "implementation detail" for me to do otherwise.) Still very interesting though.

[jbe]

@Cyborus

[…] it's perma-unstable […]

Hmmmm… "perma-unstable" sounds like an exaggeration of "perpetual-beta"

But seriously, just found the PR #46952, which I just noticed @quinedot linked in a subsequent reply ("Stabilize NonNull"). It really says:

Mark Unique “permanently-unstable”

So basically that means Unique<T> is something for compiler hackers and I have to implement it myself when/if I need it. (Not sure when/if I have to.)

@Cyborus

Since it owns the data it points to (hence "unique"), it's sorta like an unsafe , non-dropping Box . (in fact, it's used to implement Box )

That sounds a bit like ManuallyDrop<Box<T>> would behave same (or similar) like Unique<T> again? Anyway, I guess there is some magic involved with Box, as Box::new is implemented as:

pub fn new(x: T) -> Self {
    box x
}

I've only ever seen a lowercase box in the source in the std documentation. But just found this Issue #49733, explaining a bit about it.

@quinedot

I suggest reading all the source code comments.

Thanks for the link, I'll look into that, just to get a better understanding of working with pointers in Rust. I'm not even sure if I will need Unique<T> soon. I think in many cases I can use *mut T (which should fall back to safer defaults like !Send, !Sync, and invariance in T, if I understand it right). But I generally feel like my knowledge regarding Rust and pointers is insufficient to write safe code using unsafe and/or pointers (which I need for FFI). Variance and drop checking seems to be a huge issue when working with pointers, so I would like to understand it from the bottom up.

@steffahn

It still is part of std as an unstable feature, but its documentation is hidden.

Yeah I see, but I guess I shouldn't use #![feature(ptr_internals) unless I'd work on the Rust compiler itself (which I likely won't do in the near future).

@quinedot

This issue though, is very interesting!

Gankra summarizes:

[…]
Now “owns T” only matters if you use the unsafe eyepatch, which is a great place to teach the user “hey if you do this, you should add a bunch of “owns” annotations.

("Eyepatch" is #[may_dangle] . )

I had read about may_dangle / "Eyepatch" in the Nomicon on Drop Check but thought I can safely ignore that subsection as if I'd not deal with it, I would be on the safe side due to safe defaults (hope that's true).

@steffahn

Now that's interesting; if I understand that correctly, this might explain my personal experience that PhantomData doesn't matter at all in practice for drop check, even though some places in the documentation/reference/nomicon (I don't remember where exactly) claim it does.

You might have read it in the Nomicon section on PhantomData:

[…] PhantomData consumes no space, but simulates a field of the given type for the purpose of static analysis. This was deemed to be less error-prone than explicitly telling the type-system the kind of variance that you want, while also providing other useful things such as the information needed by drop check.

[…]

In order to tell dropck that we do own values of type T, and therefore may drop some T's when we drop, we must add an extra PhantomData saying exactly that: […]

The drop checking is also listed in the table at the bottom of the section.

---

Phew, so PhantomData<T> and PhantomData<fn() -> T> behave the same for now, but that might change in future? Sounds… dangerous (if you rely on that?)

Or is it guaranteed that these will behave the same in future as long as #[may_dangle] isn't used? But #[may_dangle] isn't even stabilized yet?

Which information is authoritative? What can I rely on? I'm a bit overwhelmed by the information. Maybe I'll try to re-read everything tomorrow, but if there's an easy way to summarize it, I'd be happy about it, of course .

[Cyborus]

@jbe

That sounds a bit like ManuallyDrop<Box<T>> would behave same (or similar) like Unique<T> again?

Mostly, but ManuallyDrop<Box<T>> is on the heap, while Unique<T> isn't necessarily (but can be)

[quinedot]

@jbe

Phew, so PhantomData<T> and PhantomData<fn() -> T> behave the same for now, but that might change in future? Sounds… dangerous (if you rely on that?)

First, I don't think there's a way to rely on it and get burnt by some future compiler change breaking your existing code. RFC 1238 made less things compile, not more. As I understand it, the danger is when you write code as if you had owning semantics when you don't, and dropck lets you compile something unsound as a result. But since the RFC, you always do have owning semantics (on stable). If things change back, it will be a (revived) danger for newly written code, not existing code.

But I still think acting as if it may change back is the safer take, in case the danger gets revived and you're still maintaining your code base . And is there a good reason to not use PhantomData<T> in the meanwhile?

@jbe

Or is it guaranteed that these will behave the same in future as long as #[may_dangle] isn't used? But #[may_dangle] isn't even stabilized yet?

The uncertainty is why I lean towards not assuming a guarantee.

There will be something like #[may_dangle], too many things rely on it. There could still be a guarantee made that you always get owning semantics unless you (unsafely) opt out of them, even if there are no stable ways to opt out of them yet.

Basically, what Gankra said in that PR:

All that remained was to agree that we weren’t willing to take a 5th shot at trying to make a “smart and safe” dropck. Everyone in the meeting agreed it was time to give up on such an endeavour.

The RFCs didn't commit to not taking another shot. But that PR was merged. But-but there was no FCP or anything and it's arguably just an implementation detail.

@jbe

Which information is authoritative? What can I rely on?

I don't think this particular case is very concerning, just go with the safer option that you still have to worry about ownership semantics even when not using #[may_dangle]. I.e. the safest thing to rely on is nothing . (I also think the vast majority of Rust code need not care either way. But I don't know what you're writing...)

To actually address the questions, official documentation and RFCs, mostly. FCPs and UCG can give hints. Things that are widely relied upon are generally safe. Things that are very obscure and/or can be changed without breaking much are less safe.

Ultimately, there's still no spec and few absolutes . Sadly. To wit, everything I've written in this post assumes I'm up to date now, even though I only found that PR and grokked the connection RFC 1238 today!

Quote of the week from a little while ago:

Anyway: the standard library docs say "check the nomicon"

then the nomicon says "here is some advice and ultimately we don't know, maybe check UCG"

then UCG says "ultimately we don't know it's probably like this but there's no RFC yet"

then Ralf says "probably it should be allowed if the layout matches".

[Yandros]

To quote myself, from that post @jbe linked to:

@Yandros

Using PhantomData<T> for its dropck properties for some struct Foo<T> is only required when there is:

• unsafe impl<#[may_dangle] T> Drop for Foo<T> (with the contract that T should then not be accessed except for a "potential" recursive T::drop() )

• …

and elsewhere in this forum:

@Yandros

Note: if you want to have PhantomData alter the behavior of dropck checks, you need to have used an unsafe impl<#[may_dangle]…> Drop somewhere.

See also: this other post of mine over Zulip:

@https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/.E2.9C.94.20PhantomData.20and.20may_dangle/near/252283341

As the author of that post, I still stand by that statement: the confusion about PhantomData being needed stems from a very old article that still lingers in the nomicon, which was written in a time were Rust was more error prone / #[may_dangle] was implicitly universal, I suspect. This was deemed too dangerous / error-prone, and it was made so that whenever something had an impl Drop, then dropck would require that its generic parameters not dangle when dropped. This is still the case as of today, and is what makes an "owning PhantomData" superfluous, albeit a good conversation fuel

In order to keep the old behavior available, #[may_dangle] was thus created, but as an unstable attribute. By virtue of being unstable, unsafe, and subtle / error-prone, I'd say that #[may_dangle] is one of the rarest things in user libs out there. So at the end of the day, only the stdlib has to deal with it, and when it does, then indeed it does need that "owning PhantomData" back.

In practice #[may_dangle] by being not only unsafe, but unstable, is used in almost no libraries whatsoever (maybe some of the smallvec ones can let you opt into using them), and nobody is none the wiser: missing #[may_dangle] only hurts ergonomics in a quite negligible manner!

So, in practice, one could imagine that #[may_dangle] does not exist, and that this whole PhantomData-to-express-ownership-to-dropck is rather an interesting artifact from the past.

This is about the PtrThatOwns<T> aspect of Unique<T>, which is, thus, no longer relevant.

---

There is another way more interesting aspect about Unique<T>, which is that of being, as its name conveys, an unaliased pointer! (I don't know if Stacked Borrows uses this information, but it technically could). So the question now is: can we write our own unaliased (and thus dereferenceable) "raw" pointer abstraction? In practice, that alone isn't really possible, because of extra alignment requirements, which constrain the "higher-level" pointers able to convey lack of aliasing semantics (if we were able to express [MaybeUninit<u8>; size_of::<T>()] then we'd be able to lift the alignment requirements).

Expressing an unaliased well-aligned dereferenceable "raw" pointer

AFAIK, there is no clean way to achieve this, only "dirty hacks":

First attempt

If T : 'static, this seems to be easy:

#[repr(transparent)]
pub
struct UnaliasedDereferenceableWellAlignedPtr<T> (
    &'static mut MaybeUninit<T>,
);

It does come with an extra caveat, though: the obtained pointer ends up with shrunk provenance over that T and that T only, so that doing the following is Undefined Behavior:

let ptr: UnaliasedDereferenceableWellAlignedPtr<T> = …; // provenance over that `T`
let erased = ptr.cast::<()>; // provenance shrunk down to `()` = 0 bytes!
let ptr = erased.cast::<T>(); // UB: cannot be `dereferenceable` over a non-null span of bytes anymore!

That doesn't seem good for a "raw" pointer…

Second attempt

This is the only way I've managed to come up with, but at the cost of requiring alloc ():

#[repr(transparent)]
pub
struct UnaliasedWellAlignedPtr<T> (
    ManuallyDrop<Box<T>>,
);

we do lose the dereferenceable-ity, which is a pity, since if we are looking for unaliased optimizations, then we are likely to also want dereferenceable optimizations as well.

This is the approach taken by ::stackbox, for instance:

danielhenrymantilla/stackbox.rs/blob/v0.1.2/src/ptr.rs#L1-L35

[quinedot]

@Yandros

that this whole PhantomData -to-express-ownership-to-dropck is rather an interesting artifact from the past.

Do you happen to be aware of anything more official than that PR guaranteeing it won't come back? The RFCs left it open. No one since seems to be able to pull the trigger on documentation even.

• Nomicon
• More nomicon
• Reference
• UCG

[Yandros]

I think that a very simple practical argument to the current situation not going to change is that it would be a soundness-breaking change: APIs out there that are only sound / robust to misuses thanks to the current dropck semantics would suddently be "misusable" / unsound. Granted, one could rather say that such APIs are already unsound, precisely because the situation could change, but given the comments in the issues you linked to I do think that we can currently rely on this property, and therefor, we will be able to rely on this property in the future as well.

That being said, I agree with you and @steffahn that the nomicon / or other docs ought to be updated to reflect these "new" semantics, not only for the sake of documentation, but also to make the current stance more official.

[jbe]

@Yandros

That being said, I agree with you and @steffahn that the nomicon / or other docs ought to be updated to reflect these "new" semantics, not only for the sake of documentation, but also to make the current stance more official.

Well, only if the change to fall back to a safer behavior is indeed stable.

Not sure if I get the meaning of the wording right, but the Nomicon currently says (in bold):

For a generic type to soundly implement drop, its generics arguments must strictly outlive it.

It says "arguments" (not "owned values" or something like that). Thus I would say that when you have Tc<T> which implements Drop, then the compiler will force you that T strictly outlives Tc<T>, regardless of whether a value of type T is owned or not by Tc<T>, right?

(Edit: Assuming you don't use anything like #[may_dangle], of course.)

So in a way, the Nomicon already reflects the new behavior, doesn't it? Just the section on PhantomData seems outdated.

P.S.: If it's (safely) possible to update the documentation, I would appreciate an update, as it would make things much easier for me and give me less headache to write (hopefully) sound code.

[quinedot]

@Yandros

I think that a very simple practical argument to the current situation not going to change is that it would be a soundness-breaking change: APIs out there that are only sound / robust to misuses thanks to the current dropck semantics would suddently be "misusable" / unsound. Granted, one could rather say that such APIs are already unsound, precisely because the situation could change, but given the comments in the issues you linked to I do think that we can currently rely on this property, and therefor, we will be able to rely on this property in the future as well.

I guess it comes down to one's confidence in a soundness change around this area being a strong enough demotivator, and in the PR comments being a stronger indicator that the RFC stance. There are things you can rely upon to not miscompile today (if you do the research to see how rustc is implemented), but are still UB and you can't rely on it tomorrow, so I don't think that's enough on its own. (This includes things that the compiler internals take advantage of due to their "inside knowledge" it's not exploited.)

I suspect one reason the documentation issues are stalled is that no one wants to imply an official stance without some sort of governance process.

@Yandros

That being said, I agree with you and @steffahn that the nomicon / or other docs ought to be updated to reflect these "new" semantics, not only for the sake of documentation, but also to make the current stance more official.

It'd be great if it became official; if it can't become official, the docs should at least be updated to reflect the current reality with a typical "but that's just how it works today, go read up on these issues if you want to cover all bases" disclaimer.