# PhantomData and dropck confusion **URL:** \ **Category:** help\ **Created:** [March 24, 2019, 10:32pm UTC](https://users.rust-lang.org/t/phantomdata-and-dropck-confusion/26624 "2019-03-24T22:32:22Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![ehsanmok](https://sea1.discourse-cdn.com/flex019/user_avatar/users.rust-lang.org/ehsanmok/32/21142_2.png) [@ehsanmok](https://users.rust-lang.org/u/ehsanmok)\ **Post date:** [March 24, 2019, 10:32pm UTC](https://users.rust-lang.org/t/phantomdata-and-dropck-confusion/26624/1 "2019-03-24T22:32:22Z")
I'm trying to understand the relation between `PhantomData` and dropck in _greater depth_ esp. for the case like `Vec` or `LinkedList` that `PhantomData` makes generic drop sound. I've already read nomicon as well as [RFC-0769-sound-generic-drop](https://github.com/rust-lang/rfcs/blob/master/text/0769-sound-generic-drop.md) and [RFC-1238-nonparametric-dropck](https://github.com/rust-lang/rfcs/blob/master/text/1238-nonparametric-dropck.md). Also found [this explanations by @pnkfelix and @shepmaster](https://stackoverflow.com/questions/42708462/why-is-it-useful-to-use-phantomdata-to-inform-the-compiler-that-a-struct-owns-a/42721125#42721125) or [the same example](https://users.rust-lang.org/t/is-phantomdata-useless-to-dropck-now/9495/12). However, due to many updates to rustc, none of the examples that suppose to **not** compile, compile now and I'm in a _confusion state_! [Here's the updated example given by Felix](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=596dd54e5ed494cfc122c8427d1db38d) (though this is less about `PhantomData` and raw ptr I guess)! [Here's the example given by Jake](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=e37dc67e4a8658505cda1d0e3dbb35bc). Can anyone provide a clear example capturing unsoundness? Mainly, for example, how can I clearly show that (in code) ```rust struct Vec { ptr: NonNull, cap: usize, len: usize, } ``` is unsound whereas ```rust struct Vec { ptr: NonNull, cap: usize, len: usize, _marker: PhantomData, } ``` is sound in a normal code _without_ using any uninitialized let binding `let (a, b);`? ---
**Author:** ![Yandros](https://sea1.discourse-cdn.com/flex019/user_avatar/users.rust-lang.org/yandros/32/8136_2.png) [@Yandros](https://users.rust-lang.org/u/Yandros)\ **Post date:** [March 25, 2019, 1:12pm UTC](https://users.rust-lang.org/t/phantomdata-and-dropck-confusion/26624/2 "2019-03-25T13:12:33Z")
Imagine the following "lib" (I am creating a custom `Box` rather than a `Vec` for the sake of simplicity): ### A custom `Box` type ```rust use ::std::{*, ptr::NonNull, }; /// Simplified raw allocation interface pub trait IsAllocator { /// Allocates memory for a T fn alloc () -> NonNull; /// Frees the pointed memory, but does **not** drop the pointee unsafe fn free (_: NonNull); } pub enum Allocator {} impl IsAllocator for Allocator { /* ... */ } /// Box is simpler than Vec (always len == capacity == 1) pub struct Box { ptr: NonNull, // Since we are the unique owner of T, covariance is fine } impl Box { pub fn new (value: T) -> Self { let non_null = Allocator::alloc(); unsafe { ptr::write(non_null.as_ptr(), value); } Self { ptr: non_null } } pub fn get (&'_ self) -> &'_ T { unsafe { self.ptr.as_ref() } } pub fn get_mut (&'_ mut self) -> &'_ mut T { unsafe { self.ptr.as_mut() } } } impl Drop for Box { fn drop (&mut self) { unsafe { // before freeing the allocated memory, we may need to recursively // drop the pointee, in case it owns data with drop glue // (else we would leak memory, with, for instance, a Box>) eprintln!("Dropping the Box at {:p}", self); if mem::needs_drop::() { eprintln!( "[Recursively dropping the pointee at {:p}]", self.ptr, ); ptr::drop_in_place::(self.as_mut()); } else { eprintln!("[No need to drop the pointee at {:p}]", self.ptr); } Allocator::free(self.ptr); } }} ``` ### Basic usage Now, imagine it being tested with the following program: ```rust fn main () { let s = String::from("Hello, World!"); let mut p = Box::new(None); *p.get_mut() = Some(&s); dbg!(p.get()); } // p is dropped; then s => OK ``` This [compiles and run fine](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=8dd685648d78f055b8afdbea693200d5). ### More convoluted example Let's invert the order of the `p` and `s` declarations: ```rust fn main () { let mut p = Box::new(None); let s = String::from("Hello, World!"); *p.get_mut() = Some(&s); dbg!(p.get()); } // s is dropped before p, but &String has no drop glue => OK ``` This [no longer compiles](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=24f6eb92dc4e4068f235dab60f5735a6): ```rust error[E0597]: `s` does not live long enough --> src/main.rs:92:25 | 92 | *p.get_mut() = Some(&s); | ^^ borrowed value does not live long enough 93 | dbg!(p.get()); 94 | } // s is dropped before p, but &String has no drop glue => OK | - | | | `s` dropped here while still borrowed | borrow might be used here, when `p` is dropped and runs the `Drop` code for type `Box` | = note: values in a scope are dropped in the opposite order they are defined ``` emphasis on: > [@dropck error message](#): > > borrow **might be used** [...] when `p` is dropped and runs the `Drop` code for type `Box` But in our case **the only way we might _use_ (deref) the pointee** is when dropping it if it has drop glue, which `&String` does not have. So, **when there is no drop glue** , the pointer **may dangle**. Luckily, this can be expressed with the `dropck_eyepatch` unstable feature: ```rust unsafe impl<#[may_dangle] T> Drop for Box { fn drop (&mut self) { /* [the Drop logic has not changed] */ }} ``` This [compiles and runs fine](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=1264b8ac88b84bafc17206f7f7f29345) ### "This compiles and runs fine" ?? Road to UB-land Now that we have the (`unstable` and) `unsafe` `impl Drop<#[may_dangle] T>`, with nothing else changed, we will find ourselves with the problem that without realizing it, we have told `dropck` to trust us too much: **when `T` has drop glue, we do use the pointer!** ; and there is actually no place where we have told `dropck` that (remember that as far as `dropck` is concerned, all the implementations of all functions are `loop {}`, _i.e._, it can obviously not look at the code being actually run; it just interprets function signatures). Let's abuse it in action, with the following program: ```rust #[repr(transparent)] #[derive(Debug)] struct PrintOnDrop (T); impl Drop for PrintOnDrop { fn drop(&mut self) { eprintln!("Dropping {0:?} at {0:p}", self); }} fn main () { let mut p = Box::new(None); let s = String::from("Hello, World!"); *p.get_mut() = Some(PrintOnDrop(&s)); dbg!(p.get()); } // s is dropped before p, and PrintOnDrop<&String> has drop glue => UAF ``` This [prints the following](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=a3a9951dbb1802930d4844b4416f9c75): ```rust [src/main.rs:105] p.get() = Some( PrintOnDrop( "Hello, World!" ) ) [Dropping the Box at 0x7ffedcb20b20] [Recursively dropping the pointee at 0x556876d62a40] Dropping PrintOnDrop("\u{0}\u{0}\u{0}\u{0}\u{0}\u{0}\u{0}\u{0}orld!") at 0x556876d62a40 ``` And there it is, a magnificent UAF, because `dropck` allows us to have `Foo` exit a scope with a dangling `T` (such as `T = PrintOnDrop<&'dangling String>`) when: 1. `unsafe impl<#[may_dangle] T> Drop for Foo` is used, 2. struct-wise, **`Foo` does not _own_ `T`** That second point is then paramount, and in our `Box` case, all we had was a `NonNull` pointer which on its own _does not own `T`_ (it's just a newtype wrapper around `*const T` (with a non-zero lang attached to it)). What does own `T`? Well, for instance, `PhantomData` does. ## FINAL FIXED VERSION Hence, we must fix the library code with the following one: ```rust /// Box is simpler than Vec (always len == capacity == 1) #[allow(non_snake_case)] pub struct Box { ptr: NonNull, _owns_T: marker::PhantomData, } impl Box { pub fn new (value: T) -> Self { let non_null = Allocator::alloc(); unsafe { ptr::write(non_null.as_ptr(), value); } Self { ptr: non_null, _owns_T: Default::default() } } pub fn get (&'_ self) -> &'_ T { unsafe { self.ptr.as_ref() } } pub fn get_mut (&'_ mut self) -> &'_ mut T { unsafe { self.ptr.as_mut() } } } unsafe impl<#[may_dangle] T> Drop for Box { fn drop (&mut self) { unsafe { // before freeing the allocated memory, we may need to recursively // drop the pointee, in case it owns data with drop glue // (else we would leak memory, with, for instance, a Box>) eprintln!("Dropping the Box at {:p}", self); if mem::needs_drop::() { eprintln!( "[Recursively dropping the pointee at {:p}]", self.ptr, ); ptr::drop_in_place::(self.get_mut()); } else { eprintln!("[No need to drop the pointee at {:p}]", self.ptr); } Allocator::free(self.ptr); } }} ``` This [no longer compiles](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=47827e1d780fe0e1735149114d266770), and [if we replace `PrintOnDrop(&s)` with `&s` it does still compile](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=705efed82e6bef5cb23f89f66f9f7ee8): yay! ```rust error[E0597]: `s` does not live long enough --> src/main.rs:105:37 | 105 | *p.get_mut() = Some(PrintOnDrop(&s)); | ^^ borrowed value does not live long enough 106 | dbg!(p.get()); 107 | } // s is dropped before p, and PrintOnDrop<&String> has drop glue => UAF | - | | | `s` dropped here while still borrowed | borrow might be used here, when `p` is dropped and runs the `Drop` code for type `Box` | = note: values in a scope are dropped in the opposite order they are defined ``` ### Conclusion Using `PhantomData` for its `dropck` properties for some struct `Foo` is only required when there is: - `unsafe impl<#[may_dangle] T> Drop for Foo` (with the contract that T should then not be accessed except for a "potential" recursive `T::drop()`) - `Foo` does not _own_ `T` struct-wise (for instance, when it only has a non-[`Unique`](https://github.com/rust-lang/rust/blob/d91b32b4dc9db066ab7deedbc51dc62ba597524b/src/libcore/ptr.rs#L2696) pointer to `T` like `*const T`) and yet it drops `T` ---
**Author:** ![ehsanmok](https://sea1.discourse-cdn.com/flex019/user_avatar/users.rust-lang.org/ehsanmok/32/21142_2.png) [@ehsanmok](https://users.rust-lang.org/u/ehsanmok)\ **Post date:** [March 25, 2019, 6:56pm UTC](https://users.rust-lang.org/t/phantomdata-and-dropck-confusion/26624/3 "2019-03-25T18:56:59Z")
Wow ... I could not hope to get a better explanation that this! thank you so much 😀 The conclusion makes perfect sense and I can see how `PhantomData` and `dropck_eyepatch` work in tandem elegantly in std, making dropck less conservative while providing no-UAF guarantees in these very important cases. cc. @Gankra, I hope this can be added to the nomicon in _Ownership capture_ right after `PhantomData` Section. ---
**Author:** ![system](https://sea1.discourse-cdn.com/flex019/user_avatar/users.rust-lang.org/system/32/51372_2.png) [@system](https://users.rust-lang.org/u/system)\ **Post date:** [June 23, 2019, 6:57pm UTC](https://users.rust-lang.org/t/phantomdata-and-dropck-confusion/26624/4 "2019-06-23T18:57:00Z")
Closed on June 23, 2019, 6:57pm UTC This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.