What a Phantom do

hello,

I can't understand what the commence "understand that we logically own a T" means.
Does it owns a T meaning it is the Unique's responsibility to call CustomSmartPointer.drop when It life ends?

However, the result seems CustomSmartPointer.drop() never get called, even I call drop(Unique._marker) manually.

And what's more, if I call

let ptr = &CustomSmartPointer {
    data: String::from("123"),
}

instead of

let ptr = alloc(Layout::new::<CustomSmartPointer>()) as *mut CustomSmartPointer;

the drop get called when ptr life get end. e.g., it is not moved to _marker at all.
So, I totoally get confused with Phantom,
How can I tell compiler to call drop when Phantom ends?

I can make it get called via ptr::drop_in_place;
But if it is necessary, why Phantom is here as a necessary field? I get the code from standard lib

        unsafe {
            println!("------------------{:?}",  *x.pointer);
            ptr::drop_in_place(x.pointer as *mut CustomSmartPointer);
        }

The code is as below:

use std::mem;
use std::alloc::{alloc, Layout};

#[derive(Debug)]
pub struct Unique<T: ?Sized> {
    pointer: *const T,
    // NOTE: this marker has no consequences for variance, but is necessary
    // for dropck to understand that we logically own a `T`.
    //
    // For details, see:
    // https://github.com/rust-lang/rfcs/blob/master/text/0769-sound-generic-drop.md#phantom-data
    _marker: std::marker::Phantom<T>,
}

fn testLife() {
    println!("aaa");
    {
        let x = d();
        unsafe {
            println!("------------------{:?}",  *x.pointer);
        }
    }
    println!("====================");
}

#[derive(Debug)]
struct CustomSmartPointer {
    data: String,
}

impl Drop for CustomSmartPointer {
    fn drop(&mut self) {
        println!("Dropping CustomSmartPointer with data `{}`!", self.data);
    }
}

fn d() -> Unique<CustomSmartPointer> {
    unsafe {
        let ptr = alloc(Layout::new::<CustomSmartPointer>()) as *mut CustomSmartPointer;
        (*ptr).data = String::from("ooo");

        let x = Unique::<CustomSmartPointer> {
            pointer: ptr,
            _marker: std::marker::PhantomData
        };
        x
    }
}
1 Like

Have you read https://doc.rust-lang.org/std/marker/struct.PhantomData.html and/or https://doc.rust-lang.org/nomicon/phantom-data.html ? Any correct answer to these questions would essentially be copy-pasting from introductions like those.

1 Like

That's way deep in internals, can't offer much more than rfc.
Can tell/show you that this does not compile.

It looks like you copied some code from somewhere but didn't get the whole thing, which is why it doesn't make much sense.

_marker tells dropck that Unique<T> may drop a T. This does not actually drop a T -- you still must implement Drop if you want Unique<T> to call drop_in_place on pointer. The reason for PhantomData has to do with what happens if T is a type with an embedded lifetime; for the full details you'll have to read the RFC.

Now, apart from that, you have a major problem here:

When you assign to (*ptr).data, Rust will try to drop the String already there, which because ptr points to uninitialized memory is undefined behavior. You could force the issue with ptr::write but there really is no justification for that, since d can be written in only safe code.

fn d() -> Unique<CustomSmartPointer> {
    let ptr = Box::new(CustomSmartPointer {
        data: String::from("ooo"),
    });
    Unique {
        pointer: Box::into_raw(ptr),
        _marker: std::marker::PhantomData,
    }
}

the code is for trying to understand the internal of destruction. So I didn't call drop_in_place in drop, nor I call deallocate to free memory. I just print a message when drop has been called. Unfortunately , It does not print out, meaning the phantom data holding the memory logically does not take effect.

but I don't know why and how to fix this. I do implement drop on Unique, what extra work should I do to make the drop get called?

Oh, I figured out, the problem should be I didn't implement drop on unique. But why it needn't implement drop when a

struct s<T> {
   f: T
}

life time expires to drop T?
Is it a rule that if T is phantom data , drop must implemented to drop a owned field?

1 Like

PhantomData<T> is meant to be used where a type logically owns T, but Rust can't see that for whatever reason.

For example, in Unique

struct Unique<T> {
    ptr: NonNull<T>,
    _ph: PhantomData<T>
}

The Unique logically owns a T, but Rust has no way of knowing that if we just put this

struct Unique<T> {
    ptr: NonNull<T>,
}

Rust doesn't see any T, just the NonNull<T> which it will assume means non-owning. So we must tell Rust that we intend to own a T by using PhantomData<T>. On it's own PhantomData does nothing but force Rust to check some edge cases. But these edge cases matter when you implement Drop or need to take variance into account.

Hi, your post is great, but I am confused by the following:

So our OutRef<'_, T> was covariant in T , which means that OutRef<'_, &'static str> was a subtype of OutRef<'_, &'short_lived str> since 'static : 'short_lived ,
so the call

at_s.write(&s);
// i.e.
<OutRef<T?>>::write(
    // self: OutRef<T?>
    at_s, // : OutRef<&'static str> : OutRef<&'short_lived str>
    // value: T?
    &s, // : &'short_lived str
); // Thus T? = &'short_lived str

was not prevented by Rust, thus allowing to write a pointer to a short lived local where a pointer to a longer lived local was expected; which is unsound, and is the reason why non-owning mutable "references" (of any sort) need to be invariant.

As, we know, every function(write) has a signature, meaning T is determined at compile time, in this example

let at_s = OutRef::<&'static str>::from(&mut s);

It is OutRef::<&'static str> make the T to be &'static str,
e.g., method from of OutRef::<&'static str> should been called, replacing the T with &'static str, we get the specific from method:

    impl<'a, &'static str : 'a> From<&'a mut T> for OutRef<'a, &'static str> {
        #[inline]
        fn from (p: &'a mut &'static str)
          -> OutRef<'a, &'static str>
        {
            #[allow(unused_unsafe)]
            unsafe {
                // # Safety
                //
                //   - Respects the invariant, since something
                //     init can be seen as maybeuninit
                OutRef {
                    ptr: p,
                    _lifetime: PhantomData,
                }
            }
        }
    }

can see the returned value, at_s, has type OutRef<'a, &'static str>
When calling

    {
        let s = String::from("Short-lived str");
        at_s.write(&s); // override s with a pointer to the short-lived str
    }

it is OutRef<'a, &'static str>::write being called, makes value expected a type of &'static str
but &s has a type of &'a str which 'static : 'a

That means, 'static is a subtype of 'a, at_s.write(&s); should not passe compiling.
I don't know why the code passes compiling.
What have I misunderstood?

1 Like

forget my previous post, it turns out

pub
        fn write (self: OutRef<'a, T>, value: T)
          -> &'a mut T
        {
            let ptr = self.ptr.as_ptr();
            unsafe {
                // # Safety
                //
                //   - this is `MaybeUninit<T>::write`
                ptr.write(value);
                &mut *ptr
            }
        }

is really

pub
        fn write (self: OutRef<'static, &str>, value: &str)
          -> &'a mut &str
        {
            let ptr = self.ptr.as_ptr();
            unsafe {
                // # Safety
                //
                //   - this is `MaybeUninit<T>::write`
                ptr.write(value);
                &mut *ptr
            }
        }

so at_s.write(&s) get called,

fn main ()
{
    let mut s: &'static str = "Static str"; // s pointer to static str
    let at_s = OutRef::<&'static str>::from1(&mut s); // pointer to s
    {

        let s = String::from("Short-lived str");
        at_s.write(&s); // override s with a pointer to the short-lived str
        dbg!(s); 
    }
    dbg!(s); // Use after free!
}

value: &str accept any &str.
But what confuses me is

struct OutRef<'a, T : 'a>

why T: a is not appied to &str, where 'a is 'static, which should obviously forbidden a 'short_live_lofetime

This is where you are not totally right.

To see this, let's take an example where covariance is actually useful:

#[derive(Clone, Copy)]
struct Str<'lifetime> /* = */ (
    &'lifetime str,
);

impl<'lifetime> Str<'lifetime> {
    fn my_eq (self: Str<'lifetime>, other: Str<'lifetime>)
      -> bool
    {
        self.0 == other.0
    }
}

fn main ()
{
    let s1: Str<'static> = Str("Static");
    let local = String::new("Short lived");
    let s2: Str<'_> = Str(&*local);
    if s1.my_eq(s2) { // <-- HERE
        unreachable!()
    }
}

If what you said was true, then the above code would not compile and fail at the s1.my_eq(s2) line.
Indeed, that would become:

Str::<'static>::my_eq(
    s1, // : Str<'static> => fine
    s2, // : Str<'local> !: Str<'static> => error
)

So, as you can see with this example, Rust does need to have a mechanism to be more permissive when it is sound to be so. That's what variance is for (I should have started my post about variance with this example, so thanks for having pointed this out).

So the Rust solution here is to instead allow to make s1 "coerce" into some super type, with the hope to find some "common denominator ground" where s1 and s2 can "play together".

For this, it will imagine that the lifetime (used in the function call) is a "free parameter", and will try to see if type inference yields a solution to the constraint that both the s1 and s2 args need to be "compatible" with the result of this type inference (I'll use 'x to denote the lifetime inference variable / parameter):

Str::<'x>::my_eq(
    s1, // `Str<'static> : Str<'x>`? Yes, because `'static : 'x` and `Str` is covariant
    s2, // `Str<'local> : Str<'x>`? Only if `'local : 'x`
)

So the question now is: can Rust find a lifetime 'x which is not greater than 'local ? Well yes of course, it can use 'x = 'local !

So this can become:

Str::<'local>::my_eq(s1, s2);

and s1 in that expression happens to be seen as a Str<'local> rather than a Str<'static>.


So, back to:

at_s.write(&s);
// i.e.
<OutRef<T?>>::write(
    // self: OutRef<T?>
    at_s, // : OutRef<&'static str> : OutRef<&'short_lived str>
    // value: T?
    &s, // : &'short_lived str
); // Thus T? = &'short_lived str

simplifying the type inference to T? = &'x str, where 'x is the inference parameter, Rust has to try and see if there exists a solution to:

at_s.write(&s);
// i.e.
<OutRef<&'x str>>::write(
    // 1st param must be of type `OutRef<&'x str>`
    at_s, // `OutRef<&'static str> : OutRef<&'x str>`? Yes, because `'static : 'x` and **`OutRef` is covariant**
    // and 2nd param must be of type `&'x str`
    &s, // : `&'short_lived str : &'x str`? Only if `'short_lived : 'x`
);

which can be solved with, for instance, 'x = 'short_lived.

it seems your code indeed

#[derive(Clone, Copy)]
struct Str<'a> /* = */ (
    &'a str,
);

impl<'a> Str<'a> {
    fn my_eq (self: Str<'a>, other: Str<'_>)
      -> bool
    {
        self.0 == other.0
    }
}

fn main ()
{
    let s1: Str<'static> = Str("Static");
    let local = String::from("Short lived");
    let s2: Str<'_> = Str(&*local);
    if s1.my_eq(s2) { // <-- HERE
        unreachable!()
    }
}

Hi, I totally confused by the rule of lifetime
I runs the code is taken from yours, with some modification, and shows that currently to me,
the life time and variance rules seems unpredictable, will you kindly help me to achieve a deeper understanding

I post it here: