Totally confused by the rule of lifetime and variance

Hi, I totally confused by the rule of lifetime, can anybody can make an explanation for the following code?
The code is taken from PhantomData and dropck confusion, with some modification.
What confuses me most is:

seems

 let addr: *mut Option<&'static String> = p.ptr as *mut Option<&'static String>;

will bound life time 'static to addr, or to T in Option,. the raw pointer which should be nothing with lifetime;

because

 let addr: *mut Option<&'static String> = p.ptr as *mut Option<&'static String>;
 let addr = addr as *mut Option<&String>;
 ptr::write(addr, Some(&s));

will fails to compile with error: requires s0 to be borrowed from 'static, event this line

let addr = addr as *mut Option<&String>;

executes before ptr::write

but this code will pass compiling

 let addr: *const Option<&'static String> = p.ptr as *const Option<&'static String>;  
 let addr = addr as *mut Option<&String>;
 ptr::write(addr, Some(&s));

e.g. this time, 'static does not bind to addr, or Option

And what is more:

 let s0 = String::from("Hello, World!");
 let mut p: Box<Option<&String>> = Box::new(Some(&s0));
 // this line will pass compiling:
 let addr: *mut Option<&'static String> = p.ptr as *mut Option<&'static String>; 
 // and this does not, requires `s0` is borrowed for `'static` in [Box::new(None)]
 let addr: *const Option<&'static String> = p.ptr as *const Option<&'static String>; 

full code is here:

fn main() {
    test::test();
}

mod test {
    #![allow(unused_mut)]
    #![allow(unused_imports)]
    #![allow(unused_variables)]
    use std::mem;
    use std::ptr;
    use std::ptr::NonNull;
    use std::alloc::alloc;
    use std::alloc::Layout;
    use std::alloc::dealloc;

    pub trait IsAllocator2 {
        fn alloc<T> () -> *mut T;
        unsafe fn free<T> (_: *mut T);
    }

    pub enum Allocator2 {
    }

    impl IsAllocator2 for Allocator2 {
        fn alloc<T> () -> *mut T {
            let ptr = unsafe {
                alloc(Layout::new::<T>()) as *mut T
            };
            ptr
        }

        unsafe fn free<T> (ptr:*mut T) {
            dealloc(ptr as *mut u8, Layout::new::<T>());
        }
    }

    impl<T> Box<T> {
        pub fn get (&'_ self) -> &'_ T {
            unsafe {
                & *self.ptr
            }
        }

        pub fn new (value: T) -> Self {
            let non_null = Allocator2::alloc();
            unsafe { ptr::write(non_null, value); }
            Self { 
            ptr: non_null,
            }
        }
    }

    pub struct Box<T> {
        ptr: *const T,
    }

    pub fn test() {
        let s0 = String::from("Hello, World!");
        let mut p: Box<Option<&String>> = Box::new(Some(&s0)); // p.ptr ==> *const Option<&'s0 String>
        // let mut p: Box<Option<&String>> = Box::new(None);   // p.ptr ==> *const Option<&'static String>
        
        unsafe { 
            let s = String::from("Hello, World!");

            // not pass
            let addr: *mut Option<&'static String> = p.ptr as *mut Option<&'static String>; // requires that `s` is borrowed for `'static` in [Box::new(Some(&s0)), Box::new(None)]
            // pass
            let addr: *mut Option<&String> = p.ptr as *mut Option<&String>;
            // pass
            let addr: *const Option<&'static String> = p.ptr as *const Option<&'static String>; // requires `s0` is borrowed for `'static` in [Box::new(None)]
            // pass
            let addr = p.ptr; 

            let addr = addr as *mut Option<&String>;
            ptr::write(addr, Some(&s));
            println!("{:?}", p.ptr);
        }
        dbg!(p.get());
    }
}

Playground

You have to erase the type as *mut () (i.e. not original) before you can cast to a variant.

Don't know your post is about which one of my questions

Basically the main thing to realize here is that when casting pointers, it will not cast the lifetime if the type is otherwise the same type. I'm not quite sure why pointer casts behave this way, and I was surprised too, but this is how it seems to work.

The rest is explained by *const being covariant and *mut being invariant. Let's go through each case.

let mut p: Box<Option<&String>> = Box::new(Some(&s0));
let addr: *mut Option<&'static String> =
        p.ptr as *mut Option<&'static String>;

let addr = addr as *mut Option<&String>;
ptr::write(addr, Some(&s)); // error

This case fails because the Some(&s) passed to write does not live long enough. This is because the first addr is obviously a mutable pointer with a 'static lifetime. Since the second addr is a cast from the first with the same type behind the pointer besides lifetimes, it will not change the lifetime in the cast, thus the second addr also has 'static in the reference, leading to the error when assigning a non-static reference.

let mut p: Box<Option<&String>> = Box::new(Some(&s0));
let addr: *mut Option<&String> = p.ptr as *mut Option<&String>;
//                               ^^^^^ lifetime shortening happens here

let addr = addr as *mut Option<&String>;
ptr::write(addr, Some(&s)); // ok!

In this case we start out by casting p.ptr (which is a *const pointer) to a mutable pointer with no lifetime specified on the inner reference. Since the use of addr in the write requires the inner reference to have a short lifetime to be valid, it deduces that the inner lifetime behind the const pointer p.ptr must be that short. Since p.ptr is const, it is covariant, so shortening the lifetime of the const pointer before the cast to mutable pointer is not an issue. Thus the lifetime behind the mutable pointers was always the shorter lifetime needed to assign Some(&s).

let s0 = String::from("Hello, World!");
let mut p: Box<Option<&String>> = Box::new(Some(&s0)); // error!
let addr: *const Option<&'static String> =
        p.ptr as *const Option<&'static String>;

let addr = addr as *mut Option<&String>;
ptr::write(addr, Some(&s));

In this case the error happens somewhere else. In this case p.ptr is being casted to a pointer with a static inner reference. Since casts refuse to change the lifetime, this is actually a claim that p.ptr itself is a 'static lifetime, thus the error. Note that variance has no influence here because fixing it would require a wider lifetime, not a shorter one.

let mut p: Box<Option<&String>> = Box::new(None);
let addr1: *const Option<&'static String> =
        p.ptr as *const Option<&'static String>;

let addr2 = addr1 as *mut Option<&String>;
//          ^^^^^ lifetime shortening happens here
ptr::write(addr2, Some(&s)); // ok!

In this case addr1 is a const pointer with a static inner reference. However the compiler will notice that if it shortened the lifetime inside addr1 just before casting it to mutable, the lifetime would be short enough. It can do this because addr1 is const and therefore covariant.

let mut p: Box<Option<&String>> = Box::new(Some(&s0));
let addr1 = p.ptr;
//          ^^^^^ lifetie shortening can happen here

let addr2 = addr1 as *mut Option<&String>;
//          ^^^^^ or here
ptr::write(addr2, Some(&s)); // ok!

In this case the lifetime is also shortened by the virtue of some reference being const. It can actually happen in two places in this case.

2 Likes

@alice has detailed the reasoning, so I'm just gonna summarize the key concepts: type F<X> = *const X is covariant whereas type G<X> = *mut X is not (that is, by the way, the only actual difference between *const and *mut).

  • Also, and not related to subtyping although it may look similar, *mut Thinggy can coerce to *const Thinggy, so in your experiments if you start from *mut X and try to see if it typechecks against *const Y, know that the result depends on whether Rust chooses to coerce before subtyping (i.e., does it use an intermediate *const X "state"?), or the other way around (i.e., *mut X must be a subtype of *mut Y, which given the invariance of *mut _, means that X and Y would need to be equal).

Here is a playground you can toy with to test variance and covariance things: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=c2d6ab91cea297f87ea999982b1f03e0


More generally, one must see these variance questions as the monotonic properties of "math functions":

Asking whether F<X> : F<Y> is like asking whether f(x) ≥ f(y).

We know for sure that f(x) ≥ f(y) if:

  • x ≥ y and f is monotonically increasing,

  • y ≥ x and f is monotonically decreasing,

  • and the "duh" case: x == y (because then f(x) == f(y) which implies that f(x) ≥ f(y))

Similarly, we know for sure (i.e., can type-check) if:

  • X : Y and F<_> is covariant,

  • Y : X and F<_> is contravariant,

  • and the "duh" case: X and Y are the same type.

And it so happens than the "function" type F<X> = *mut X is neither covariant nor contravariant (this is called "invariant"), meaning that from x ≥ y we cannot deduce that f(x) ≥ f(y) or that f(y) ≥ f(x).

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.