Change Lifetime on Struct with Generics

I am trying to create a library that safely allows having a reference to non-'static data in a 'static context by dynamically holding the reference active until it is no longer accessed. Feel free to just tell me about an existing crate that does this. My crate is called static-newtype .

With simple references, it works:

use static_newtype::ParentNewtype;

fn main() {
    // not available in 'static lifetime
    let mut data = 0;

    let parent = ParentNewtype::new(&mut data);
    let child = parent.child();
    // move access to non-'static data into place where static data is required
    moo(move || {
        println!("{}", {
            let child = child.get().unwrap();
            *child
        });
    });

    //destructor on ParentNewType blocks until all children are dropped, so the reference is kept active
}

fn moo(test: impl FnOnce() + 'static) {
    test();
}

But, when I try to use a non-static struct with non-static references (which I need in my use case), I get errors:

use static_newtype::ParentNewtype;

struct RefContainer<'i> {
    u: &'i mut u32,
}

fn main() {
    // not available in 'static lifetime
    let mut data = 0;
    let mut ref_container = RefContainer { u: &mut data };

    let parent = ParentNewtype::new(&mut ref_container);
    let child = parent.child();
    // move access to non-'static data into place where static data is required
    moo(move || {
        println!("{}", {
            let child = child.get().unwrap();
            *child.u
        });
    });
    
    //destructor on ParentNewType blocks until all children are dropped, so the reference is kept active
}

fn moo(test: impl FnOnce() + 'static) {
    test();
}

gives

error[E0597]: `data` does not live long enough
  --> examples/struct.rs:10:47
   |
9  |     let mut data = 0;
   |         -------- binding `data` declared here
10 |     let mut ref_container = RefContainer { u: &mut data };
   |                                               ^^^^^^^^^ borrowed value does not live long enough
11 |
12 |     let parent = ParentNewtype::new(&mut ref_container);
   |                  -------------------------------------- argument requires that `data` is borrowed for `'static`
...
23 | }
   | - `data` dropped here while still borrowed

For more information about this error, try `rustc --explain E0597`.

I know that this is because (see my library's code below) I am using the same T to refer to RefContainer<'a>, RefContainer<'static>, and then again when returning from my Deref implementations, despite the fact that these are all different types. But, I do not now how to fix this issue since I do not know how to go between these types when all that is available is a generic parameter T.

This is related to GATs, but GATs only work with traits and I do not see a way to transform this into something that's sufficiently trait-based for this to work.

My library:

use parking_lot::{ArcRwLockReadGuard, ArcRwLockWriteGuard, RawRwLock, RwLock};
use std::marker::PhantomData;
use std::ops::{Deref, DerefMut};
use std::sync::{Arc, Weak};

pub struct ParentNewtype<'a, T: 'a> {
    data: Arc<RwLock<Option<*mut T>>>,
    _phantom: PhantomData<&'a ()>,
}

pub struct ChildNewtype<T> {
    data: Weak<RwLock<Option<*mut T>>>,
}

pub struct ChildNewtypeReadGuard<T> {
    data: ArcRwLockReadGuard<RawRwLock, Option<*mut T>>,
}

pub struct ChildNewtypeWriteGuard<T> {
    data: ArcRwLockWriteGuard<RawRwLock, Option<*mut T>>,
}

impl<'a, T: 'a + Sized> ParentNewtype<'a, T> {
    pub fn new(data: &'a mut T) -> Self {
        Self {
            data: Arc::new(RwLock::new(Some(unsafe { std::mem::transmute(data) }))),
            _phantom: PhantomData::default(),
        }
    }

    #[inline]
    pub fn child(&self) -> ChildNewtype<T> {
        ChildNewtype {
            data: Arc::downgrade(&self.data),
        }
    }
}

impl<T> ChildNewtype<T> {
    pub fn get(&self) -> Option<ChildNewtypeReadGuard<T>> {
        let arc = self.data.upgrade()?;
        let data = arc.read_arc();
        if data.is_none() {
            return None;
        }
        Some(ChildNewtypeReadGuard { data })
    }

    pub fn get_mut(&self) -> Option<ChildNewtypeWriteGuard<T>> {
        let arc = self.data.upgrade()?;
        let data = arc.write_arc();
        if data.is_none() {
            return None;
        }
        Some(ChildNewtypeWriteGuard { data })
    }
}

impl<T> Deref for ChildNewtypeReadGuard<T> {
    type Target = T;

    #[inline]
    fn deref(&self) -> &Self::Target {
        let ptr = self.data.clone().unwrap(); //unwrap will not panic because this is created only via ChildNewtype::get which checks for None, and we hold a & reference to it continuously preventing it from being changed out from under us
        let ptr_const = ptr as *const T;
        unsafe { &*ptr_const } //safety, we create a reference with the same lifetime as Self, but keeping Self keeps the original data alive via the Drop implementation
    }
}

impl<T> Deref for ChildNewtypeWriteGuard<T> {
    type Target = T;

    #[inline]
    fn deref(&self) -> &Self::Target {
        let ptr = self.data.clone().unwrap(); //unwrap will not panic because this is created only via ChildNewtype::get which checks for None, and we hold a & reference to it continuously preventing it from being changed out from under us
        let ptr_const = ptr as *const T;
        unsafe { &*ptr_const } //safety, we create a reference with the same lifetime as Self, but keeping Self keeps the original data alive via the Drop implementation
    }
}

impl<T> DerefMut for ChildNewtypeWriteGuard<T> {
    #[inline]
    fn deref_mut(&mut self) -> &mut Self::Target {
        let ptr = self.data.clone().unwrap(); //unwrap will not panic because this is created only via ChildNewtype::get which checks for None, and we hold a & reference to it continuously preventing it from being changed out from under us
        unsafe { &mut *ptr } //safety, we create a reference with the same lifetime as Self, but keeping Self keeps the original data alive via the Drop implementation
    }
}

Don't.

You are creating unsound code. Stop trying to circumvent rust and instead write algorithms that are structured within the limits of lifetimes.

1 Like

The use case here is to deal with libraries that require 'static when they don't need to, probably to simplify their code. I find that I encounter this a few times per year. It isn't feasible, because limited time, for me to learn the internals of every such library that I encounter to PR a fix.

If my solution is unsound, please give feedback on how to correct that.

It definitely is.

The closes thing I can think of as far as a suggestion goes is to use something like replace_with to temporarily work with owned data when only given a &mut OwnedData. Maybe you could adapt a similar approach with your starting point (but I haven't thought it through).

You mean it's not feasible to investigate some library internals when you have time and not under pressure, but it's totally feasible to do the same when something is deployed and clients are ringing off the hook and your boss is asking to report every hour?

You have strange priorities.

Knowing whether library actually needs 'static bound or not is not possible without deep investigation and, worse yet, even if it's implementation doesn't need it today it may start relying on it in any minor update.

In many libraries that “not needed” 'static bound is there precisely to be able to issue such updates – which means that the only way to use such library with a trick crate like yours is to fork it and thoroughly investigate its internals (because now you are on your own and couldn't rely on support from upstream)… at this point removing 'static bound is trivial.

If a library requires 'static, please describe why do you think it is unnecessary? Please describe some specific examples. It is hard to believe that the requirement is unnecessary, but maybe you are right and I am just not aware of these particular libraries.

Most often these are crates made by people new to Rust who often don't even know that dyn Foo means dyn Foo + 'static or who just add + 'static when compiler suggests it.

It's not surprising that there are crates like these, but the problem is that if someone introduced such bound without thinking then chances are very high that, as the next step s/he can do something permitted by such bond, too! Like passing it into some async function or another thread where it may outlive the creator. Worse: they may do that, as I have said, in any minor update because it's not a breaking change in the public API.

The only way to ensure that library that includes seemingly “useless” 'static bound doesn't do anything “criminal” with it is to thoroughly investigate it (by removing that bound and trying to make compiler happy). But after you'll do that you would have a fork of such a crate which would no longer have 'static bound and then playing with unsafe would no longer be needed.

2 Likes

I have a question.

I am trying to create a library that safely allows having a reference to non-'static data in a 'static context by dynamically holding the reference active until it is no longer accessed.

Data pointed to by the 'static lives for the entire program, and @john01dav wants to have a reference to a non-static variable available for the whole program.

Couldn't this be solved using Pin and Box? Use Box to create a smart-pointer to the heap you want to hold. The use Pin so the data is not moved.

Now this data will remain available for the whole current scope. Then use Box::leak to convert the lifetime from the current scope to a lifetime out of scope (e.g 'static)?

Note that 'static values are capable of existing for the entire lifetime of the program, but don't necessarily, and they often don't. You can think of them as owned values vs borrowed values.

Box, Rc, Arc, can help with this, yes. (I don't think the same is true of Pin, but I know little about it.) But if you have an owned value, you don't necessarily need Box (etc) to make it 'static.

If you do use Box (etc), you have to Box the original, owned value. Once you have a reference to a value, you can't extend the borrowed lifetime with Box.

1 Like

If they have a T: 'static, they don't need Pin or anything,[1] they can just Box::leak(Box::new(the_t)). As far as I understand the use case, they don't have a T: 'static, they have a &'non_static mut _ as their starting point.

You can't Box::leak(Box::new(a_non_static)) and get a &'static mut _.


  1. why do you think it needs to be pinned in place? ↩︎

3 Likes

After mulling a bit I don't think there's any salvaging it.[1] One can just

    let parent = ParentNewtype::new(&mut data);
    let child = parent.child();
    std::mem::forget(parent);
    // The borrow of data has ended here

leaving you with the active Arc<Rwlock<T>>-esque child after the borrow of data has expired, with no way to check for conflicts.[2]


  1. "it" == a sound, non-unsafe API of the same basic shape as the OP ↩︎

  2. If Rust had drop guarantees, you could abort or deadlock in the parent's destructor if Arc::into_inner failed, but Rust does not have drop guarantees. Even if it did, I'm not sure how acceptable you would find that behavior. ↩︎

In a lot of cases the library does need 'static to be safe generally, but not so with the way that I am using it. So, I thought to create a system that enforces it dynamically, like how interior mutability can enforce safety rules dynamically. Although, I have also encountered what @khimru spoke of.

1 Like

My plan was to make this a sound API that enforces such guarantees dynamically (like interior mutability), such that I would not have undefined behavior problems later.

Oh, you're right. Good point. Thank you for this. I had intended to have a Drop implementation that mutably borrows the RwLock and then changes the data to None, after which future attempts to access it fail. It would block if needed to keep the reference alive. It looks like I forgot to write that. But, I did not know that Drop was not guaranteed to run. Hence, that will not work. I could make my ParentNewtype::new constructor unsafe, and specify that users must ensure that the destructor runs and then this could possibly be a useful construct.