Lifetimes in map closure?

Hello,

Is it possible to specify the lifetime of an anonymous map closure or the smart-pointer "guard" object, such that I can borrow stuff from the outside?

Here is what I'm trying to do:

use parking_lot::{RwLock, RwLockReadGuard, MappedRwLockReadGuard};

struct Accessor<'a> {
    lock: &'a RwLock<i32>,
    other: &'a i32
}

impl <'a>Accessor<'a> {

    fn access(&self) -> MappedRwLockReadGuard<i32> {

        let guard = self.lock.read();
        let other = self.other;
        RwLockReadGuard::map(guard, |_guard| {
            other
        })
    }
}

fn main() {
 
 let mut lock = RwLock::new(42);
 let mut accessor = Accessor{lock: &mut lock, other: &21};
 
 let value = *accessor.access();
 
 println!("{}", value);
}

In this case, 'a needs to outlive the MappedRwLockReadGuard

Thank you.

No, this isn't possible. RwLockReadGuard::map is defined as:

impl<'a, R: RawRwLock + 'a, T: ?Sized + 'a> RwLockReadGuard<'a, R, T> {
    pub fn map<U: ?Sized, F>(s: Self, f: F) -> MappedRwLockReadGuard<'a, R, U>
        where
            F: FnOnce(&T) -> &U,

The only lifetime that can be returned from that closure is the lifetime of the argument to the closure.

1 Like

The bounds on map are:

pub fn map<U: ?Sized, F>(s: Self, f: F) -> MappedRwLockReadGuard<'a, R, U> where
    F: FnOnce(&T) -> &U, 

And that last one desugared a bit is:

    F: for<'any> FnOnce(&'any T) -> &'any U, 

This is what the error is about: You're supposed to work for any lifetime, but due to your capture, you can only work for lifetimes capped by 'a.

You can exploit the implicit bounds of references to make this work. For example if T is inherently limited to 'a, then the bound involving &'any T will have an implicit 'a: 'any, and the entire bound:

    F: for<'any> FnOnce(&'any T) -> &'any U, 

Will act like the (not possible to write directly in Rust) bound:

    F: for<'any where 'a: 'any> FnOnce(&'any T) -> &'any U, 

And here's one way to do that:

struct Accessor<'a> {
    // Infect the type with an implicit bound
    lock: &'a RwLock<(i32, PhantomData<&'a i32>)>,
    other: &'a i32
}

Granted, it's not very ergonomic.

3 Likes

You are a maestro!

Sorry, but I have a follow-up question.

It turns out that, if I try to implement is approach in the actual project, explicit lifetime annotations end up exploding everywhere, because the data behind the lock isn't actually an i32. It's not really workable.

But do you see anything unsound about using an unsafe transmute, like this:

struct Accessor<'a> {
    lock: &'a RwLock<(i32, PhantomData<&'static i32>)>,
    other: &'a i32
}

and then:

  let transmuted_lock: &RwLock<(i32, PhantomData<&'a i32>)> = unsafe{ std::mem::transmute(self.lock) };
  let proj_lock: RwLockReadGuard<'a, _> = transmuted_lock.read();

My thinking is that the definition of Accessor means the resulting MappedRwLockReadGuard won't outlive other, and the actual data behind the lock is 'static, so this should be safe.

Said more simply, I'm using the unsafe to tighten the bounds, not loosen them, so I don't think I will get into trouble.

Am I missing something?

Thanks again!

Do you have an example that exhibits the problem?

RwLock is invariant over its contents, and I'm hesitant to make any assumptions on that level.

Shortening lifetimes can be just as UB as lengthening them.

1 Like

Here is a simplified version that still exhibits the essential lifetime tangle that emerges.

use core::marker::PhantomData;
use parking_lot::{RwLock, RwLockReadGuard, MappedRwLockReadGuard};

type WithLifetime<'a, T> = (T, PhantomData<&'a T>);

// struct DataStore {
//     data: Vec<RwLock<i32>>,
//     other: i32
// }

struct DataStore<'a> {
    data: Vec<RwLock<WithLifetime<'a, i32>>>,
    other: i32
}

impl DataStore<'_> {
    fn accessor<'a>(&'a self) -> Accessor<'a> {
        Accessor{locks: &self.data, other: &self.other}
    }
}

struct Accessor<'a> {
    locks: &'a Vec<RwLock<WithLifetime<'a, i32>>>,
    other: &'a i32
}

impl <'a>Accessor<'a> {

    fn access(&self, idx: usize) -> MappedRwLockReadGuard<i32> {

        let lock = self.locks.get(idx).unwrap();
        let guard = lock.read();
        let other = self.other;

        RwLockReadGuard::map(guard, |_guard| {
            other
        })
    }
}

fn main() {
 
 let data_store = DataStore{data: vec![RwLock::new((42, <_>::default()))], other: 21};
 let accessor = data_store.accessor();
 
let value = *accessor.access(0);
 
 println!("{}", value);
}

This change lets it compile again:

-impl DataStore<'_> {
-    fn accessor<'a>(&'a self) -> Accessor<'a> {
+impl<'a> DataStore<'a> {
+    fn accessor(&'a self) -> Accessor<'a> {
         Accessor{locks: &self.data, other: &self.other}
     }
 }

Playground.

Alternatively you could separate the outer and inner lifetimes, but Accessor::access needs them to be the same so you can map from the inner lifetime to the outer lifetime.

However...

Either way is likely to make your overall use-case unworkable in combination with RwLock being invariant in the lifetime, because your borrow has to last the entire life of DataStore. I.e. the same restriction that made you consider transmute is still problematic.

I haven't thought of workaround yet.

2 Likes

You are 100% correct in surmising my use case and why the internal lifetime of the DataStore can't be the same as the accessor's lifetime.

I'm at a loss. From the outside, it seems like it should be allowable (in concept) to borrow outside data in the map closure - but the semantics of the function signature make it problematic.

Thanks for taking the time to look at it.

It does seem that a Fn(&'a T) -> &'a U bound would work. So this RFE might solve it (if applied to MappedRwLockRearGuard as well).

1 Like

Thank you again! Fingers crossed that issue will see some movement.

It looks like this might also help, and it looks like some kind of resolution is imminent.

Even it it's not the whole solution it looks like it paves the way for the fix you mentioned, which has been stalled for the last 7 months.

Thanks again!

I admit that I haven't followed the entire thread, but would something like this work?

struct DetachedReadGuard<'lock,T:?Sized> {
    guard: MappedRwLockReadGuard<'lock, ()>,
    data: &'lock T
}

impl<T> std::ops::Deref for DetachedReadGuard<'_, T> {
    type Target = T;
    fn deref(&self)->&T { self.data }
}

impl<'a,'b> Accessor<'a, 'b> {
    fn access(&self, idx: usize) -> DetachedReadGuard<'_, i32> {
        let lock = self.locks.get(idx).unwrap();
        let guard = lock.read();
        let other = self.other;

        DetachedReadGuard {
            // Below can be any non-borrowing function of guard
            data: if guard.0 % 2 == 0 { &*self.other } else { &0 },
            guard: RwLockReadGuard::map(guard, |_| &())
        }
    }
}
2 Likes

While that will be a boon, there are existing workarounds for the problem it fixes. It doesn't apply directly to your case because you don't have a closure that can be for<'a> |&'a i32| -> &'a i32. You want to be able to use a |&'something_specific i32| -> &'something_specific i32.

However, apparently it's holding up the RFE to allow a |&'something_specific T| -> U for reasons I didn't look into, just saw in passing.

1 Like

Nice. Indirect way to solve that RFE for owned data too (if one doesn't need the methods of the lock).

The implied lifetime hack isn't needed with this approach.

2 Likes

Your idea create a custom smart pointer is really good.

I do need to return a result that can borrow the stuff protected by the guard and the stuff behind the other reference, but I was able to adapt your idea to by moving all of the logic into the deref trait. So the access function now does practically nothing.

It feels wrong to me - like deref should be cheap instead of having a whole bunch of logic (including executing custom closures) - but it works.

Thank you!