NLL vs dropglue? Lifetime parameters on traits

help
1

I noticed an interesting lifetime problem from
You probably should avoid putting lifetime parameters on traits.

The example in that blog is

trait Lockable {
    type Guard<'a>: Locked<'a> where Self: 'a;
    fn lock(&self) -> Self::Guard<'_>;
}
trait Locked<'a> {
    type Iter: Iterator<Item = &'a u32> + 'a;
    fn iter(&'a self) -> Self::Iter;
}
fn test<T: Lockable>(t: &T) {
    let x = t.lock();
    x.iter();
}

error[E0597]: `x` does not live long enough
  --> src/main.rs:16:5
   |
16 |     x.iter();
   |     ^^^^^^^^ borrowed value does not live long enough
17 | }
   | -
   | |
   | `x` dropped here while still borrowed
   | borrow might be used here, when `x` is dropped and runs the destructor for type `<T as Lockable>::Guard<'_>`

Rust Playground

The explanatiom from the author:

fn test<'lifetime_of_t, T: Lockable>(t: &'lifetime_of_t T) {
   let x = t.lock();
   // ...
}

This mean the type of x is actually T::Guard<'lifetime_of_t>, which implements Locked<'lifetime_of_t>. And since Locked<'_> is invariant w.r.t the lifetime, Guard implementing Locked<'lifetime_of_t> doesn't mean it implements Locked<'shorter> for any 'shorter lifetime. When we call x.iter(), it can only return Locked::Iter: 'lifetime_of_t. Which means x is actually borrowed for 'lifetime_of_t! It's borrowed for a lifetime that is actually longer than its own lifetime! (I think it's fair to say rustc's diagnostic here can use some polish.)

But I think it's not the reason, since covariance can happen before the invariance, thus x is not of type T::Guard<'lifetime_of_t>. The error is the same when we test t: T instead of t: &T.

The author gave a non-GAT version failed code emitting the same error: Rust Playground

trait X<'a> {
    type Borrowed: 'a;
    fn borrow(&'a self) -> Self::Borrowed;
}

trait Y<'a> {
    type X: X<'a> + 'a;
    fn get_x(&'a self) -> Self::X;
}

fn check<T: for<'a> Y<'a>>(v: T) {
// fn check<T: for<'a> Y<'a>>(v: &T) {
    let x = v.get_x();
    x.borrow();
}
1 Like
2

Indeed. The problem they diagnosed is right, just not for 'lifetime_of_borrow_of_t, but rather, 'lifetime_of_x:

  • that lifetime, no matter how small/shrinked by covariance, is still inside x: Locked<'lifetime_of_x>.

  • By virtue of being a fully opaque type, it has to be conservatively assumed to have drop glue using '_ (which, btw, is indeed the case of a MutexGuard<'_, …>, so here the compiler is completely right);

  • The problem comes from the resulting fn iter signature, which in MutexGuard-parlance would have been equivalent to the good old anti-pattern:

    // trait Locked<'a> {
impl<'a> MutexGuard<'a, […]> {
   fn iter<'a>(self: &'a MutexGuard<'a, […]>) -> …
    • (which, for MutexGuard still works since 'a is covariant there, but not for an opaque impl Locked<'a>)

    It thus requires a .iter() borrow which itself also lasts for 'lifetime_of_borrow_of_x.

This last bullet thus overlaps with the drop glue from the second bullet.

Solution

They need a lifetime-generic fn iter<'any>(self: &'any Lockable::Guard<'lock>)

  • rather than a lifetime-fixed fn iter(self: &'lock Lockable::Guard<'lock>)

Should they still have a trait Locked<'lifetime_here_rather_than_in_the_fn> (which the author claims is an anti-pattern; I'd rather say that it just shouldn't be written unnecessarily), the solution to make it "generic again" is higher-order/higher-rank/for<>/universal quantification:

  trait Lockable {
-     type Guard<'lock>: Locked<'lock> where Self: 'lock;
+     type Guard<'lock>: for<'iter> Locked<'iter> where Self: 'lock;

And hey, look at that, everything is rather readable the moment you stop naming lifetime parameters 'a everywhere like the Rust book incorrectly showcases :pensive:

Technicality

The issue right now with higher-order quantification is that by default it is unbounded not only towards the "small lifetime" side (allowing the usage of short-lived / local-to-the-function borrows, as desired), it's also unbounded towards the 'static side. And this can make concrete MutexGuard<'lock, …> implementations not be able to satisfy this, since they are upper-bounded by 'lock.

  • Ideally, we'd need a for<'iter where 'lock : 'iter> quantification, which is not possible in Rust to keeps things seemingly simpler for beginners, and painful for everybody else.

  • Another option that could strike a nicer simple-and-yet-flexible balance is a forlocal<'iter> quantification[1]:

    type Guard<'lock> : forlocal<'iter> Locked<'iter>
where
    Self : 'lock,

In current Rust, this can only be achieved through implicit bounds, which would happen by doing something like:

/// So that `Locked<'iter>` carries an *implicit*
/// `where Self : 'iter` bound.
trait Locked<'iter, ImplicitBound = &'iter Self> {
  • so that Self : for<'iter> Locked<'iter> is actually Self : for<'iter where Self : 'iter> Locked<'iter>

  1. which would be implemented as an existentially-upper bounded quantification for those curious ↩︎

3 Likes
3

Thanks for your awesome reply. It's totally reasonable to me except this sentence:

Could you elaborate on it a bit more?

1 Like
4

Sure! I was indeed a bit terse in that area. We have:

   let locked: T::Guard<'lock> = thing.lock();
   let iter: <T::Guard<'lock> as Locked<'lock>>::Iter = locked.iter();
// drop(iter);
// drop(locked);
// <- 'lock cannot end before this point

  1. With 'lock being the lifetime of the borrow of thing in that 1st line, which must span beyond the 4th line since it "uses" the locked instance and thus its 'lock lifetime inside it (no NLL due to conservative/potential drop glue).

    • (hence the 'lock cannot end before… line)

  2. The locked local, itself, is borrowed for 'lock too in that 2nd line, since that's the only operation the trait Locked<'lock> allows for (contrary to a for<'any> Locked<'any>).

  3. So it means that the locked variable is borrowed until that final "beyond-4" line, but the locked variable is dropped at that 4th line, hence the problem.

Another way of seeing that 3rd bullet is to look at it like the borrow checker does: what are the constraints/inequalities on 'lock?

  • It must span beyond 4 for the usage of locked to work: 'lock > '4 in pseudo-code;

  • It must have ended before 4 so that the borrow of locked that occurs in the .iter() line can end (otherwise locked would be undroppable): '4 > 'lock in pseudo-code.

These constraints have no solutions / cannot be satisfied, hence triggering a borrow-checker error.

To expand on this: the return value of .iter() plays no role whatsoever. You could consider having trait Locked<'iter> { fn iter(&'iter self); } and we'd still have this problem.

Much like you cannot call a fn borrow_for_static(it: &'static impl ?Sized); function on the borrow of a local variable, even through borrow_for_static does not return anything. Here, the lifetime-restricted Locked trait was thus offering a .borrow_for_'lock() kind of function, which is why the drop(iter); line plays no role whatsoever.

3 Likes
5

:heart: The core deduction is what I'm looking for and appreciate!

You're the lifetime master :kissing_heart:

BTW: I know you're writing a new lifetime book Rust lifetimes: from 'static to ecstatic [1], and I've recommended it to people wanting to learn more about lifetimes. Your explanations are always helpful and informative. Thank you very much for your help. Best regards!

  1. and the repo is here ↩︎

3 Likes
6

Hi, author of the blog post here. Thank you for this awesome explanation!

IIUC, I was wrong in saying that type of locked is Guard<'lifetime_of_t>, instead it's Guard<'lock> whose lifetime argument is shorter. Otherwise my point on invariance of trait lifetimes still stands. And locked in a way is indeed borrowed for longer than its entire lifetime (as it's borrowed for 'locked, which lasted beyond drop of locked). Which means my explanation is only slightly wrong, I would call that a win! :laughing:

I agree the compiler was 100% correct, just it's diagnostic being confusing. Surely there is a better way to word it.

7

the solution to make it "generic again" is higher-order/higher-rank/for<>/universal quantification:

I have to disagree with you here. If you think about the semantics of a lock guard, it cannot implement Locked<'iter> for all 'iter. When the guard is dropped, there can be no borrow of data behind the lock exists. It's more like this (using non-existent syntax):

type Guard<'lock>: for<'iter where 'lock: 'iter> 
    Locked<'iter> + 'lock where Self: 'lock;

But at this point why not just drop the lifetime on Locked, which solves the problem too.

8

True, but the diagnostic in that case is much easier to understand:

error[E0597]: `x` does not live long enough
 --> src/lib.rs:5:23
  |
5 |     borrow_for_static(&x);
  |     ------------------^^-
  |     |                 |
  |     |                 borrowed value does not live long enough
  |     argument requires that `x` is borrowed for `'static`
6 | }
  | - `x` dropped here while still borrowed
9

Yes, the invariance does exsist in your example, but it's just not that important:

  • and you can see what I mean by saying covariance can happen before the invariance: Rust Playground

Your newest explanation still doesn't make sense to me:

Let's call the type of x T::Guard<'lock> . 'lock is the lifetime of the implicit borrow of t that happened when we called t.lock() . Since x borrows from this lifetime (because of the signature of fn lock ), 'lock must last longer than x .
... Which means x is actually borrowed for 'lock ! It's borrowed for a lifetime that is actually longer than its own lifetime!

I'd say 'lock doesn't need to last longer than x. Read the first reply from @Yandros.

We don't need that syntax, since it works with what we now have as the first reply raised:

It wroks due to the lifetime bounds implied by the compiler.

The last thing I totally agree with @Yandros about your blog post is:

10

Well that's from this paragraph from @Yandros:

Specifically the first point.

Yeah, thanks for correcting me. But I'd rather argue this is a point against the teachability of Rust. Implied lifetime bounds are invisible, and you "just have to know it", which I don't think is a good thing.

I'd rather say that it just shouldn't be written unnecessarily

This is what I was trying to say with this blog post too.

11

Heh, this was supposed to remain low profile :shushing_face: until it was more fully-fledged :upside_down_face: but I'm glad you already like the embryos of posts I have in there :blush:

Indeed! I think you and @vague were both right: you had the right understanding overall, but for the 'lifetime_of_t technicality, and @vague had correctly identified that 'lifetime_of_t unaccuracy :slightly_smiling_face:

Oh definitely, the compiler is not always able to produce nice lifetime diagnostics

One can imagine so, yeah, but we wouldn't expect a compiler error to emit a whole post like the ones in in this forum, so it's hard to gauge what it should say and what it shouldn't (but having an example of an ideal error message can be very helpful to letting somebody else take at stab at tweaking the compiler and trying to implement the nicer diagnostic); it's definitely a hard thing (but rust contributors have historically not let this difficulty deter them :muscle:, which is why thanks to them we already have these awesome diagnostics, but because of which the cases were they don't shine do stand out :sweat_smile:)

You're completely right, it's not fully universal :ok_hand: But I stand by my "generic again" phrasing, since it's much like a fn iter_generic<'iter>(&'iter self) ... would have been: it's generic, but with 'iter (implicitly) bounded by Self : 'iter.

I mention this, and the proper workaround, here:

I think I phrased my remark poorly, my bad. I just wanted to insist on not demonizing generics, even if it was your point as well, just in case the title itself, alone, ("probably should avoid"), were deemed a bit ambiguous. But you are right that many people make the mistake of making things excessively generic. I think @BurntSushi phrases all this very elegantly in that reddit thread :slightly_smiling_face:, especially regarding that conclusion:

Which in the case of Locked, for instance, indeed yields your:

2 Likes
12

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.