How does the compiler determine the variable escape outside of closure?

help
1

Consider this example:

struct Scope<'a,'b>(PhantomData<* mut &'a ()>,PhantomData<* mut &'b ()>);

impl<'a,'env> Scope<'a,'env>{
	fn spawn<F>(&'env self,mut f:F) where F:FnMut(){
		f();
	}
}
fn scope_spawn<'b,F>(mut f:F)
where F:  for<'a> FnOnce(&'a Scope<'a,'b>){
}
fn main(){
  	scope_spawn(|s|{
		s.spawn(||{  // #1
		});
	});
}

In this example, the compiler reports an error at #1:

borrowed data escapes outside of closure
s escapes the closure body here

What's the reason here, if the lifetime ''a' introduced in the higher-ranked bound is changed to declare in the signature of scope_spawn, something like this

fn scope_spawn<'a, 'b:'a,F>(mut f:F)
where F:  FnOnce(&'a Scope<'a,'b>){
}

That error will be eliminated. What's the difference here? IIUC, the compiler error may be caused by that for<'a> FnOnce(&'a Scope<'a,'b>) imposes that 'b:'a, so borrowing the &'a Scope<'a,'b> for 'b will exceed the lifetime of &'a Scope<'a,'b>, however, why the changed one does not have an error caused by the same reason?

2

Maybe you find this comment helpful?

The whole discussion is quite in-depth.

The docs might also be helpful.

IIRC the HRTB on 'scope is needed so that you can never leak a Scope handle outside of the closure you pass to thread::scope.

1 Like
3

HRTB requires the FnOnce impl is valid for all 'a. The latter one only requires it's valid for one specific 'a (for example, the compiler is free to choose 'a == 'b).

1 Like
4 
struct Scope<'a,/* 'b */>(PhantomData<* mut &'a ()>,/* PhantomData<* mut &'b ()>*/);
impl<'a,/*'b*/> Scope<'a,/*'b*/>{
    fn take<T:'a>(&'a self, t:T) where T:FnMut(){}
}

fn test<'d,F>(f:F)
where F: for<'a> FnOnce(&'a Scope<'a>){

}

fn main(){
    let i = 0;
    test(|s|{
      s.take(||{
         let r = i;  // #1
      })
   });
}

Without the second lifetime in the comment, #1 has an error argument requires that i is borrowed for 'static. I wonder whether the &'a Scope<'a,'d> implicit that 'd:'a, and in this situation does 'a denote any lifetime anymore? Is where F: for<'a> FnOnce(&'a Scope<'a,'d>) a higher-rank trait bound anymore?

5

No, there is no implicit 'd: 'a. For reference, Scope has the 'd: 'a (aka 'env: 'scope) bound in its declaration.

Sure. It just happens to constrain 'd if we have the 'd: 'a bound in Scope or in the function signature of test:

To get a for<'scope where 'env : 'scope> quantification in current Rust, we carry an implicit 'env : 'scope bound inside the Scope type itself, so that for<'scope> quantifications that "see" the Scope<'scope, 'env> type end up correctly constrained. Hence why Scope also needs to be infected with 'env 2.

IIUC, the difference between 'd: 'a and for <'a where 'd:'a> (which is not real syntax but used to describe the relationship in the tracking issue of scoped threads) is that 'd can never be equal to 'a, because 'a is not a lifetime that the caller can specify. Hence we can't leak a Scope or a ScopedJoinHandle outside of the closure we pass to test (aka thread::scope).

1 Like
6

However, even though there is no trait bound like 'd:'a in the declaration of Scope, it still work, if comment the second lifetime for Scope, the compiler says i should be borrowed for 'static, what's the reason here?

7

Try using functions instead of closures in your OP for potentially better errors.

fn outer<'a, 'b: 'b>(s: &'a Scope<'a, 'b>) {
    let _s: &'b Scope<'_, 'b> = s;
    // s.spawn(|| {});
}

spawn requires the reference lifetime be the same as the second lifetime parameter; the lifetimes are invariant so you can't shrink and you would need to create a &'b Scope<'_, 'b> specifically; you only have a reference lifetime equal to the first lifetime parameter, so you get a lifetime error.

It does denote any lifetime. For example, it could denote 'static. The error message mentioning 'static is just an example of a lifetime that could be problematic. It doesn't mean 'a can only be 'static or that 'static is the only problematic lifetime.

&'a Scope<'a, 'd> in a signature does imply 'd: 'a. (But perhaps making this a property of std's Scope itself is needed elsewhere. Or maybe they're just being prudent.)

I don't see why 'd couldn't be equal. Edit: Ok, after refreshing my memory, maybe you meant in the std Scope implementation specifically; I think that's true there (but it's not a conclusion you can make from the bound).

But at least one purpose of the higher-ranked bound in this case and cases like GhostCell is that the closure body has to work for all lifetimes 'a, which mean the closure body can't "know" what it is; you can't assume that it's equal (or not equal) for example.

The where 'd: 'a in the case of Scope is to put an upper limit on the unknown lifetime 'a.

8

Yes, I think I misinterpreted the following quote (my emphasis) from this comment when I said 'd can never be equal to 'a:

"some lifetime that is internal to 'scope but longer than the call to this closure"

I wrongfully equalled 'scope to specify the lifetime of the closure call, but it is slightly longer than that:

The 'scope lifetime represents the lifetime of the scope itself. That is: the time during which new scoped threads may be spawned, and also the time during which they might still be running. Once this lifetime ends, all scoped threads are joined. This lifetime starts within the scope function, before f (the argument to scope) starts.

So 'env can be equal to 'scope, but 'scope (thanks to the invariance of it in Scope) will always be longer than the lifetime of the closure call.

1 Like
9

@quinedot Seems there is a remaining confusion here

struct Scope<'a,'b>(PhantomData<* mut &'a ()>,PhantomData<* mut &'b ()>);

impl<'a,'env> Scope<'a,'env>{
	fn spawn<F>(&'env self,mut f:F) where F:FnMut(){
		f();
	}
}
// fn scope_spawn<'b,F>(mut f:F)
// where F:  for<'a> FnOnce(&'a Scope<'a,'b>){
// }
fn scope_spawn<'a, 'b:'a,F>(mut f:F)
where F:  FnOnce(&'a Scope<'a,'b>){
}
fn main(){
  	scope_spawn(|s|{
		s.spawn(||{  // #1
		});
	});
}

Why is this example ok? At #1, the type of s is &'a Scope<'a,'b>, and s.spwan(...) can be simplified to let tmp:& 'b Scope<'a,'b> = s, the lifetime 'a and 'b is invariant on Scope, the implicit requirement in the signature of scope_spawn is 'b:'a(inferred from type &'a Scope<'a,'b>) and that requirement is also the explicit one, we didn't impose 'a:'b such that 'a == 'b, why is this example ok?

10

You've taken away the requirement that the caller of scope_spawn work with every lifetime 'a shorter than 'b, and given the caller the ability to choose 'a (and still choose 'b). The caller can just choose any lifetime 'x so that the out closure accepts &'x Scope<'x, 'x> for example, which satisfies the signature of spawn.

The key part is that you gave away the choice of 'a to the caller. That took away the imposition that the closure works with some unknown lifetime (within the constraints). With the higher-ranked bound, the closure has to work with lifetimes shorter than 'b. When the caller can choose a single lifetime, it doesn't have to.

1 Like
11

Hmmm, that is, the caller selects a lifetime for 'a that is the same as 'b, and the selected one forms a function whose parameter type is &'b Scope<'b,'b> that also satisfy the requirement in fn scope_spawn<'a, 'b:'a,F>(mut f:F) where F: FnOnce(&'a Scope<'a,'b>){ } because 'b:'b.

In contrast to the first example where the compiler reports an error. Since 'a and 'b are invariant on Scope, if s.spawn were to success, the compiler would select 'a as 'b, which would implement the closure not general enough, so the compiler can only select one from two options, it selects one that the implementation is general enough

1 Like
12

@quinedot A variant example:

struct Scope<'a,'b>(PhantomData<&'a ()>,PhantomData<&'b ()>);

impl<'a,'env> Scope<'a,'env>{
	fn spawn<F>(&'env self,mut f:F) where F:FnMut(){
		f();
	}
}
fn scope_spawn<'b,F>(mut f:F)
where F:  for<'a> FnOnce(&'a Scope<'a,'b>){
}
fn main(){
  	scope_spawn(|s|{
		s.spawn(||{  // #1
		});
	});
}
use std::marker::PhantomData;

In this example, the lifetime 'a and 'b are covariant on Scope, the signature of scope_spawn implicitly has a requirement 'b:'a(i.e. for<'a where 'b:'a>), the &'a Scope<'a,'b> can be coerced to &'b Scope<'a,'b> because it is regulated to by another subtype relation, right?

Higher-ranked function pointers and trait objects have another subtype relation. They are subtypes of types that are given by substitutions of the higher-ranked lifetimes.

Since non-higher-ranked lifetime 'b substitutes higher-ranked lifetime 'a, hence &'a T is a subtype of &'b T, the coercion is permitted here, so the example can be compiled, right?

13

&'a _ can't expand to &'b _, but it can coerce down to &'a Scope<'a, 'a> or some smaller &'b0 Scope<'a0, 'b0> that meets the bound.

There's a higher-ranked trait bound here, but no higher-ranked types. (for<..> as part of the type (which might be elided for fn pointers and dyn Fn types)).

14 
  	scope_spawn(|s:/* &'a Scope<'a,'b> */|{
       // let borrow:&'b Scope<'b,'b> = s;  // #1
		/* s => borrow */.spawn(||{  
		});
	});

As shown in #1, we borrow a reference from s to call spawn, is 'a representing higher-ranked lifetimes in this context? It appears to me that the lifetime 'a is like something as if it were introduced like

for<'a where 'b:'a> | s:/* &'a Scope<'a,'b> */ |{
    // let borrow:&'b Scope<'b,'b> = s;  // #1
   // borrow.spawn(||{});
}

'b:'a expresses that 'b outlives 'a? If 'a is a higher-ranked lifetime, &'a T should also be the subtype of & 'b T, right? So coercing &'a T to &'b T is ok and vice versa, right?

15

You're annotation on s: is correct, with the additional information that it has to work for every 'a[1] but only one 'b.

But #1 can't be a &'b Scope<'b, 'b> because 'a may be shorter than 'b and covariant lifetimes cannot grow longer, only shorter. However, it could be &'a Scope<'a, 'a>,[2] because 'b is also covariant in this example.

Yes.

&'a T isn't a higher-ranked type. In order to call the closure, one specific lifetime 'a has to be chosen first. The body in this example has to work for any 'a,[3] but 'a is a single lifetime within the body. There are no higher-ranked types in this example, so the sub/supertype relationships around higher-ranked types is a red herring (an irrelevant distraction).

Rust has no for<'a> &'a T types -- no higher-ranked reference types. Function pointers and dyn Trait are the only higher-ranked types.

In covariant position, longer lifetimes are subtypes of shorter lifetimes (which can be confusing). You can coerce a subtype to a supertype but not vice-versa.[4] So you can coerce &'b T to &'a T but not vice-versa.

  1. where 'b: 'a ↩︎

  2. The compiler doesn't have to choose any particular lifetimes, it just has to prove that a solution exists. ↩︎

  3. where 'b: 'a ↩︎

  4. Every cat is an animal, but not all animals are cats. ↩︎

16

I thought any type constructed from a lifetime parameter that denotes any lifetime was a higher-ranked type, but I'm wrong.

BTW, is a function type a higher-ranked type, for example:

fn foo<'a>(v:&'a i32){}

Is the type of foo higher-ranked type? It seems not.

Function pointers and dyn Trait are the only higher-ranked types.

17

As far as I know, function item types are not higher-ranked types.

1 Like
18

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.