Why rustc says the return value contains lifetime?

This is a simplified part of my project.

use std::any::Any;

pub struct Context;

pub trait Component: 'static {
    fn create(self, ctx: &Context) -> impl Any;
}

pub fn component<T>(prop: T)
where
    T: Component,
{
    move |ctx: &Context| prop.create(ctx);
}

And we got an error

error: lifetime may not live long enough
  --> src/lib.rs:13:26
   |
13 |     move |ctx: &Context| prop.create(ctx);
   |                -       - ^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
   |                |       |
   |                |       return type of closure `impl Any` contains a lifetime `'2`
   |                let's call the lifetime of this reference `'1`

Undoubtly, all the Any implementors have 'static lifetime, but why rustc says it capture lifetimes? Is that a bug? Do we have some alternatives to keep the opaque return type?

This works:

+#![feature(precise_capturing_in_traits)]
use std::any::Any;

pub struct Context;

pub trait Component: 'static {
-    fn create(self, ctx: &Context) -> impl Any;
+    fn create(self, ctx: &Context) -> impl Any + use<Self>;
}

References:

Implementation examples: Rust Playground

Example1:

impl Component for String {
    fn create(self, ctx: &Context) -> impl use<> + Any {
        self
    }
}
Error with impl use<Self> + Any
// impl Component for String {
error[E0799]: `Self` can't be captured in `use<...>` precise captures list, since it is an alias
  --> src/lib.rs:11:48
   |
10 | impl Component for &'static str {
   | ------------------------------- `Self` is not a generic argument, but an alias to the type of the implementation
11 |     fn create(self, ctx: &Context) -> impl use<Self> + Any {
   |                                                ^^^^

Example2:

impl Component for &'static str {
    fn create(self, ctx: &Context) -> impl use<> + Any {
        self
    }
}
Error with impl use<'static> + Any
// impl Component for &'static str {
error: expected lifetime parameter in `use<...>` precise captures list, found `'static`
  --> src/lib.rs:17:48
   |
17 |     fn create(self, ctx: &Context) -> impl use<'static> + Any {
   |                                                ^^^^^^^

That part could be considered a shortcoming, I think. At least, I'm pretty sure the compiler could soundly draw some conclusions due to the 'static restraints and not capture the lifetime in the definition of the RPITIT.[1]

As was pointed out we don't have precise capturing in traits yet, and that would solve this problem. But IMO your code should compile now anyway, even with the lifetime capture.[2] That is to say, even with the lifetime capture, there's a bug or shortcoming here.

Let me drill down some more to explain what I mean and where the error is coming from.


Because the RPITIT captures the implementing type and lifetime, it notionally acts like this:

// Conceptually how the RPIT is defined.  `Self` can be considered
// an input to any given implementation (along with the lifetime `'a`)
trait ComponentRpit<'a> {
    type Create: Any;
}

pub trait Component: 'static + for<'a> ComponentRpit<'a> {
    fn create(self, ctx: &Context) -> <Self as ComponentRpit<'_>>::Create;
}

With one significant difference being that the RPITIT is opaque -- it can never normalize, even if the implementor is known.[3]

In a generic context like component (where the implementor is not known), the above pattern can't normalize either, so the closure here

pub fn component<T: Component>(prop: T) {
    move |ctx: &Context| prop.create(ctx);
}

has an intended[4] signature like so

for<'a> fn(&'a Context) -> <T as ComponentRpit<'a>>::Create

and in this form we can see the "lifetime in the return type".[5]

But it still seems like this should work. The error now looks like this:

error: lifetime may not live long enough
  --> src/lib.rs:16:26
   |
16 |     move |ctx: &Context| prop.create(ctx);
   |                -       - ^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
   |                |       |
   |                |       return type of closure is <T as ComponentRpit<'2>>::Create
   |                let's call the lifetime of this reference `'1`

Ah, with less elision, this is looking more familiar. It's Rust's notoriously horrible inference for "pass through a borrow" closures. When we're able to name things, we can fix this up:

pub fn component_with_funnel<T: Component>(prop: T) {
    funnel::<T, _>(move |ctx: &Context| prop.create(ctx));
}
fn funnel<T: Component, F>(f: F) -> F
where
    F: FnOnce(&Context) -> <T as ComponentRpit<'_>>::Create,
{ f }
Side note on the downside and unifying types

Here's the downside compared to just not capturing the lifetime: if the OP worked like I sketched out above, with the "intended" closure signature, it would mean the Context stays borrowed as long as the return value is around, which you probably don't want. The signature you really want is something like

for<'a> fn(&'a Context) -> OneNonLifetimeAnnotatedType

So even if your OP compiled like it should, you may have found it to be annoying to use.

You can work around "too much capturing" with the ComponentRpit<'_> pattern by unifying all the associated types with another type parameter. In that case the compiler realizes there's no way for the lifetime to be "captured". (component_with_unified_type in the playground.) That's what you really want (which is analogous to not capturing the lifetime).

But there's no syntax for anything like that with RPIT. Even with return type notation, I'm not sure if it will be possible. You would need the ability to write a type equality bound against the return type. (I haven't kept up-to-date on the feature enough to know if that's a planned possibility or not.)


Unfortunately with the opaque RPITIT, we can't name things, and you're in the same conundrum as a lot of async related closures and bounds -- no way to write the funnel (even if you were okay with the downsides).

Those are getting some async-specific language additions to help soon, but since this isn't async, you won't benefit. (I haven't had time to look into that feature and don't know why there isn't a non-async counterpart.)

The typical workaround in async today is to type erase everything. That allows you to name things again. In this case, the lifetime disappears altogether with type erasure, so you don't even need to deal with the borrowing downsides or need a funnel anymore:

pub fn component<T>(prop: T)
where
    T: Component,
{
    move |ctx: &Context| Box::new(prop.create(ctx)) as Box<dyn Any>;
}

(Playground.)

This lets you keep the opaque return type in the trait... but not in the closure.


I think there are number of developments that could help this use case and I don't know which are coming first (or at all).

  • Precise capturing in traits
  • Associated type impl Trait
  • Return type notation (could probably emulate associated type impl Trait or maybe unify types to emulate precise capturing)
  • Better inference or annotations or AsyncFn like improvements for non-async closures
  • impl Trait in bounds ... could enable funnels for unnameable types
  • RPIT for closures (with precise capturing?) ... could emulate annotations

  1. That was part of the plan before we got precise capturing as I understand it. ↩︎

  2. Albeit with some downsides that capture avoidance does not have. ↩︎

  3. And another being that RPITIT supports unnamed types, which this pattern can't do until we can use impl Trait in associated types ("ATIT", I think; the soup of acronyms is awful) ↩︎

  4. eh, sort of, see side-note further down ↩︎

  5. I don't think the type system has a good way to treat this lifetime parameter in some special way relative to other lifetimes. ↩︎

3 Likes

Thanks for reply! I got your workaround, but my project need to stay in stable rust, your workaround probably not available for it.

Thanks for your detailed explanation, it takes some time for me to get into this, now I think I got it. Please correct me if there's something wrong with my understanding:

That is to say,

  1. fn create(self, ctx: &Context) -> impl Any in rustc's perspective it was for<'a> fn(self, ctx: &'a Context) -> <Self as ComponentRpit<'a>>::Create. The latter part is generated automatically and anonymous.
  2. Although impl Any always captures no lifetimes no matter what exact type it is, the exact type of impl Any changes as lifetime of &Context change, so the final impl Any type is dependent on the lifetime of &Context.
  3. Because we're unable to name an opaque type, so unable to constrain for<'a> <Self as ComponentRpit<'a>>::Create to be only one type, so we have to bring the lifetime with impl Any wherever it goes, that is what you called "downside".
  4. Although the return type captures lifetime, it can work in most scenarios. But thanks to the... drawback of closure lifetime inferring, we cannot bring that impl Any out of the closure because it dependents input lifetime.
  5. Eventually we got that error.

The solution you gave, and maybe the only one I thought, is to eliminate the type. We even get no means to eliminate the type then cast it back.

I think it's pretty much right. When I mentioned "downside" I was mainly referring to the fact that the return type of the closure would borrow the input, even if the closure inference worked correctly.

The unwanted capturing of the lifetime in the first place is also a drawback, which will eventually be solved by precise capturing (or associated type impl Trait).

Hmm... you can sort-of achieve this. Although now the closure is returning the erased type of the create_static method, from a type perspective (hence "sort of"). Or perhaps... like so, and now the return type in terms of my "conceptual" example is ComponentRpit<'static>. (We know they're all the same, but the type system doesn't.)

I don't think there's a way to emulate precise capturing with this on stable though.


I have a half-written follow-up that has mostly turned into an exploration of return type notation (RTN) on my part. I'll post the short version here and might post the rest later (or might not, it's half dead-ends and half frustrated griping).

@vague already provided examples of how this fixes the OP.

Playground. Random thought: this pattern is also basically refinement where implementors can choose to expose their associated type or not (especially if paired with a default associated type of impl Any). Oh but I bet... yeah we have that already, nevermind.

RTN is disappointing in this area. It doesn't support type equality, being used outside of where clauses (like in type position), or referring to the higher-ranked inputs. So far as I've been able to tell, it's a dead-end for the OP; at least, a dead-end without the other features (which solve it better directly).

I have been having a question about this compiled error

error: lifetime may not live long enough
  --> src/lib.rs:16:26
   |
16 |     move |ctx: &Context| prop.create(ctx);
   |                -       - ^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
   |                |       |
   |                |       return type of closure is <T as ComponentRpit<'2>>::Create
   |                let's call the lifetime of this reference `'1`

As shown in the above signature for<'a> fn(&'a Context) -> <T as ComponentRpit<'a>>::Create The lifetime in the parameter is associated with the one in the returned type, why does the compiler still introduce another lifetime 'l for the parameter?

The extra introduced lifetime for a parameter is not only in this example but also exists in a simpler example when using a closure. For example

fn test<F>(f:F)
where F: for<'b> FnOnce(&'b i32){}


fn main(){
    test(|s:&'static i32|{});
}

The compiler will report an error

 --> src/main.rs:11:11
   |
11 |     test(|s:&'static i32|{});
   |           ^ help: if this is intentional, prefix it with an underscore: `_s`

error: lifetime may not live long enough
  --> src/main.rs:11:11
   |
11 |     test(|s:&'static i32|{});
   |           ^ - let's call the lifetime of this reference `'1`
   |           |
   |           requires that `'1` must outlive `'static`

The lifetime of s has been explicitly specified as 'static, why does the compiler instead introduce another lifetime 'l for that parameter?

I tried to search the keyword "inference for "pass through a borrow" closures", however, I didn't find the relevant discussion.

We only wish that was the signature. The way the compiler actually decides how to handle lifetimes, outside of a context with a direct Fn-trait bound,[1] is something like:[2]

  • If the input isn't annotated, infer a single lifetime
  • If the input is annotated with an elided lifetime, be higher-ranked over that lifetime
  • Always infer a single lifetime for the output (fn-like elision rules do not apply)

Examples.

The first two are annoying and the last one is crazy frustrating... especially with unnameable types, since the funnel hack cannot be applied.

That looks like a bad diagnostic to me. The real problem is that test requires being able to take any lifetime, whereas your closure can only take the 'static lifetime.


  1. funnel in my example above ↩︎

  2. there is no spec to cite and this may not be completely accurate ↩︎

This way is pretty good, and I made some optimizations. This should be good now. :grinning:

1 Like

Nice :slight_smile:

I'm confused by this example

let closure = |s: &str| { let s: & str = s; };

As said in the comment: «If the input is annotated with an elided lifetime, be higher-ranked over that lifetime»

The parameter s has the type &'l str where 'l denotes a higher-ranked lifetime, and the let binding introduces another s whose type is &'s str where 's represents a specific lifetime that is alive in the closure's body, why can &'l str be coerced to &'s str? I didn't find the corresponding rule in the Rust Reference.

's can be the same as 'l. But it could also be shorter than the closure body, so I'll just come back to that in a moment.

"be higher-ranked" really means, "implement the Fn traits for all lifetimes". Consider this function with an analogous signature (ignoring the captures):

//                   vvvv a higher-ranked -- i.e. generic -- lifetime
fn trim_and_print(s: &str) {
    //  vvv also has some lifetime
    let sub = s.trim();
    println!("{sub}");
}

The language would be a lot less useful if it patterns like this didn't work!

But why does it work? Generics limit what you can do with a value, because you can only do things that every type which meets the generic's bounds could do. In this case, you know it's a &'? str with some lifetime longer than the function body, but you don't know the exact lifetime.

You can make use of any trait it implements that doesn't require a specific lifetime. That includes traits such as Copy and std::fmt::Display. You can also coerce it to a &str with a shorter lifetime due to variance -- &str with shorter lifetimes, such as shorter than the function body, are supertypes of the input type.

However, there's no need for sub to have a lifetime shorter than the function body. It can have the same lifetime as the argument. Remember, Rust lifetimes are about the duration of borrows, and not about the liveness scope of values. Otherwise you could never put a &'static str into a local variable, for example.

Being able to exercise what you know about a generic type is somewhat fundamental, but I guess this is the closest thing in the reference. There is also this section for supertype related assignment,[1] but again, that's not actually needed in the example.


  1. and here for coercions ↩︎

For my above question, I think I conflated the higher-ranked lifetime in the implementation and the lifetime in the specific instance. When we say the lifetime is higher-ranked, we mean that it denotes all lifetimes in implementing some traits.

impl<'h> Fn(&'h str) for AnonymousLambda{
   // extern "rust-call" fn call(&self, args: (&'h str)) -> (){...}
}

The lifetime 'h in the Fn(&'h str) is the so-called higher-ranked lifetime.

However, in the lambda expression |s: &'h str| { let s: & str = s; }; or AnonymousLambda::call, 'h is instantiated to some specific lifetime corresponding to the actual argument, so coercing from &'h str to &'s str obeys the common subtype relation. Is my understanding right?

You're right, my phrasing was pretty loose there.

When the closure is called -- so from the perspective of the closure body -- it has been instantiated to some specific lifetime, right.

Ok, so it implies that the lifetime 'h is still a higher-ranked lifetime in the function's signature.