What's the semantic difference between `use<'a>` and `+'a` in the returned opaque type place

Consider this example:

fn show<'a>(i:&'a i32)->impl Future + 'a { 
   async {}
}

fn show2<'a>(I:&'a i32)->impl Future + use<'a>{
   async {}
}

In show, saying the returned concrete type is T, the syntax +'a means T:'a. However, I still cannot completely understand what use<'a> exactly means in show2. IIUC, the first effect is to explicitly specify that T captures the lifetime 'a as if it behaved as T<'a>. Under this concept, T<'a>:'a still holds. So, what's the difference between use<'a> and +'a for another purpose?

the detailed explanation can be found in rfc 3617, but in short words, + 'a is an outlives bound, while + use<'a> is a precise capture specifier.

the outlives bound on an opaque return type is similar to lifetime bounds in generic type parameters.

the precise capture simply means the opaque return type has generic parameters. note, it is not limited to lifetimes, it can also include types, e.g. impl Trait + use<'a, T>.

With + 'a you're saying that the returned type will live at least as long as 'a, but not less. Crucially this will require you to also ensure that any other generic parameter used in the opaque type also meets that bound (which can be quite restricting/annoying).

With + use<'a> you're saying that whatever the opaque type is, it might contain/use the 'a lifetime, and hence behaves like any type containing that lifetime in that it lives at most as long as 'a, but it could live less than 'a, for example if it contained another generic parameter that lives less than 'a.

Ultimately there's not much difference when you only use a single generic, but once you start adding more you'll generally have to use use

So, is the difference like between T:'a(corresponding to +'a) and T<'a>(corresponding to +use<'a>)?

Ok, IIUC, the syntax +use<'a> only means the opaque type is specified only to capture 'a as if it were T<'a>, right?

Pretty much.

Yep! Which in turn matters when you have (ultimately) multiple lifetime parameters.

:information_source:
Notably, if we have 'a : 'i, and 'b : 'i, then Type<'a, 'b> : 'i, and vice versa.

This, in turn, applies to this -> impl …, with, say, the following signature:

fn f<'a, 'b, 'i>(a: &'a (), b: &'b ()) -> impl …
where
    'a : 'i,
    'b : 'i,
// thus: `…<'a, 'b> : 'i`
{
    (a, b)
}

Here we have:

                 : 'i
type Ret<'a, 'b> = (&'a (), &'b ()) = impl …
   + use<'a, 'b> + 'i;

That is, we have both:

  • type Ret<'a, 'b> = impl use<'a, 'b> + …

  • type Ret<'a, 'b> = impl 'i + …

Thence, intuitively :

The important intuition connecting use<'lts, '..> with + 'usability

:information_source:
impl use<'a, 'b> ≈ impl 'i (when 'a : 'i, 'b : 'i)

With that having been said, or that "nerd"/intellectual curiosity having explored, let's take a step back, and focus on:

use<'lts, '..> vs. + 'usability in practice

:information_source:
This whole discussion is about their usage in fn return type:

  • -> impl … + use<…>
  • -> impl … + 'usability

As of current Rust, there is no other place where + use<…> can be used, making this whole comparison pointless or irrelevant then.

The main points are the following:

  • Try not to use both + use<…> and + 'usability.

    Indeed, you're probably going to end up overconstraining your return type without major benefit.

    • Caveat: if you do use + 'usability, the compiler may still require you to provide + use<…> in certain cases[1] For Reasons™ (that we'll talk about later).
  • :information_source:
    When using -> impl …, favor + use<…> over + 'usability

    To get an intuition as to why, going back to our f() example above, notice how using + 'usability required us to introduce an extra, otherwise unused, lifetime parameter, and to "artificially" constrain it to be "smaller than"[2] every other lifetime (or type) parameter present. Indeed, (&'a (), &'b ()) is neither : 'a nor : 'b (when 'a and 'b are both arbitrary, and thus, potentially unrelated / with no ordering property between them).

    So using + 'usability may sometimes require "thinking of" / "coming up with" this "trick", which is not as easy as merely listing every generic parameter used in the return type (a rather mechanical and intuitive operation), which is what using + use<…> is about.

  • :information_source:
    When the package.edition is ≥"2024", eliding use<…> is equivalent to a hidden, automagically inserted by the compiler, + use<'every, 'single, Param, In, Scope>.
    So unless you're doing something a bit specific, favor using neither use<…> nor + 'usability: things will Just Work™ this way.

  • :warning:
    + use<…> cannot be used with -> PtrTo<dyn …>. So you have to use + 'usability in that case.

    And this is, as a matter of fact, the main reason I've been writing this post: whilst it would have been great being able to say "-> … + 'usability is an artifact from the past, to be superseded by the more intuitive -> … + use<…>", it turns out not to be true, because of → PtrTo<dyn … + 'usability>, which "brings us back" to pre-use<…> Rust questions.

Why the need for these use<…> vs. + 'usability annotations to begin with?

This is, in general, the "whole point", or at least, the "main point" of this annotation to begin with:

  1. the return type is some type-erased or opaque type whose details we do not wish to expose;
  2. but there is an important "detail" that we cannot overlook nor hide, no matter how much it may displease us: the "effective region/lifetime of safe usability of (instances of) our type"

Indeed, consider:

fn foo(s: &str) -> Box<…> { Box::new("some static str") }

fn bar(s: &str) -> Box<…> { Box::new(s) }

Given the fn bodies, we know the following call-site to be safe and sound:

let ret = {
    let local = String::from("local");
    foo(&local)
};
stuff(ret);

But the following one should very much be rejected!

let ret = {
    let local = String::from("local");
    bar(&local) // return type still borrows from `local`
}; // <- `local` is dropped here
stuff(ret); // <- Uh-oh…

Of course, if we see the fn bodies, or rather, if the Rust compiler/borrow-checker sees the fn bodies, it should be able to reject this for us, no matter what we write for in our -> Box<…> return type, right? We may just want to mention we're returning "some Display" type (granted, inside a Box), whatever that may be.

  • using dyn(amic dispatch and unifying-type-erasure): -> Box<dyn Display>,
  • using impl ("opaque" type): -> Box<impl Display>[3].

Well, papering over the technical feasability challenges of such an approach regarding an all-seeing compiler, if this were to work, it would mean that changing the fn body of, say, foo(), to match that of bar(), without any syntactical change to its function signature, would make our example call-site cease to compile. That is, a change of fn body/implementation with no syntactical change of the fn signature would be a breaking change :scream:

This would be catastrophic for the ecosystem, and does, in fact, go against the very ethos of Rust's non-SemVer-footgunny API design philosophy.

So we want:

  • a function such as foo to be definable, and in a way that allows our example callsite to compile;
  • a function such as bar to be definable, and in a way that prevents our example callsite from compiling;
  • some syntactical marker in the fn signature to signify, within the API contract, whether our fn body implementation is more like bar(), i.e., borrowing from the input (or rather potentially borrowing from the input), or more like foo(), i.e., not borrowing from the input (guaranteed not to).

We could envision several possible ways to make this annotation (we could have imagined, initially, some #[borrowing(true)] and #[borrowing(false)] lang-magical annotation or whatnot, but eventually, as the number of borrowing args increases, we'd need to say which args we may be borrowing from, and as a matter of fact, which parts of which args we may be "borrowing". Eventually, we need to be talking of lifetimes, so yes, we do end up needing some lifetime annotations. Or, back to our original "type-erasure/opaquification" goal, generally, we cannot really erase or be opaque over lifetime usage[4].

Now, the reason for the two use<…> + 'usability approaches for this very annotation comes rather naturally if we rewrite our situation using a trait API:

fn f(s: &str) -> Box<… '_ …>;
// is sugar for:
fn f<'s>(s: &'s str) -> Box<… 's …>;

So, with a trait, we could have:

trait OurFn : 'static {
    fn call<'s>(s: &'s str) -> Box<Self::Ret<'s>>;
// where:
    type Ret<'s> : Display; // Option A
// or:
    type Ret<'s> : 'static + Display; // Option B
}

// or:
trait OurFn : 'static {
    fn call<'s>(s: &'s str) -> Box<Self::Ret>;
    type Ret : Display;
// Note: `type Ret :` can also be spelled out as:
//  type Ret<> : Display; // Option C
//          ^
//        no 's!
}

alongside our callsite:

fn callsite<F: OurFn>() {
    let ret = {
        let local = String::from("local");
        F::call(&local)
    }; // <- `drop(local);`
    println!("{ret}");
}

It turns out these three "Option"s I've listed are valid ways to describe this "family" of APIs, and that if we do so, our:

  • callsite() fails to compile in the Option A case,
    which is the least restrictive for the implementor, and thus, the less powerful / most restrictive for the user/caller/callsite;

  • callsite() does compile in the Option B case,
    which does have a : 'static restriction on Ret<'s>; thus, a : 'static property which the user/caller/callsite can make use of (for their function to compile);

  • callsite() does compile in the Option C case,
    which does have a non-<'s>-using restriction on type Ret; thus, a "non-'s-infected property" which the user/caller/callsite can make use of (for their function to compile).

If you stare at the three type Ret … definitions, and you sed/strip the type Ret "common prefix" on all them (and the : and ;), and instead, write -> impl, you end up with:

// Option A: type Ret<'s>: Display;
-> impl<'s>: Display;
// Option B: type Ret<'s>: 'static + Display;
-> impl<'s>: 'static + Display;
// Option C: type Ret<>: Display;
-> impl<>: Display;

If we replace <…>: with use<…> +, we end up with:

-> impl /* may */use<'s> + Display
-> impl /* may */use<'s> + 'static + Display
-> impl /* may */use<> + Display

Which is exactly the three kinds of annotations we can use for our -> impl … types!

But how do these differ in semantics with one another? Which ones are more restrictive for the callee/implementation/fn body, and which for the caller/callsite?

The exact meaning/difference in semantics between use<..> and + 'usability

  • IOW, using use<…> vs. + 'usability as restrictions (for the callee), or properties (for the caller).

For this, we need to focus on use<'lt> vs. + 'usability. Or, equivalently, given some unknown/arbitrary or we daresay adversarial choice of F: OurFn, what can we say about:

  • F::Ret<'lt, ...> vs.
  • F::Ret<...> : 'usability ?

Well, remember the golden rule:

:information_source:

Notably, if we have 'a : 'i, and 'b : 'i, then Type<'a, 'b> : 'i, and vice versa.

Which we can generalize:

:information_source:
Type<'a, 'b, ...> : 'i'a : 'i, 'b : 'i, ...

  • 'a : 'i, 'b : 'i, ... can be written as:
    'a ⊇ 'i, 'b ⊇ 'i, ... i.e.,
    ('a ∩ 'b ∩ ...) ⊇ 'i

There is another important property, related to this, at least as far as intuition is concerned:

:information_source:
If T : 'u, then[5] (any instance of type) T is guaranteed not "to dangle", i.e., not to contain any dangling/expired references anywhere inside it.
It is thus safe to use (any instance of type) T for the "duration"/region/lifetime 'u (notably, "for the lifetime 'static" means "forever").
We could even say that (any instance of type) T is safely usable within/for 'u.
Hence my calling this "the lifetime of usability of T"; I even strive[6] to name a lifetime in this position 'usability.

From all this stems an important, and in our case, meaningful, corollary:

Type<'a, ...> : 'u'a : 'u ('a ⊇ 'u)

"Thus", use<'a, ...> : 'u'a : 'u... Kind of. It's a decent intuition, but the devil here is in the details.

There is a difference between Type<'a, ...> when it is a concrete type (e.g., &'a str, Box<&'a str>, Box<dyn 'a + Display>, Cow<'a, str>, BorrowedFd<'a>), when the 'a : 'usability entailment does hold, and when you have some "abstract", or somehow overly generic, or outright opaque type that happens to "mention 'a".

  • Note: dyn is deemed a "concrete" type in this regard: it "definitely uses 'a". It is impl which is "opaque": it only may use 'a.

For instance, let's focus on some F: OurFn + 'static (notice how F itself, in this example, is not allowed to "know" of any lifetime other than 'static[7]): what could F::Ret<'s> possibly be? As in, for every possible choice of F, or, equivalently, for a masterfully clever adversary providing some F with the purpose to annoy us.

Well, they could write, notably, the two following impls, and provide us either of the implementors:

enum A {}
impl OurFn for A {
    type Ret<'s> = &'s str;
    ...
}

enum B {}
impl OurFn for B {
    type Ret<'s> = &'static str; // Allowed!
    ...
}

That is, with a GAT (as well as with a trait's own generic parameters), an associated type is not forced to be infected with / mentioning each and every one of these parameters in scope: we are allowed to "ignore" 's!

So we have both:

  • A::Ret<'s> = &'s str :! 'static,
  • B::Ret<'s> = &'static str : 'static,

i.e.,

  • F::Ret<'s> : ?'static | ?use<'s>, in pseudo-code.

    :bulb:
    I write this as /* may */use<'s, ...>

So, which is it? Well, whilst the implementor is not forced to use 's, a user/caller/callsite of such an abstract / generic / "adversarially chosen" / opaque type cannot assume this to be the case. Quite the contrary! :information_source: Barring any extra information, they must assume the worst: that F::Ret<'s> may start dangling once 's expires :information_source:

:information_source:
impl /* may */use<'a, ...> + NoOtherLifetimeInfo : 'u'a : 'u ('a ⊇ 'u)

So back to our original, practical question. Given:

fn a<'s>(s: &'s str) -> impl /* may */use<'s> + Display;
fn b<'s>(s: &'s str) -> impl /* may */use<'s> + 'static + Display;
fn c<'s>(s: &'s str) -> impl /* may */use<> + Display;
fn d<'s>(s: &'s str) -> impl /* may */use<> + 'static + Display;

which return types can be used for 'static (no matter how small 's be)?

To illustrate, if we had f be some function ranging over a, b, c, d, the question is whether the following snippet would compile:

// a, b, c, or d.
use ... as f;

fn callsite() -> impl 'static + Display {
    let local = String::from("...");
    let s: &'_ str = &local;
    /* return */ f(s)
}
  1. Well, d() obviously can; as a matter of fact, I'll skip mentioning this one in the future, since there is no subtlety whatsoever about this one.

  2. As it turns out, so can c(): whilst it does not strictly stipulate that it be : 'static, it does stipulate that the exhaustive list of generic parameters it /* may */use is empy. With no non-'static info at its disposal, it has no other choice than to be 'static.

  3. We have seen case a() in our previous :information_source::

    :information_source:

    impl /* may */use<'a, ...> + NoOtherLifetimeInfo : 'u'a : 'u

    Our a() function is indeed a case of impl /* may */use<'s> + NoOtherLifetimeInfo. Thus, in order for it to be : 'static, we'd need for 's to be : 'static, which we're assuming not to hold.

    Thus, a() will not be callable/usable in our callsite().

  4. Remains b(): here, we have something which may be using 's, but which also stipulates it is : 'static. The latter "takes precedence", and our resulting type is indeed : 'static

Hence:

:information_source:

-> impl use<'s> + Bounds...;           // ❌ not `: 'static`, just `: 's`
-> impl use<'s> + Bounds... + 'static; // ✅ `: 'static`
-> impl use<>   + Bounds...;           // ✅ `: 'static`
  • (Unless Bounds... in and of itself entailed : 'static (such as with Any), of course; otherwise the first and second ones would boil down to being equivalent)

  • Tangent about impl 's + 'static + ...

    Whilst it cannot be written explicitly, if we were able to write impl 's + 'static, then the resulting type would have lifetimes that dangle neither within 's nor within 'static, and since the latter entails the former, it is all equivalent to lifetimes which do not dangle "just" within 'static:

    impl 's + 'static + Bounds... = impl use<'s> + 'static + Bounds...

More generally, going back to the use<...>-with-2-or-more-lifetimes case (when use<> is truly useful and/or meaningful vs. using bare + 'lifetime), and naming this 'intersection lifetime of 'x and 'y (so impl use<'x, 'y>impl 'i), we have:

//! Given `fn<'x, 'y, 'i>(...) where 'x : 'i, 'y : 'i`

-> impl use<'x, 'y> + Bounds...; // `: 'i`

-> impl use<'x, /*no 'y*/> + Bounds...; // `: 'x`

// Reminder:
-> impl use<'x, 'y> + Bounds... + 'x; // `: 'x`. And so on and so forth.

Practical example: 3-1 lifetimes being use<..>d

This can get trickier (and use<...> can get especially handier) when you get to 3 or more lifetimes, and some of which are not use<..>d (and you intend to let the caller/callsite take advantage of that fact):

fn f_impl<'a, 'b, 'c, /* 'i */>(
    a: &'a str,
   _b: &'b str,
    c: &'c str,
) -> Box<impl use<'a, /*no 'b,*/ 'c> + Debug>
where
    // ... : 'i, ...
{
    Box::new((a, c)) // : Box<(&'a str, &'c str)>
}

fn callsite() -> impl 'static + Debug {
    let local = String::from("b");
    /* return */ f_impl("a", &local, "c") // OK because `'b` not `use<..>`d.
}

This last snippet compiles fine, courtesy of impl use<...> managing to:

  • intuitively describe the fact that b is not being used in the return type,
  • so the caller/callsite can take advantage of that fact for a snippet such as fn callsite() to be allowed.

But what about when use<..> is not available? Notably, when returning dyn, rather than impl?

fn f_dyn<'a, 'b, 'c, /* 'i */>(
    a: &'a str,
   _b: &'b str,
    c: &'c str,
) -> Box<dyn use<'a, /*no 'b,*/ 'c> + '_ + Debug>
where
    // ... : 'i, ...
{
    Box::new((a, c)) // : Box<(&'a str, &'c str)>
}

fn callsite() -> impl 'static + Debug {
    let local = String::from("b");
    /* return */ f_dyn("a", &local, "c") // OK because `'b` not `use<..>`d.
}

Exercice Playground

Well, le me hint you that the answer, which as you may have guessed, will involve:

  1. :light_bulb: adding some kind of 'intersection lifetime, i.e., some <'i>-introduced lifetime,

  2. :information_source: Using this extra 'i lifetime as the one lifetime annotation inside the returned dyn, in the form of -> Ptr<dyn 'i + ...>

  3. And finally, adding the approprisate where ... : 'i kind of bounds. You can either try to come up with the right bounds straight away, or "fail on purpose", e.g., not writing down any bounds, and then letting the compiler suggestions lead you towards the solution :slightly_smiling_face:.

    For instance, the following snippet yields the following diagnostics:

    Click to show the intermediary step
    fn f_dyn<'a, 'b, 'c, 'i>(
        a: &'a str,
       _b: &'b str,
        c: &'c str,
    ) -> Box<dyn 'i + Debug>
    where
        // ... : 'i, ...
    {
        Box::new((a, c)) // : Box<(&'a str, &'c str)>
    }
    

    yielding:

    error: lifetime may not live long enough
      --> src/lib.rs:11:5
       |
     3 | fn f_dyn<'a, 'b, 'c, 'i>(
       |          --          -- lifetime `'i` defined here
       |          |
       |          lifetime `'a` defined here
    ...
    11 |     Box::new((a, c)) // : Box<(&'a str, &'c str)>
       |     ^^^^^^^^^^^^^^^^ function was supposed to return data with lifetime `'i` but it is returning data with lifetime `'a`
       |
       = help: consider adding the following bound: `'a: 'i`
    
    error: lifetime may not live long enough
      --> src/lib.rs:11:5
       |
     3 | fn f_dyn<'a, 'b, 'c, 'i>(
       |                  --  -- lifetime `'i` defined here
       |                  |
       |                  lifetime `'c` defined here
    ...
    11 |     Box::new((a, c)) // : Box<(&'a str, &'c str)>
       |     ^^^^^^^^^^^^^^^^ function was supposed to return data with lifetime `'i` but it is returning data with lifetime `'c`
       |
       = help: consider adding the following bound: `'c: 'i`
    
    help: the following changes may resolve your lifetime errors
      |
      = help: add bound `'a: 'i`
      = help: add bound `'c: 'i`
    

Hence the answer:

Spoiler: click here to reveal the answer!
fn f_dyn<'a, 'b, 'c, 'i>(
    a: &'a str,
   _b: &'b str,
    c: &'c str,
) -> Box<dyn 'i + Debug>
where
    'a : 'i,
 // 'b : 'i not written!
    'c : 'i,
{
    Box::new((a, c)) // : Box<(&'a str, &'c str)>
}

fn callsite() -> impl 'static + Debug {
    let local = String::from("b");
    /* return */ f_dyn("a", &local, "c") // OK because `'b` not bounded by `'i`.
}

Going further: quirks of use<...> and how variance interacts with it

TODO


  1. (when the lifetimes in are not all used covariantly) ↩︎

  2. by "smaller than" I mean when : is interpreted as meaning ; e.g., any lifetime/region 'x verifies that the never-ending / "infinite" lifetime 'static is "greater" than 'x:
    'static ⊇ 'x, and we know that 'static : 'x,
    hence this choice of interpretation.
    I am pointing this out because there is also the "set of constraints" interpretation (wherein 'static would represent the empty set of constraints, so 'static would be the smallest and not the biggest lifetime; this is what Polonius uses, for instance), as well as the "sublifetiming" type-system-relation (wherein 'static <: 'x, so as to be able to say that &'x () is covariant over 'x). All these are matters of perspective; it does not matter so long as one is consistent with the terminology.
    I, myself, prefer to view 'static as the "biggest" lifetime, and my mentioned 'i, as (some subset of) the intersection of 'a and 'b. Using the opposite point of view, 'i would rather be (bigger than) the union/max of 'a and 'b. ↩︎

  3. Granted, exposing the Box alongside an "I do not care about implementation details" usage of impl Bounds… is kind of an oxymoron: a real-life usage of -> … impl Bounds… would probably hide the Box inside the impl … opaque type as well. But for the sake of the -> dyn-vs.--> impl comparison I am making, exposing the Box even in the impl case will help make them look more similar, so their inherent differences more easily stand out. ↩︎

  4. Actually, we can hide some lifetime information! What we cannot hide is some set of "expiry points" beyond which our return type is not to be used. But a single lifetime can describe this restriction! So, as a matter of fact, we can erase lifetimes, but at most down to a single lifetime: their intersection. That last lifetime, for the most general of the APIs, cannot be erased. Do note that in doing so, some other APIs may suffer, as describing a set of lifetimes does contain more information than their mere intersection. So when you erase a set of lifetimes down to their intersection, much like with other aspects of type erasure, it does mean that some type/lifetime information has been lost, and that the resulting API/usability of the so-returned (instances of that) type may be more restricted/restrictive than if we had specified the full, real, non-erased-nor-opaque, type. ↩︎

  5. Alas, the converse/inverse is not true. Consider fn(&'s str) (for some 's): it won't be : 'static (at most, it will be : 's), and yet this function pointer, in and of itself, does not contain any dangling references/pointers through which UAF could ever be caused. ↩︎

  6. given the previous :information_source:, (the biggest possible choice for) this lifetime happens to be the intersection of all the lifetime params ultimately appearing in the type, so I sometimes also name it 'inter or just 'i. ↩︎

  7. if it "knew" of some 'a, it would require that this 'a be somewhere in its own generics, and as you know, Rust will reject this unless you somehow use the lifetime in some field, so you'd end up with F = ...<'a> : 'static ↩︎