`-> T + ‘outlives` seems unnecessary - always

help
1

I’ve been trying to understand why the lifetime + ‘outlives in the return position impl Trait + ‘outlives annotation was ever required.

In the 2024 edition the “opaque” type captures all of the lifetimes in scope. That ensures the opaque type lifetime does not exceed that of the refs it depends on. No lifetime annotations required to capture the dependency. Got it.

In the 2021 edition: I don’t see any possible scenario where the hidden type could ever compile using a ref that wasn’t tied to the input ref lifetimes. I.e., the compiler would always ensure we return a ref that isn’t dangling. This holds without having to analyze the implementation of the trait. So why did we need this + outlives annotation?

Is this a case of “Rust prefers explicit” going wrong by logically implying the possibility of the impossible? Or am I missing a semantic understanding of the annotation?

See the RFC that discusses the topic. I wonder if the RFC would benefit from explicitly saying - “we were wrong to require the annotation because it is already part of what the compiler guarantees”.

2

it's not to that the hidden type can have a lifetime "shorter" than the input lifetime, but in 2021 edition, the default is: "capturing none lifetimes, unless for those explicitly mentioned lifetimes, e.g. the Captures trick [1]", or in other words, at return position, impl Trait essentially means impl Trait + 'static.

and it turns out this default is undesirable, so in 2024 edition, the default is flipped: "capturing all, unless there is an explicitly use<...> specifier".

  1. the "outlives" constraint is different from the Captures trick, details in the rfc ↩︎

1 Like
3

Great answer - pithy and lucid.

So in 2024 edition, if I have multiple lifetimes in the inputs, I can actively limit which lifetime the output depends on when it’s not the least (shortest) of them using + ’this_lifetime. Yes?

Finally, another source of confusion for me: I don’t get how any lifetime annotation in the return position can ever mean anything other than “cannot live longer than” some input lifetime (‘return in relation to ‘input). The use of “at least” vs “at most” is confusing to me here.

I mention this because in the RFC discussion they talk about how using + in 2021 worked because both interpretations overlap when the lifetimes are equal (which I get); I don’t get how return is ever at least as long as…

4

If all other generics are known to meet that bound, and you don't hit various limitations around how much flexibility an outlives bound can offer, that may work. But the proper fix is to be explicit about which lifetimes to capture or not capture using + use<...>. For example:

// We don't want to capture the lifetime here.  But `+ 'static` is the
// wrong "solution" as it will force `T` to be `'static`.
fn process<T>(ctx: &Context, data: Vec<T>) -> impl Iterator<Item = T> + use<T> {
    context.query(...);
    data.into_iter()
}

And similarly if you had multiple lifetimes, etc.

// In edition 2024+, the return type means the same as
// ```
//     -> impl Iterator<Item = T> + use<'c, T>
// ```
// But `T: 'c` is *not* required, which is what this would mean:
// ```
//     -> impl Iterator<Item = T> + 'c
// ```
fn process<'c, T>(ctx: &'c Context, data: Vec<T>) -> impl Iterator<Item = T> {
    data.into_iter()
        .inspect(|item| ctx.query(item))
}

Or perhaps think of it like so.

trait Ex1<'a, 'b> {
    type Opaque;
}

impl<'a, 'b> Ex1<'a, 'b> for () {
    type Opaque = (&'a str, &'b str);
}

<() as Ex1<'a, 'b>>::Opaque does not outlive 'a and does not outlive 'b. But the type depends on both 'a and 'b.

-> impl Trait captures (+ use <..>) define which generics the opaque type is allowed to depend on.

An alternative to reading the RFC is this blog post.

4 Likes
5

Thank you @quinedot It’s been a while since my last visit to the forum. The community is lucky to continue to benefit from your explanations. I’m going to take some time to consider your answer… greatly appreciated.

6

Both answers helped get me where I needed to go. The link to the blog post helped me nail down where I was confused.

In a nutshell, despite my understanding what T: 'a means in a where clause "T must live at least as long as 'a (if not longer)"

I was thinking -> impl Trait + 'a meant something different in the return position... in part because I cannot imagine a scenario where this expression makes sense in the return position (the return value's lifetime depends on how long the caller keeps the value, non-nonsensical and backwards)... I just did what the compiler told me to do when I encountered errors when I did not include the + 'a - totally not trying to figure it out because it made no sense in how I was thinking about it.

So I needed to confirm that the following syntax in fact say the same thing about T - That T must live at least as long as the lifetime 'a:

T: 'a in a where clause and
T + 'a in the return position (i.e., impl Trait + 'a)

My working hypothesis is that the second expression never makes any sense and should have never been able to compile. Certainly in version 2024 because we no longer need the "happenstance" covariance that made this error work in 2021.

Am I wrong?

7

Rust lifetimes -- those '_ things -- are not directly about how long values are kept around. It's an unfortunate overlap in terminology. For example, String: 'static, but that doesn't mean one never drops and deallocates Strings.

Rust lifetimes are a type-level property that generally corresponds to the duration of a borrow. The main connection with value liveness are conflicts between being borrowed and being moved or destructed, etc.

T: 'a means that for any lifetime 'x that is in the fully resolved type T, 'x: 'a. This usually corresponds to, "any borrows T has are valid for at least 'a."

-> impl Trait + 'a means that R: Trait + 'a, where R represents the opaque return type.

So one could read this as a guarantee to the caller than any borrows in the opaque return type are valid for at least 'a. The most common reason for a caller to care would be when they need something 'static which they can send to another (non-scoped) thread or such.

However, in reality I believe you're most likely to see + 'a in -> impl Trait to force the lifetime to be captured when it otherwise wouldn't have been.[1] On edition 2024, all lifetimes are captured by default, so that no longer happens. And even on earlier editions, you can use + use<'a> now instead (and generally that's what you want to do).

Longwinded details

There was a period of time where

  • we had -> impl Trait
  • but it didn't capture any lifetimes unless they were mentioned in the -> impl bounds[2]
  • so if you needed to capture the lifetime, you had to find a way to mention it
  • and we didn't have precise capturing (+ use<..>), which is usually what you want
  • and the easiest-to-code substitute for + use<'a> was + 'a
  • and the compiler would even recommend using + 'a
  • and most people didn't understand that + 'a had downsides or know how to code up the alternative (the "Captures trick")

It's possible people still add -> impl Trait + 'a when they don't need to due to bad habits picked up during this period, or because they think it will fix some borrow checker error (which it could do, even if use<..> would have been the better fix).

It's probably directly useful in some situations (and probably most of those involve 'static). It may let more things borrow check in some situations too. But I think having such a situation that couldn't be solved with use<> is pretty rare, which is inline with your intuition.[3]

I would agree that most existing uses of -> impl Trait + 'some_lifetime should have been impl Trait + use<'some_lifetime>, and probably got there due to the circumstances I outlined in the "longwinded details" above.

  1. I.e. the author isn't trying to guarantee anything, they just want to be able to return something borrowed. ↩︎

  2. outside of traits ↩︎

  3. I don't see any reason it should never have been accepted. It's "just" another bound the opaque has to meet. ↩︎

2 Likes
8

I agree that the -> impl Trait + 'lifetime can make sense when the lifetime is static. That is only because it is something the compiler can know without depending on the input lifetime types. So that is a modification to how I see it. Otherwise, we are saying the caller sets the lifetime type by how it uses the return value. That can't be simultaneously true when the same lifetime type is set by the inputs. That is by definition, invalid. Thus, the 2024 edition should only allow -> impl Trait + 'static. Are you convinced?

I also appreciate you bringing up how lifetimes live in the type space and how that can get muddled when we talk about how long a value exists. For now in how I model it, I see the type space as a way to set the relativity rules, and the actual lifetime as needing to comply with those rules...?

9

Function signatures define a contract that both the function body and the caller have to abide by, so I guess I don't see it as either side "setting" something. I'm not sure exactly what your mental model is either, so it's a bit challenging to discuss. I suppose I could throw a bunch of examples about how lifetimes work across function boundaries to try to get you aligned with my mental model...

But at any rate, no I am not convinced. For one, it would be a breaking change for no benefit. For two, it would introduce an inconsistency.[1] And for three, it's not invalid, it's just something I haven't seen many use cases for.

Opaques work something like this:

trait OpaqueFactory {
    // -> impl Iterator<Item = &'inner str> + outer
    type One<'outer, 'inner: 'outer, D: Display>:
        Iterator<Item = &'inner str> + 'outer;
    fn one<'outer, 'inner, D: Display>(
        _: D,
        slice: &'outer [&'inner str],
    ) -> Self::One<'outer, 'inner, D>;

    // -> impl Iterator<Item = &'inner str>
    type Two<'outer, 'inner: 'outer, D: Display>: Iterator<Item = &'inner str>;
    fn two<'outer, 'inner, D: Display>(
        d: D,
        slice: &'outer [&'inner str],
    ) -> Self::Two<'outer, 'inner, D>;

    // -> impl Iterator<Item = &'static str> + use<D>
    type Three<D: Display>: Iterator<Item = &'static str>;
    fn three<'outer, 'inner, D: Display>(
        _: D, 
        _: &'outer [&'inner str]
    ) -> Self::Three<D>;
}

The parameters on the GATs correspond to the + use<..> captures, and the bounds on the GATs correspond to the rest of the opaque (including + 'lifetime). Would you say it's invalid to require the lifetime bound on One?

I'll come back to that in a second. For now let's consider Three. It's not parameterized by lifetimes, while One and Two are. What does that tell us in a generic context? It tells us that if we call three, the &'outer [&'inner str] we pass in isn't tied to the return type -- the slice and strs don't have to remain borrowed while the returned iterator is in use. Why not? Because there's no way that the GAT Three could name those lifetimes.

Does the lifetime bound on One tell us anything? Yes, it tells us that the validity of the iterator does not depend on the validity of D -- because there is no requirement that D: 'outer. The GAT must be defined for all D: Display, even if D does not meet a 'outer bound. Similar to the situation with Three, this means that if D happens to be or contain a borrow, that borrow does not need to stay active while the returned iterator is in use. We cannot draw the same conclusion for Two (as it has no outlives bound).

We can demonstrate that the borrow checker exploits this.

fn test<O: OpaqueFactory>(s: &mut str) {
    // This compiles fine.
    let mut iter = O::one(&mut *s, &[]);
    println!("{s}");
    let __ = iter.next();

    // This results in a borrow checker error.
    let mut iter = O::two(&mut *s, &[]);
    println!("{s}");
    let __ = iter.next();
}

Or equivalently.

// Remove the `+ 'outer` to see an error.
fn same_thing_but_standalone<'inner: 'outer, 'outer, D: Display>(
    _: D,
    slice: &'outer [&'inner str],
) -> impl Iterator<Item = &'inner str> + 'outer {
    slice.into_iter().copied()
}

fn example(s: &mut str) {
    let mut iter = same_thing_but_standalone(&mut *s, &[]);
    println!("{s}");
    let __ = iter.next();
}

So that's an example of an opaque type with a non-'static lifetime bound which is valid and has semantic impact.

Note also that precise capturing cannot yet opt out of capturing generic types:

// error: `impl Trait` must mention all type parameters in scope in `use<...>`
) -> impl Iterator<Item = &'inner str> + use<'outer, 'inner> {

So that is not an available alternative for this example.

  1. The inability to assert outlives bounds in opaque types, in contrast with trait bounds. ↩︎

1 Like
10

This is an amazing presentation. I truly appreciate your patience working through this with me.

1 Like