Lifetimes in smol::Executor

help
8

The following signature works (Playground):

/// The "`Captures<'_>` hack" to work around `impl` _vs._ `dyn` inconsistency
trait Captures<'__> {}
impl<T : ?Sized> Captures<'_> for T {}

fn run_broken<'a, 'ex : 'a> (executor: &'a Executor<'ex>)
  -> impl Future + 'a + Captures<'ex>
{
    async move {
        executor.spawn(async move {
            println!("inner!");
        }).detach();
    }
}

Explanation

  1. I've removed the incorrect 'a ≥ 'ex bound: &'a Executor<'ex> implies that 'ex ≥ 'a, so if we had that extra 'a ≥ 'ex bound, we'd end up with 'a == 'ex and the classic horrible "nested borrow with same lifetime" hell. Made the 'ex ≥ 'a bound explicit, although that is not necessary (I'd even recommend that it be removed to avoid the danger of getting the bound wrong): it is just here to teach the notion better.

  2. At that point, the returned async move { … } (outer) future is capturing a type that mentions 'a and 'ex, thus its area / span / region of owned-usability (the lifetime appearing in the impl signature) is the intersection / minimum of 'a and 'ex. Since 'ex ≥ 'a, it is 'a == min('a, 'ex).

  3. That's when we hit the infamous:

    error[E0700]: hidden type for `impl Trait` captures lifetime that does not appear in bounds
  --> src/main.rs:26:61
   |
26 | fn run_broken<'a, 'ex : 'a>(executor: &'a Executor<'ex>) -> impl Future + 'a { // + Captures<'ex> {
   |                                                             ^^^^^^^^^^^^^^^^
   |
note: hidden type `impl Future` captures the lifetime `'ex` as defined on the function body at 26:19
  --> src/main.rs:26:19
    Detailed explanation

    This is basically what I personally consider to be a bug, and is, at the very least, a big inconsistency with the dyn counterpart. That is, if this were (pin-)boxed, returning a Box<dyn 'a + Future…> would work at this point. This is because once a type is (dyn-)erased, the only lifetimes that matter are those appearing on traits (here, none), and the lifetime of owned-usability. If the type initially had an internal non-'static lifetime (necessarily bigger than that of its owned-usability), then if such lifetime does not appear in the visible trait bounds of the dyn object, then it can't play any role anymore.

    Sadly, this can't be the case for impl Trait…, since the -> impl… mechanic is just a "mild" / "fake" type erasure; the compiler still sees the real type behind it and may thus interact with more lifetimes than the ones visible. So it can't erase such an inner special lifetime (here, 'ex). And indeed, with the initial designs of -> impl Trait, it didn't do so: a lifetime such as 'ex would still be present in the returned type, hidden, even if it wouldn't appear in the public API (and that is sound, as we will see with the workaround at the end of this post).

    What's the problem then? Well, Rust further developed a currently unstable / nightly feature: type_alias_impl_trait (you can actually experiment with min_type_alias_impl_trait already, the soon to be stabilized subset of it). And they decided, legitimately, to reduce special casing and increase consistency by making -> impl Trait… use a hidden / compiler-generated type_alias_impl_trait under the hood.

    And the catch is that, when using type_alias_impl_trait, one needs to explicitly list the generic (type and lifetime (and const?)) parameters that the existential type is allowed to depend on. And when they implemented this shift, they changed the logic of -> impl Trait w.r.t. captured generic parameters (for no good reason imho): the returned type will only be generic over the parameters appearing in the return type (rather than being generic over all the parameters present in scope).

    TL,DR

    That means that when writing -> impl 'a + Future…, Rust is, under the hood, writing:

    type HiddenReturnedFut<'a> = impl 'a + Future…;

    and then trying to unify:

    HiddenReturnedFut<'a> := CompilerGeneratedFuture<'a, 'ex> {
    capture_0: &'a Executor<'ex>,
}

    , which it can't do because of that 'ex lifetime parameter which may (and often will) be strictly bigger than 'a, and because Executor<'ex> is not covariant in 'ex (this is the reason why often this limitation is dodged; very often the inner lifetime is in covariant position and Rust can thus then shrink the lifetime 'ex down to 'a).

    The workaround

    Until the compiler / lang team decides to acknowledge this is a bug (it looks like the status quo is to say this is "suprisingly" Working As Intended™), and then fix it (that is, make the compiler-defined HiddenReturnedFut be generic over an 'ex parameter too), the only solution / workaround at the moment is to make the 'ex parameter explicitly appear in the returned type impl … signature, by using a helper dummy trait to be infected with that lifetime parameter: hence the dummy trait Captures<'__> {}, the dummy blanket impl (so that adding it to a signature does not really add any constraints), and then the actual addition of + Captures<'ex>.

9 Likes