Is it possible to achieve these lifetimes in this generic FnOnce -> Future?

help
5

I decided to explore this awhile[1] to see if I could I could shake anything new out of the tree... like a clearer explanation of why things don't work, if anything else.

TL;DR

A doubly-type-erased version that works

One of the challenges is that you have a return type that's needs to be valid for the maximum of the function's validity and the input's validity -- the intersection of the two -- because you're returning a future that contains both the "fields of the closure" (the &'f mut Vec<_> the closure captured) and the input &'s mut S.

Type erasure is used to work around a lot of Rust's higher-ranked shortcomings, but it's hard to write up this requirement in type-erased form, even.

// There's no syntax for `'int = intersection('f, 's)`
dyn 'f for<'s> FnOnce(&'s mut S) -> BoxFuture<'int, ()>
where // there is no support for `dyn ... where`
    'f: 'int, 's: 'int

However, you wanted this to be higher-ranked over 's so that utilizers could pass in a local borrow, whereas the closure is coming from the caller. So it's presumably sufficient to just say 'f: 's, in which case the intersection is 's.

dyn 'f for<'s where 'f: 's> FnOnce(&'s mut S) -> BoxFuture<'s, ()>

There's no for<'s where 'f: 's> either, but we can fake it by mentioning a type that requires that in order to be well-formed.

// "where 'f: 's"                                    vvvvvvvvvv
dyn 'f + for<'s> FnOnce(&'s mut S) -> BoxFuture<'s, [&'s &'f (); 0]>

That's enough for the type erased version.

Problems with a direct translation to generic bounds

You can then try to walk backwards from there to make a trait/generic version, which may look something like this.

// Where `Self: 's` (`'f: 's`)
trait MakeFutureOnce<'f, 's, Guard = &'s &'f Self>: 's + FnOnce(&'s mut S) -> Self::Fut {
    type FutOut;
    type Fut: Send + Future<Output = Self::FutOut>;
}

// Implemented for `Fn`s with our hacky lifetime guard
impl<'f, 's, F, Fut> MakeFutureOnce<'f, 's> for F
where
    F: FnOnce(&'s mut S) -> Fut,
    Fut: Send + Future<Output = [&'s &'f (); 0]>,
{
    type FutOut = [&'s &'f (); 0];
    type Fut = Fut;
}

// Higher-ranked version with implied `'f: 's` bound
trait MakeFuture<'f>: 
    for<'s> MakeFutureOnce<'f, 's, FutOut = [&'s &'f (); 0]> {}
// Implemented for everything that meets the bounds
impl<'f, M: for<'s> MakeFutureOnce<'f, 's, FutOut = [&'s &'f (); 0]>> MakeFuture<'f> for M {}

And the compiler does actually accept these bounds, somewhat. For example, this works:

async fn make_future<'f, F: MakeFuture<'f>>(f: F) {
// ...

    let bx: BoxMakeFuture<'_> = Box::new( same as before );
    make_future(bx);

So our type-erased version meets these bounds.

But then you get a new problem. Off-hand it looks like Rust's poor higher-ranked closure inference strikes again, and that our F: MakeFuture<'f> bound gets in the way of the usual "funnel it through a function" workaround. That's at least partially true, but not the whole story.

When we made the translation from type-erased form to trait bounds, we actually lost a property that's important to "conditional higher-ranked bounds" -- for<'s where 'f: 's> emulated by an implied bound. Our "lifetime guard" [&'s &'f (); 0] got moved from part of the type into only associated types of traits.

If the lifetime relationship (&'s &'f) isn't part of the implementing type and it's also not an input to the trait (e.g. a trait parameter), it's not going to be an implied bound of the implementation, which will stymie higher-ranked bounds elsewhere (like in our MakeFuture<'f> supertrait bound).

I suspect these bounds not working for non-type-erased types is required for soundness.

Related issues:

Moving the lifetime guard to be a trait input

One way to fix this for the closure trait would be to make [&'s &'f (); 0] an argument of the closure, although that requires passing some dummy [] every time we call it.

But as it turns out, we can make some progress by just adding it to our trait. And we can also add a subtrait of Future that hoists its associated type into a trait parameter, too.[2]

trait ShinyFuture<Out>: Send + Future<Output = Out> {}
impl<F: ?Sized + Send + Future<Output = Out>, Out>
    ShinyFuture<Out> for F {}

//                           vvvvvv newly hoisted from associated type
trait MakeFutureOnce<'f, 's, FutOut, Guard = &'s &'f Self>
where
    Self: 's + FnOnce(&'s mut S) -> Self::Fut
{
    type Fut: ShinyFuture<FutOut>;
}

impl<'f, 's, F, Fut> MakeFutureOnce<'f, 's, [&'s &'f (); 0]> for F
where
    F: FnOnce(&'s mut S) -> Fut,
    Fut: ShinyFuture<[&'s &'f (); 0]>,

This can let us get rid of one layer of type erasing (the closure). Note that we still needed a funnel to influence the closure inference:

fn funnel<'f, F>(f: F) -> F
where
    F: for<'s> FnOnce(&'s mut S) -> 
        Pin<Box<dyn 's + ShinyFuture<[&'s &'f (); 0]>>>,

If you try a bound at a higher level, like just F: MakerFuture<'f>, it's not enough. I had to drill down to the Fn trait level with the bound.

I'm currently of the opinion that this would be a solution for having no type erasing, if there was a way to properly override the higher-ranked capturing of the closure.[3] But as far as I know, there isn't. The next section looks at an attempt which falls short.

N.b. since we didn't get rid of the future type erasure, we didn't actually need the ShinyFuture trait to hoist the associated type into a type parameter.[4]

Failed attempt to not name the future

One of the reasons our funnel used Pin<Box<dyn ShinyFuture<..>>> is that the Fn trait sugar forces you to name the output type, even though it's an associated type. And it's not a single type since it captures a lifetime under the binder, so a generic parameter won't work. But compiler-generated futures don't have names, and there's no stable alternative yet either.[5]

However, you don't have to use the sugar on nightly. Perhaps that's a way forward?

First, note again that we couldn't influence the higher-ranked nature of the closure until we drilled our funnel bounds down to the Fn traits exactly. But if we do that on nightly starting from our last playground, we end up with something like...

fn funnel<'f, F>(f: F) -> F
where
    for<'s> F: FnOnce<(&'s mut S,)>,
    for<'s> <F as FnOnce<(&'s mut S,)>>::Output: ShinyFuture<[&'s &'f (); 0]>,

Note how in the first bound, we've lost the "lifetime guard" [&'s &'f (); 0]. That means that the first bound has to be met when 's is 'static, which is not possible for our use case.

We can attempt to fix this by making [&'s &'f (); 0] an input parameter like we mentioned before...

fn funnel<'f, F>(f: F) -> F
where
    for<'s> F: FnOnce<(&'s mut S, [&'s &'f (); 0])>,
    for<'s> <F as FnOnce<(&'s mut S, [&'s &'f (); 0])>>::Output: 
        ShinyFuture<[&'s &'f (); 0]>,

But this doesn't influence closure inference like we wish it did, either.

And I think I know why. Just because because the output type of a function implements a trait that has an input that mentions 's doesn't mean that it captures 's. It's perfectly reasonable that String: From<&'s str> for example, even though String: 'static. (Adding 's under the binder doesn't improve things and adds a "due to current limitations" error.)

Whereas when we were mentioning a concrete type, if that type had 's as a parameter, it definitely captured 's and thus the closure had to be passing its higher-ranked input into the output in some sense.

So trait bounds on the associated type aren't enough, and this is probably a dead-end. It also means that inline associated type bounds aren't likely to do anything either.

// We don't have these either but it probably doesn't matter
// for this use case anyway
where
    for<'s> F: FnOnce<
        (&'s mut S, /* optionally: [&'s &'f (); 0] */),
        Output: ShinyFuture<[&'s &'f (); 0]>,
    >,

What we really need is probably something like

fn funnel<'f, F>(f: F) -> F
where
    for<'s> F: FnOnce(&'s mut S, [&'s &'f  (); 0])
        -> impl use<'s, 'f> ShinyFuture<[&'s &'f (); 0]>

Or even better

fn funnel<'f, F>(f: F) -> F
where
    for<'s where 'f: 's> F: FnOnce(&'s mut S)
        -> impl use<'s> Future<Out = ()> + Send

...assuming they work like we wish they would work, anyway.

Parting thoughts

That's as far as I got this time. As far as I know it's still not possible without type erasure (or unsafe). (But I'm also not familiar with the various helper traits that exist in the ecosystem, so it's possible I missed something.)

Unlike the Fn traits, you can implement Future on stable. But if you had some LocalFuture<F: Future>, you'd still have to name F in the funnel, so that doesn't necessarily gain you much.

TAIT might be another way forward as it gives -> impl Trait bounds indirectly / gives a way to name otherwise unnameable things. I made a few attempts, but they didn't work. I didn't give it as much effort as it perhaps deserves, though (ran out of steam).[6]

  1. for the umpteenth time ↩︎

  2. spoilers, this ended up not helping, but anyway ↩︎

  3. without naming the output type ↩︎

  4. But I'm too lazy to remove it. ↩︎

  5. generic type constructor, capturing -> impl Trait in bounds, etc. ↩︎

  6. Note that the "due to current limitations in the borrow checker" error occurs here too. For all I know it's internally equivalent to the failed associated type bound. ↩︎

5 Likes