Failing to implement Higher-Rank Trait Bounds in certain circumstance [Minimal reproducible examples]

Very new to Rust and I am struggling to understand why a generic function has a type mismatch. I'd really appreciate any help with the following snippets.

This code works fine:

use std::future::Future;

#[derive(Clone, Debug)]
struct MyStruct {
    data: i32,
}

async fn async_function1(data: &MyStruct) -> i32 {
    // Your asynchronous logic here
    data.data * 2
}

async fn async_function2(data: &MyStruct) -> i32 {
    // Your asynchronous logic here
    data.data * 3
}

async fn my_generic_async_function<'a, F, T>(func: F, data: &'a MyStruct) -> i32
where
    F: Fn(&'a MyStruct) -> T,
    T: Future<Output = i32>,
{
    func(&data).await
}

#[tokio::main]
async fn main() {
    let data = MyStruct { data: 42 };

    let result1 = my_generic_async_function(async_function1, &data).await;
    let result2 = my_generic_async_function(async_function2, &data).await;

    println!("Result 1: {}", result1);
    println!("Result 2: {}", result2);
}

But I'm curious why this variant does not.
I am only changing my_generic_async_funtion to take ownership of the data to which is passed. I believe this means that I have to use Higher Rank Trait Bounds to manage the lifetimes of the references the async functions require.

use std::future::Future;

#[derive(Clone, Debug)]
struct MyStruct {
    data: i32,
}

async fn async_function1(data: &MyStruct) -> i32 {
    // Your asynchronous logic here
    data.data * 2
}

async fn async_function2(data: &MyStruct) -> i32 {
    // Your asynchronous logic here
    data.data * 3
}

async fn my_generic_async_function<F, T>(func: F, data: MyStruct) -> i32
where
    F: for<'a> Fn(&'a MyStruct) -> T,
    T: Future<Output = i32>,
{
    func(&data).await
}

#[tokio::main]
async fn main() {
    let data = MyStruct { data: 42 };

    let result1 = my_generic_async_function(async_function1, data.clone()).await;
    let result2 = my_generic_async_function(async_function2, data).await;

    println!("Result 1: {}", result1);
    println!("Result 2: {}", result2);
}

Unfortunately, I'm failing to understand the compiler error, and can't find a solution:

error[E0308]: mismatched types
  --> src/main.rs:30:19
   |
30 |     let result1 = my_generic_async_function(async_function1, data.clone()).await;
   |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
   |
   = note: expected opaque type `impl for<'a> Future<Output = i32>`
              found opaque type `impl Future<Output = i32>`
   = help: consider `await`ing on both `Future`s
   = note: distinct uses of `impl Trait` result in different opaque types
note: the lifetime requirement is introduced here
  --> src/main.rs:20:36
   |
20 |     F: for<'a> Fn(&'a MyStruct) -> T,
   |                                    ^

error[E0308]: mismatched types
  --> src/main.rs:31:19
   |
31 |     let result2 = my_generic_async_function(async_function2, data).await;
   |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
   |
   = note: expected opaque type `impl for<'a> Future<Output = i32>`
              found opaque type `impl Future<Output = i32>`
   = help: consider `await`ing on both `Future`s
   = note: distinct uses of `impl Trait` result in different opaque types
note: the lifetime requirement is introduced here
  --> src/main.rs:20:36
   |
20 |     F: for<'a> Fn(&'a MyStruct) -> T,
   |                                    ^

These are just minimal reproductions that extrapolate from some other code. I still feel like my understanding of Rust is failing and I want to know why the second option doesn't work, and what would need to change to have it work while remaining generic.

If nightly Rust are available for you, async fn in traits is just stablized there.

trait Async {
    async fn async_function(data: &MyStruct) -> i32;
}
async fn my_generic_async_function<F: Async>(data: MyStruct) -> i32 {
    F::async_function(&data).await
}

Rust Playground


One of the stable solution is to make a lifetime contract between the input and output for async functions via a trait Rust Playground

trait AsyncCallback<'a, T>: FnMut(&'a MyStruct) -> Self::Fut {
    type Fut: Future<Output = T>;
}
impl<'a, T, Out, F> AsyncCallback<'a, T> for F
where
    Out: Future<Output = T>,
    F: FnMut(&'a MyStruct) -> Out,
{
    type Fut = Out;
}

async fn my_generic_async_function<F>(mut func: F, data: MyStruct) -> i32
where
    F: for<'a> AsyncCallback<'a, i32>,
{
    func(&data).await
}

the downside is that it won't work for closures.

To support closures returning a Future, BoxFuture is the way to go. E.g. Lifetime bounds to use for Future that isn't supposed to outlive calling scope

1 Like

Thank you for your time and providing the examples!
Can move on and keep learning now!

I'm a little confused by your examples. The future returned by async_function1 has a lifetime, i.e. it would desugar to

fn async_function1<'a>(data: &'a MyStruct) -> impl Future<Output=i32> + 'a

It needs to dereference data when it is polled. Where is that reflected in any of the constraints below?

Trait bounds on my_generic_async_function didn't convey that, so the error says about it

F: for<'a> Fn(&'a MyStruct) -> BoxFuture<'a, T> is the common way to express it.

But interestingly the trick in my answer is that Rust can see the introduced lifetime requirement here without annotation hint

actually should mean

trait AsyncCallback<'a, T>: FnMut(&'a MyStruct) -> Self::Fut {
    type Fut: 'a /* implicit lifetime requirement here! */ + Future<Output = T>;
}
impl<'a, T, Out, F> AsyncCallback<'a, T> for F
where
    Out: 'a /* implicit lifetime requirement here! */ + Future<Output = T>,
    F: FnMut(&'a MyStruct) -> Out,
{
    type Fut = Out;
}
2 Likes

Is that a limitation of the Rust language?

I think so. At least extended HRTB syntax like for<'a, Out: 'a> is desired by some people, but not sure whether it's worth it.

async fn my_generic_async_function<F, T>(func: F, data: MyStruct) -> i32
where
    F: for<'a> Fn(&'a MyStruct) -> T,
    T: Future<Output = i32>,

Context: T can't capture 'a in the snippet.

It's because type parameters like T must resolve to a single type, but types that differ even only by lifetime are distinct types. So there's no way T could be capturing the lifetime 'a; the output has to be the same for every lifetime input lifetime 'a. You need something like a generic type constructor or a way to put a bound on the output without unifying it with a generic type parameter, and Rust doesn't have those directly today. Instead we work around it.


That's not quite the same thing because T: 'a is more restrictive than "T capured 'a". But it's often sufficient.


The first snippet working is just a reflection that Out can capture 'a just fine, because 'a is a single lifetime and not part of a higher ranked binder. It's not an implicit bound, it's more that it's possible for FutureBlob<'a> to be the resolved type of Out. The bound in the second snippet is actually more restrictive, as above.[1]


I'm not sure that would mean the same thing in any proposal I've seen. Either Out is still a single type variable from the outer scope, in which case it still can't work, or Out is being introduced in the binder, in which case you have to be able to return any type that meets the 'a bound. Maybe if it was for<'a> exists<Out> ...

Other things I've seen that could work directly include:

where // Avoid `FnMut` return-type "sugar"
    for<'a> F: FnMut<(&'a MyStruct,)>,
    for<'a> <F as FnOnce<('a MyStruct,)>::Output: Future<Output = T>,

But the compiler is still limited.

where // `impl Trait` in bounds...
    // ...probably the lifetime capture is implicit
    for<'a> F: FnMut(&'a MyStruct) -> impl Future<Output = T> /* + Captures<'a> */,

But what else it captures is potentially a thorny question.

// Generic type constructors
impl<Out<'*>, F, T> ...
where
    for<'a> F: FnMut(&'a MyStruct) -> Out<'a>,
    for<'a> Out<'a>: Future<Output = T>,

But I haven't seen a serious proposal for this, just as a hypothetical used when explaining things.

...and probably others, but I'll stop for now :slight_smile:.


  1. Not that I haven't been guilty of unnecessarily introducing such bounds myself. ↩︎

3 Likes

Thanks for clarifying that. :heart:

Thank you for the detailed explanation. It is going to take some time to digest. What does "capture" mean in this context? As opposed to a constraint of the form T: 'a or 'a: T?

Should I think of dyn Trait + 'a as a type constructor (over the variable 'a)? Vs impl Trait + 'a which means (there exists) a type T satisfying T: Trait and T: 'a?

Yes, that's a great summary really.

"Is parameterized by" is a good way to think of it. Here's a playground based on the RFC appendix D. The capturing versions don't impose an outlive bound on the entire type, and thus don't impose an outlives constraint on any other parameters (lifetime or type parameters) that happen to get captured.

In fact the notional desugaring is something like

// Unstable TAIT feature (type alias `impl Trait`)
// (n.b. no `: 'a` bound and no `T: 'a` bound)
type _AnonAsyncFnBarReturn<'a, T>: impl Future<Output = ()>;
fn bar<'x, T>(t: T, x: &'x ()) -> _AnonAsyncFnBarReturn<'x, T> {}

Note that the captured async lifetime / "parameter" appears to be invariant as far as I've been able to test, but I'm not 100% that it needs to be.[1] If it could be covariant or contravariant, that would be a leaky abstraction, but one that matches structs. I haven't played with TAIT enough to be confident it works the same, either (in part because they keep tweaking how it works so I'm uncertain that testing would tell me much about what will be stabilized).

In contrast the dyn Trait + '_ lifetime is always covariant (and then some).

Those constraints don't exist in Rust.


  1. Well it has to be with the trait Captures<'_> trick, I mean for implicitly captured lifetimes. ↩︎

3 Likes

Luckily the trick and lifetime capture rules are being documented now

There's a lot of related information here. In short, there's no requirement that a TAIT constrains its parameters, so in that way parameterized opaque types are more like a GAT or a parameterized trait with an associated type in that you can have

&'b Opaque<'a>,
// made-up syntax meaning "`'a: 'b` would fail"
!('a: 'b),
// E.g. because `Opaque<'a> = ()` say, so this could hold
Opaque<'a>: 'static,

So internally, they are being treated like projections, which don't constrain their lifetimes (and thus, AFAIK, can't convey any variance). I guess in the GAT case, everything leaks post-normalization, but there is no normalizing of an opaque type outside of the compiler.

Except for auto-traits, which seems more like an exception to me under this way of thinking about things.


Here's another issue about capturing, this time more focused on type parameters (though largely the outlives implications of captured type parameters), and this analysis goes into some of the subtleties. IIUC, unifying the "nameable captures" and "using captures" would be a breaking change for things like

fn ex<T, 'a>(t: T, a: &'a str) -> impl Future<Output = ()> {
    async move {
        // Don't actually move `t` (by not mentioning it)
        let _actually_move = a;
    }
}

By making them act like

// (how `async fn` works)
fn ex<T, 'a>(t: T, a: &'a str) -> impl Future<Output = ()> {
    async move {
        let _actually_move = (t, a);
    }
}

Anyway, now I have things to think about which will take some time to digest too :slightly_smiling_face:.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.