Problem with GATs in an async trait after updating nightly Rust

I had problems expressing a Send bound for a GAT within an async trait (using async-trait). While composing an example, to show my problem, I noticed that the Playground acts differently than my local compiler. I thus updated my compiler to rustc 1.58.0-nightly (072799443 2021-11-06). This required me to add some extra bound in the code below (where Self: 'a). But I then get another error that seems unfixable. See example below:

#![feature(generic_associated_types)]

use async_trait::async_trait;
use std::ops::Deref;

async fn some_async_task() {}

#[async_trait]
trait Source {
    type T;
    // I would like to remove the `Send` bound here:
    type Wrapper<'a>: Deref<Target = Self::T> + Send
    where
        Self: 'a;
    async fn retrieve(&mut self) -> Self::Wrapper<'_>;
}

struct S {
    state: i32,
}

impl S {
    fn new() -> Self {
        S { state: 0 }
    }
}

#[async_trait]
impl Source for S {
    type T = i32;
    type Wrapper<'a> = &'a Self::T;
    async fn retrieve(&mut self) -> Self::Wrapper<'_> {
        self.state += 1;
        &self.state
    }
}

#[async_trait]
trait User {
    // But after updating nightly rustc, this won't work anymore:
    async fn process<'a, 'b, S>(
        &'a self,
        source: &'b mut S,
    ) -> S::Wrapper<'b>
    where
        S: Source + Send,
    {
        let result = source.retrieve().await;
        some_async_task().await;
        result
    }
}

struct U {}

#[async_trait]
impl User for U {}

#[tokio::main]
async fn main() {
    let mut s = S::new();
    let u = U {};
    let value: &i32 = u.process(&mut s).await;
    println!("Result = {}", value);
}

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
error[E0311]: the parameter type `S` may not live long enough
  --> src/main.rs:47:5
   |
41 |       async fn process<'a, 'b, S>(
   |                                - help: consider adding an explicit lifetime bound...: `S: 'c`
...
47 | /     {
48 | |         let result = source.retrieve().await;
49 | |         some_async_task().await;
50 | |         result
51 | |     }
   | |_____^ ...so that the type `S` will meet its required lifetime bounds

error: could not compile `playground` due to previous error

If I follow the compiler's advice literally, then of course I'll get:

error[E0261]: use of undeclared lifetime name `'c`

If I declare 'c as a lifetime parameter, then the compiler demands I should add a bound S: 'd, and so on. No idea what this is about and whether it's a problem of nightly Rust or of async-trait.

You (the macro) might be hitting this new lint which is the outcome of this issue. I'm not sure of that, just aware of the linked issues, but perhaps they will lead you somewhere useful.

2 Likes

Not sure if it helps me solving the particular issue, but at least I understand now why I suddenly have a dozen compiler errors in my code. :sweat_smile:

I tried the macro expansion:

#![feature(prelude_import)]
#![feature(generic_associated_types)]
#[prelude_import]
use std::prelude::rust_2021::*;
#[macro_use]
extern crate std;

use async_trait::async_trait;
use std::ops::Deref;

async fn some_async_task() { }

trait Source {
    type T;
    // I would like to remove the `Send` bound here:
    type Wrapper<'a>: Deref<Target = Self::T> + Send where Self: 'a;
    #[must_use]
    #[allow(clippy :: type_complexity, clippy :: type_repetition_in_bounds)]
    fn retrieve<'life0, 'async_trait>(&'life0 mut self)
    ->
        ::core::pin::Pin<Box<dyn ::core::future::Future<Output =
                                                        Self::Wrapper<'_>> +
                             ::core::marker::Send + 'async_trait>>
    where
    'life0: 'async_trait,
    Self: 'async_trait;
}

struct S {
    state: i32,
}

impl S {
    fn new() -> Self { S{state: 0,} }
}

impl Source for S {
    type T = i32;
    type Wrapper<'a> = &'a Self::T;
    #[allow(clippy :: let_unit_value, clippy :: type_complexity, clippy ::
            type_repetition_in_bounds, clippy :: used_underscore_binding)]
    fn retrieve<'life0, 'async_trait>(&'life0 mut self)
     ->
         ::core::pin::Pin<Box<dyn ::core::future::Future<Output =
                                                         Self::Wrapper<'_>> +
                              ::core::marker::Send + 'async_trait>> where
     'life0: 'async_trait, Self: 'async_trait {
        Box::pin(async move
                     {
                         if let ::core::option::Option::Some(__ret) =
                                ::core::option::Option::None::<Self::Wrapper<'_>>
                            {
                             return __ret;
                         }
                         let mut __self = self;
                         let __ret: Self::Wrapper<'_> =
                             { __self.state += 1; &__self.state };

                         #[allow(unreachable_code)]
                         __ret
                     })
    }
}

trait User {
    // But after updating nightly rustc, this won't work anymore:
    #[must_use]
    #[allow(clippy :: let_unit_value, clippy :: type_complexity, clippy ::
            type_repetition_in_bounds, clippy :: used_underscore_binding)]
    fn process<'a, 'b, 'async_trait, S>(&'a self, source: &'b mut S)
     ->
         ::core::pin::Pin<Box<dyn ::core::future::Future<Output =
                                                         S::Wrapper<'b>> +
                              ::core::marker::Send + 'async_trait>> where
     S: Source + Send, 'a: 'async_trait, 'b: 'async_trait, S: 'async_trait,
     Self: ::core::marker::Sync + 'async_trait {
        Box::pin(async move
                     {
                         if let ::core::option::Option::Some(__ret) =
                                ::core::option::Option::None::<S::Wrapper<'b>>
                            {
                             return __ret;
                         }
                         let __self = self;
                         let source = source;
                         let __ret: S::Wrapper<'b> =
                             {
                                 let result = source.retrieve().await;
                                 some_async_task().await;
                                 result
                             };

                         #[allow(unreachable_code)]
                         __ret
                     })
    }
}

struct U {
}

impl User for U { }

I don't see any lifetime 'c. The compiler thus makes up a non-existing lifetime (and maybe expects me to define it myself somehow?

I tried requiring S: 'static. That works! (Maybe it can be relaxed more, but it might be okay to require a static type in my case.) (solved first problem in Playground)


Now I tried what I actually wanted to do: remove the Send bound on:

type Wrapper<'a>: Deref<Target = Self::T> + Send
where
    Self: 'a;

And adding an HRTB here:

async fn process<'a, 'b, S>(
    &'a self,
    source: &'b mut S,
) -> S::Wrapper<'b>
where
    S: Source + Send + 'static,
    // But this doesn't work:
    for<'z> <S as Source>::Wrapper<'z>: Send,
{ /* … */ }

(Playground with HRTB)

I get the following error:

   Compiling playground v0.0.1 (/playground)
error[E0277]: `<_ as Source>::Wrapper<'z>` cannot be sent between threads safely
  --> src/main.rs:64:25
   |
64 |     let value: &i32 = u.process(&mut s).await;
   |                         ^^^^^^^ `<_ as Source>::Wrapper<'z>` cannot be sent between threads safely
   |
   = help: the trait `for<'z> Send` is not implemented for `<_ as Source>::Wrapper<'z>`

For more information about this error, try `rustc --explain E0277`.
error: could not compile `playground` due to previous error

Any clue what's wrong here? I'm not sure if I understand the error message. Commenting out the last two lines of main will make the code compile at least…

I don't understand the error message (the one about that lifetime 'c) either. Feel free to open an issue on Github. (Minimizing the example a bit from the full macro-expansion code might be a good idea.)

(Haven't read the second part of you last post yet.)

Edit: Regarding second part with Send, I'm not even sure why <S as Source>::Wrapper<'b>: Send, shouldn't be enough in the first place. Wouldn't result simply be of type <S as Source>::Wrapper<'b>? When not questioning the necessity of the HRTB, I'm not too surprised to see the compiler having a problem with properly applying a HRTB for a GAT, I feel like I've seen problems like that before.

Definitely give feedback on Github regardless, highlighting how these cases worked before and are now errors, and how the errors didn't really make sense. They're depending on experience reports to tune how GATs will work going forward.

Yeah, the only Wrapper created is 'b, :face_with_raised_eyebrow: so I also don't see why it needs to be Send for every lifetime. When I don't use a HRTB, but just demand <S as Source>::Wrapper<'b>, I get:

   Compiling playground v0.0.1 (/playground)
error: implementation of `Send` is not general enough
  --> src/main.rs:48:5
   |
48 | /     {
49 | |         let result = source.retrieve().await;
50 | |         some_async_task().await;
51 | |         result
52 | |     }
   | |_____^ implementation of `Send` is not general enough
   |
   = note: `<S as Source>::Wrapper<'0>` must implement `Send`, for any lifetime `'0`...
   = note: ...but `Send` is actually implemented for the type `<S as Source>::Wrapper<'b>`

error: could not compile `playground` due to previous error

If you remember where, let me know…

I will try to boil it down further. Not sure if it will happen without the macro-expansion, but maybe it does. I think I somehow need to replace async-trait's demand for Send by creating an async block with explicit type annotation or something like that (have done that for other examples I presented here).

Where exactly should I give that feedback best? The original issue is closed. I'm not very familiar with the exact proceedings, so if you can give me a hint where it's best to post it, I'd be thankful.

As for PR #89970, the direct impact wasn't too bad. PR #89970 did not really break anything. The problem with getting rid of the Send bound in the associated type (Source::Wrapper<'_> in my example) by adding a bound to the method (User::process in my example) that uses that associated type wasn't possible before the update either, but error messages were different, I think. If I remember right, I got pages of errors with long lists of… I don't remember what it was, but it was worse :sweat_smile:.

However, PR #89970 resulted in the bloaty syntax of my code to de-sugar async trait methods to get bloated a bit more :grinning_face_with_smiling_eyes::

pub trait Binary: Sized {
    type Dump<'a, 'b, W>: Future<Output = io::Result<()>> + Send
    where
        Self: 'a,
        W: 'b;
    fn dump<'a, 'b, W>(&'a self, writer: &'b mut W) -> Self::Dump<'a, 'b, W>
    where
        W: AsyncWrite + Unpin + Send;

    type Restore<'a, R>: Future<Output = io::Result<Self>> + Send
    where
        R: 'a;
    fn restore<'a, R>(reader: &'a mut R) -> Self::Restore<'a, R>
    where
        R: AsyncRead + Unpin + Send;
}

Luckily I can use a shorter notation without where at least for implementation of that Binary trait:

impl Binary for SomeStruct {
    type Dump<'a, 'b, W: 'b> = impl Future<Output = io::Result<()>> + Send;
    fn dump<'a, 'b, W>(&'a self, writer: &'b mut W) -> Self::Dump<'a, 'b, W>
    where
        W: AsyncWrite + Unpin + Send,
    {
        async move {
            /* … */
        }
    }
    type Restore<'a, R: 'a> = impl Future<Output = io::Result<Self>> + Send;
    fn restore<'a, R>(reader: &'a mut R) -> Self::Restore<'a, R>
    where
        R: AsyncRead + Unpin + Send,
    {
        async move {
            /* … */
        }
    }
}

It was ugly beforehand and still is ugly, so not a big issue. What bugs me most is the extra indentation due to the async move, which is unrelated to PR #89970 and was necessary before.

Side note: If I get it right, such (ugly) desugaring might be necessary for a while when async trait methods shall guarantee to return a Sendable future. See also limitations/workarounds in the MVP of the async fn fundamentals initiative. But I'm happy that there is progress in getting support for async trait methods!

I was mostly referring to the first issue you resolved with the 'static bound. I'm not happy with that static bound and not happy with the suggested non-existant lifetime 'c. The Send bounds should not matter for that issue.

The HRTB-and-Send stuff is probably issue worthy, too.

In general, try to get rid of all dependencies when minimizing. It's also okay not to minimize anything, other people will generally help out providing minimized versions for you when you just open an issue describing the problem with the original code example.

I don't see a specific open issue either, so I guess you could ask where in #44265 to make sure the right people see it. Or you could...

  • Open a new issue
  • Tag it F-generic_associated_types
  • Mention the suspicion of 87479
  • CC 44265

(As is probably obvious, I don't know where the best place to give feedback is either.)

In my mind the problems here are not really a #87479 thing including the first issue; at least they are not an argument against the Self: 'a bound, since in my opinion the code in OPs first Playground should just compile fine1. It doesn't– which is a separate bug independent of #87479. Of course it doesn't hurt mentioning the connection in the discovery process.

1 The Self: 'a bound means that <S as Source>::Wrapper<'b> should just require S: 'b which is already an implied bound by the &'b mut S argument to User::process.

1 Like

I haven't had time to delve into things and could certainly believe it's just a timing coincidence (but also wanted to assure feedback gets communicated as we get closer to stabilization / making decisions we'll all live with indefinitely).

What do you mean by "timing coincidence"? As far as I understand things, #87479 forces the code of OP to add the Self: 'a bound which runs into the problem producing the faulty error[E0311]: the parameter type `S` may not live long enough message, however that's the message you got in that code with the Self: 'a bound anyways, even before #87479 was implemented. There's a casual connection between #87479 and OP running into the problem - no coincidence - still the actual buggy behavior is an independent issue. I'd also say that it makes sense to add the Self: 'a bound to the code here; it just unfortunately doesn't play nicely with limitations/bugs in the compiler.

Also, as far as I understand, the current implementation stage of just conservatively requiring explicit Self: 'a doesn't yet commit in either direction on having the Self: 'a be an implicit default or not.

I just noticed that both issues only appear when holding the variable result: S::Wrapper<'b> over an .await point. If this triggers the compiler seeing some need for a HRTB Send bound S::Wrapper<'c>: Send for all lifetimes 'c, perhaps by the same token it somehow thinks that this requires S::Wrapper<'c> to be a valid type for all lifetimes 'c. (Which it isn’t; it’s only valid for those lifetimes that outlive 'b.)

So the problem appears to – perhaps – be some problem of the interaction of async blocks/fns with GATs.

Edit: The plot thickens... the lifetime problem is – somehow – related to the Send bound !?

Here's a small example

#![feature(generic_associated_types)]
use std::{future::Future, marker::PhantomData};

trait Trait {
    type Associated<'a>: Send
    where
        Self: 'a;
}

fn future<'a, S: Trait + 'a, F>(f: F) -> F
where
    F: Future<Output = ()> + Send,
{
    f
}

fn foo<'a, S: Trait + 'a>() {
    future::<'a, S, _>(async move {
        let result: PhantomData<S::Associated<'a>> = PhantomData;
        async {}.await;
    });
}

(playground)

1 Like

When I replace Send with SomeProperty that is implemented for any T, I don't get an error (Playground). But demanding Sync will give an error.

I'm a bit confused now whether we talk about a single problem or two problems. Maybe it's not specifically related to Send but to an issue between GATs and auto-traits?

Sure it's related to auto-traits. After all, the Send bound on the future depends on the Send-ness of all the local variables held over .await points that are stored in the future precisely due to Send being an auto-trait.

So both issues are a case of weird behavior of auto-trait bounds for local variables in async blocks/fns, but IMO they still seem like they’re probably separate issues.

For the record, the example works (i.e. fails) with an async fn, too:

#![feature(generic_associated_types)]
use std::{future::Future, marker::PhantomData};

trait Trait {
    type Associated<'a>: Send
    where
        Self: 'a;
}

fn future<'a, S: Trait + 'a, F>(f: F) -> F
where
    F: Future<Output = ()> + Send,
{
    f
}

async fn f<'a, S: Trait + 'a>() {
    let result: PhantomData<S::Associated<'a>> = PhantomData;
    async {}.await;
} 

fn foo<'a, S: Trait + 'a>() {
    future::<'a, S, _>(f::<'a, S>());
}

Note that when I use #[async_trait(?Send)] (see async-trait doc on non-threadsafe futures), the 'static bound isn't required. (Playground)

Right, I’ve noticed as much myself, too, by now, hence the

1 Like

As far as I understand:

  • The two problems seem to be related to each other, at least.
  • There was already a problem before the recent change in Rust nightly (at least for me), which made it impossible for me to express the Send bound in the method where I use the GAT (but instead requiring me to make the GAT Send in the trait where it was defined, as a workaround).
  • The most recent change in Rust made things worse when I use that workaround (demanding the Send bound in the GAT definition), as I now get weird errors demanding I should add another lifetime bound ('c, which didn't exist), ultimately forcing me to add a 'static bound (or remove Send from the GATs definition).
  • The issue might not be specific to Send, but to any auto-trait in that matter.

(Related as in both are problems with bounds on GATs. The two issues still might have independent roots, and maybe only one of them is related to auto-traits while the other isn't.)


@steffahn Maybe it's best to submit your minimal example(s) to the GAT tracking issue? Though it might also be specific to async blocks perhaps? Can you reproduce the problem without async? (Sorry to ask, I feel not experienced enough to provide good minimal examples…)

I think this is async-specific. I also think that the minimal example is best used for demonstration in a newly opened issue. It's definitely relevant to the GAT tracking issue as it's a GAT problem (that probably deserves solving before GATs are stabilized). I don't know what the right procedure is to ask whether this really is a "blocking bug" for GATs (i.e. should be added to the list in the tracking issue), you can definitely ask on the newly opened issue itself, perhaps you could also ask in that tracking issue.

I’m not too opinionated on this point anymore; maybe putting everything together into a single new issue gives a more coherent picture – it doesn’t seem unlikely that both problems can be fixed together in the same way.