Trait is not general enough for Send-able future

I'm having some trouble understanding lifetimes as they relate to futures. I've read the other posts on here I could find related to the "not general enough" error, and while I vaguely understand them, I don't understand well enough to apply them to my situation. I'm also relatively new to async programming in Rust.

I've got a struct that's holding onto a Box<tokio::sync::Mutex<sqlx::Transaction>>, and I'd like to use that transaction in the context of a generic function that takes any connection that implements sqlx::Acquire. I'd like to call that generic function from a method on an async_trait decorated trait. However, I get an error: implementation of sqlx::Acquire is not general enough.

Through getting this down to a minimal reproducible example and removing use of async_trait, I've figured out that the error comes from the Send requirement that async_trait adds to the bounds of the return type.

I am able to implement my trait using a struct that just holds a sqlx::PgPool, where I'm passing a &PgPool into the method, so it seems to be somehow related to the lifetime of the Transaction itself, but even if I set this lifetime to static, I get the same error.

However, I am struggling to understand why that's the case, so if anyone could help me understand the following, I'd really appreciate it.

  • Why does the Send bound lead to trait is not general enough?
  • Is there any way to do what I'm trying to do?
  • I could remove the Send bound, but is that a bad idea? Generally, none of the implementations of the trait are spawning new threads or futures, so it seems like removing the bound might be okay. What would I be losing by removing the bound?

My minimal-ish reproduction is below. Unfortunately I haven't yet been able to figure out how to reproduce with a custom trait, so the reproduction has sqlx as a dependency.

// so we've got some function that uses something that implements Acquire (impl
// syntax vs. normal generic syntax doesn't make any difference)
async fn uses_txn<'a>(cxn: impl sqlx::Acquire<'a, Database = Postgres>) {
    let mut cxn = cxn.acquire().await.unwrap();
    let _exec = cxn.deref_mut();
}

// A non-async trait version, should be basically the same as what async_trait
// expands to
trait UseTxnSend {
    fn use_txn<'a>(&'a self) -> Pin<Box<dyn core::future::Future<Output = ()> + Send + 'a>>;
}

// A non-async trait version, without the Send requirement
trait UseTxnNoSend {
    fn use_txn<'a>(&'a self) -> Pin<Box<dyn core::future::Future<Output = ()> + 'a>>;
}

// We've got a thing that holds an owned transaction in a mutex (put it in a box
// so that it's Send).
struct TxnHolder<'a> {
    txn: Box<tokio::sync::Mutex<sqlx::Transaction<'a, Postgres>>>,
}
// regualr impl
impl<'a> TxnHolder<'a> {
    // An async method that uses the txn and calls the function.
    // No compiler error here!
    async fn use_txn(&self) -> () {
        let txn = &mut *self.txn.lock().await;
        uses_txn(txn).await
    }
}
impl<'a> UseTxnNoSend for TxnHolder<'a> {
    // Implementing without the Send requirement has no errors.
    fn use_txn<'b>(&'b self) -> Pin<Box<dyn std::future::Future<Output = ()> + 'b>> {
        Box::pin(async move {
            let txn = &mut *self.txn.lock().await;
            uses_txn(txn).await
        })
    }
}
impl<'a> UseTxnSend for TxnHolder<'a> {
    // The send requirement leads to the `Acquire is not general enough` error
    fn use_txn<'b>(&'b self) -> Pin<Box<dyn std::future::Future<Output = ()> + Send + 'b>> {
        Box::pin(async move {
            let txn = &mut *self.txn.lock().await;
            uses_txn(txn).await
        })
    }
}

for reference, the signatures of Acquire and its associated type trait Database are reproduced below (from here):

pub trait Acquire<'c> {
    type Database: Database;

    type Connection: Deref<Target = <Self::Database as Database>::Connection> + DerefMut + Send;

    fn acquire(self) -> BoxFuture<'c, Result<Self::Connection, Error>>;

    fn begin(self) -> BoxFuture<'c, Result<Transaction<'c, Self::Database>, Error>>;
}

impl<'a, DB: Database> Acquire<'a> for &'_ Pool<DB> {
    type Database = DB;

    type Connection = PoolConnection<DB>;

    fn acquire(self) -> BoxFuture<'static, Result<Self::Connection, Error>> {
        Box::pin(self.acquire())
    }

    fn begin(self) -> BoxFuture<'static, Result<Transaction<'a, DB>, Error>> {
        let conn = self.acquire();

        Box::pin(async move {
            Transaction::begin(MaybePoolConnection::PoolConnection(conn.await?)).await
        })
    }
}

I've been banging on this for three days with very little progress, so thanks in advance for any help you can give!

1 Like

So, I'd have to look into your code in more detail, but I remember having had to deal with what I'd qualify as a "diagnostics bug" which looked eerily similar to what you are experiencing at the moment: in my case, there were inner async fns, somewhere deep inside the call chain, which happened to yield non-Send Futures, which is the thing ultimately breaking the caller code with the + Send requirement (such as in your code). The diagnostics bug then lies in Rust complaining about a higher-order trait bound not being met rather than the simple Send requirement failing further down the line.

I recommend that you try to use #![deny(clippy::future_not_send)] across your codebase, and maybe even on some of the dependencies, such as sqlx (e.g., through running cargo vendor and/or a Cargo patch).


I'd recommend that you try to be as Send as possible, since even though you might not care about it, downstream users are very likely to sooner or later be using extra threads with the Futures, and they will be very sad if they suddenly can't because of that missing Send in a middle layer. For instance, notice how BoxFuture and .boxed() (and #[async_trait]) are all tools which feature that bound to begin with, and how one has to opt out of it through LocalBoxFuture, .boxed_local() or #[async_trait(?Send)]. This is, by the way, also the reason why I recommend that the BoxFuture alias be used, instead of Pin<Box<dyn Future…>>, since with the latter it is easy to end up forgetting the Send at some point.

1 Like

Since a working Playground is key for any "research", here is a minimal Playground which reproduces the issue:

I'll work on trimming it down until I can't reproduce it.

EDIT 1: the traits impls have indeed nothing to do, if we take the inherent impl and rewrite it to assert that it yields a Send future we get that error back:


impl<'a> TxnHolder<'a> {
    // An async method that uses the txn and calls the function.
    // No compiler error here!
    fn use_txn(&self) -> impl '_ + Future<Output = ()> + Send {
        async move {
            let txn = &mut *self.txn.lock().await;
            uses_txn(txn).await
        }
    }
}

EDIT 2: if we start asserting that everything in that body is Send (I personally use let _: &dyn Send = &thing; for that), we end up with:

impl<'a> TxnHolder<'a> {
    // An async method that uses the txn and calls the function.
    // No compiler error here!
    fn use_txn(&self) -> impl '_ + Future<Output = ()> + Send {
        async move {
            // let txn = &mut *self.txn.lock().await;
            // uses_txn(txn).await
            let lock = self.txn.lock();
            let _: &dyn Send = &lock;
            let mut txn = lock.await;
            let _: &dyn Send = &txn;
            let txn = &mut *txn;
            let _: &dyn Send = &txn;
            let fut = uses_txn(txn);
            let _: &dyn Send = &fut;
            let ret = fut.await;
            let _: &dyn Send = &ret;
            ret
        }
    }
}

and this yields an error which points to the let _: &dyn Send = &fut; line:

error: implementation of `sqlx::Acquire` is not general enough
   --> src/lib.rs:131:26
    |
131 |     fn use_txn(&self) -> impl '_ + Future<Output = ()> + Send {
    |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ implementation of `sqlx::Acquire` is not general enough
    |
    = note: `sqlx::Acquire<'1>` would have to be implemented for the type `&'0 mut Transaction<'_, Postgres>`, for any two lifetimes `'0` and `'1`...
    = note: ...but `sqlx::Acquire<'2>` is actually implemented for the type `&'2 mut Transaction<'_, Postgres>`, for some specific lifetime `'2`

error: implementation of `sqlx::Acquire` is not general enough
   --> src/lib.rs:142:32
    |
142 |             let _: &dyn Send = &fut;
    |                                ^^^^ implementation of `sqlx::Acquire` is not general enough
    |
    = note: `&mut Transaction<'a, Postgres>` must implement `sqlx::Acquire<'0>`, for any lifetime `'0`...
    = note: ...but `sqlx::Acquire<'_>` is actually implemented for the type `&mut Transaction<'_, Postgres>`
2 Likes

Oh awesome, great job on the reproduction! I'll copy that over and play with that also, since it'll be easier than messing around with code that's tied into the rest of the codebase.

1 Like

Ok, I've managed to make it compile. When back-propagating the Send bounds (note that you may need to use fix_hidden_lifetime_bug - Rust when doing that, since returning impl 'lt + Future<…> + Send can incidentally trigger that other Rust bug), at some point it will start complaining about the impl Acquire<'a> parameter not being Send (this is not that surprising, although the type you instantiated this generic function with was Send: it looks like Rust failed to derive a Send impl of the returned existential when the generic input was Send?), and so when we add a + Send to that, stuff passes

fn uses_txn<'a> (
//               2: Rust then asked us to add this.
//               vvvvvv
    cxn: impl 'a + Send + sqlx::Acquire<'a, Database = Postgres>,
) -> impl 'a + Future<Output = ()> + Send {
//                                 ^^^^^^
//                                 1: we assert here that the future is `Send`.
    async move {
        let mut cxn = cxn.acquire().await.unwrap();
        let _exec = cxn.deref_mut();
    }
}

Here is my original playground, but with that + Send added there:

- async fn uses_txn<'a>(cxn: impl sqlx::Acquire<'a, Database = Postgres>) {
+ fn uses_txn<'a>(cxn: impl 'a + Send + sqlx::Acquire<'a, Database = Postgres>)
+   -> impl 'a + Future<Output = ()> + Send
+ { async move {
      let mut cxn = cxn.acquire().await.unwrap();
      let _exec = cxn.deref_mut();
- }
+ }}

Also, if we had started with my own tip regarding the clippy::future_not_send usage (for some reason I haven't done it myself :laughing:), by applying it to my original playground with the last problematic snippet commented out, we quickly identify that the culprit was that function:

error: future cannot be sent between threads safely
   --> src/lib.rs:107:73
    |
107 | async fn uses_txn<'a>(cxn: impl sqlx::Acquire<'a, Database = Postgres>) {
    |                                                                         ^ future returned by `uses_txn` is not `Send`
    |
note: the lint level is defined here
   --> src/lib.rs:1:9
    |
1   | #![deny(clippy::future_not_send)]
    |         ^^^^^^^^^^^^^^^^^^^^^^^
note: future is not `Send` as this value is used across an await
   --> src/lib.rs:108:19
    |
107 | async fn uses_txn<'a>(cxn: impl sqlx::Acquire<'a, Database = Postgres>) {
    |                       --- has type `impl sqlx::Acquire<'a, Database = Postgres>` which is not `Send`
108 |     let mut cxn = cxn.acquire().await.unwrap();
    |                   ^^^^^^^^^^^^^^^^^^^ await occurs here, with `cxn` maybe used later
109 |     let _exec = cxn.deref_mut();
110 | }
    | - `cxn` is later dropped here
    = note: `impl sqlx::Acquire<'a, Database = Postgres>` doesn't implement `std::marker::Send`
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#future_not_send
1 Like

Ah, you had put me on the right track and I had just figured out the same thing!

Here is the playground I was using with the fix

Thank you so, so much for your help. I've been trying to figure this out all day every day since Monday, so really this is a huge relief.

I also really appreciate you showing me your process for figuring out where the error is coming from! That let _: &dyn Send trick is really stellar, and dropping the async syntactic sugar for using BoxFuture helped a lot, also.

1 Like

If you're ever in Austin, please look me up so I can buy you a beer (or a beverage of your choice!)

1 Like

Alas, I live in Europe, but thanks for the offer! :grinning_face_with_smiling_eyes:

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.