How to solve returning this value requires that `'life1` must outlive `'static`

help
1

I'm having a compilation error related to lifetimes, and I'm unsure about how to solve it though there are already similar questions.
I'm seeking some guidance or assistance in this matter. I am not experienced in Rust at all so please bear in mind that the code I've written might be quite naive.

I defined Transaction trait for a custom transaction mechanism. This trait has a method execute that takes an operation (op ) as an argument, which is a closure encapsulating whatever database operation to be executed within the transaction.
I also defined a struct SeaOrmTransaction that implements the trait for SeaORM transaction

Here is the code I wrote. (I wanted to put a reproduction code in Rust playground but I did not know how to make Sea ORM crate available there)

use sea_orm::{DatabaseConnection, TransactionTrait, DbErr};
use std::sync::Arc;
use anyhow::Result;
use futures_util::future::BoxFuture;

pub type Op<'a> = Box<dyn 'a + Send + FnOnce() -> BoxFuture<'a, Result<()>>>;

#[async_trait]
pub trait Transaction {
    async fn execute(&self, op: Op<'_>) -> Result<()>;
}

pub struct SeaOrmTransaction {
    pub db: Arc<DatabaseConnection>,
}

#[async_trait]
impl Transaction for SeaOrmTransaction {
    async fn execute(&self, op: Op<'_>) -> Result<()> {
        self.db.transaction::<_, (), DbErr>(|tx| {
            let c = async move {
                let res = op().await;
                if let Err(_) = res {
                    return Err(DbErr::Custom(String::from("error")));
                }
                Ok(())
            };
            Box::pin(c)
        })
        .await?;
        Ok(())
    }
}

When I tried building the code below, I got following error.
Why does life1 have to outlive 'static?

error: lifetime may not live long enough
  --> src/domain/test.rs:28:13
   |
19 |     async fn execute(&self, op: Op<'_>) -> Result<()> {
   |                                    -- lifetime `'life1` defined here
...
28 |             Box::pin(c)
   |             ^^^^^^^^^^^ returning this value requires that `'life1` must outlive `'static`
2

TransactionTrait::transaction has this signature:

async fn transaction<F, T, E>(&self, callback: F) -> ...
where
    F: for<'c> FnOnce(..., &'c ...) -> Pin<Box<dyn Future<...> + 'c + ...>> + ...
       ^^^^^^^             ^^^                 ^^^^^^^^^^^^^^^^^^^^
    ...

This means that the caller of F expects a future which can live as long as caller-provided 'c. The compiler has to assume worst-case when checking your future; 'c may live as long as 'static.

2 Likes
3

Thanks for your response.
I thought the future c (apologize for the confusing name) has the lifetime of life1 and therefore lifetime 'c is inferred to be life1 as well because the clousure returns Box::pin(c), but isn't this how inference works?

4

I also tried explicitly annotating lifetime as follows but still get the same error.
I think I misunderstand something basic.

#[async_trait]
impl Transaction for SeaOrmTransaction {
    async fn execute<'a>(&'a self, op: Op<'a>) -> Result<()> {
        self.db.transaction::<_, (), DbErr>(|_tx: &'a DatabaseTransaction | {
            let c = async move {
                let res = op().await;
                if let Err(_) = res {
                    return Err(DbErr::Custom(String::from("error")));
                }
                Ok(())
            };
            Box::pin(c)
        })
        .await?;
        Ok(())
    }
}
5

To learn the HRTB pointed above, you can reduce your code Rust Playground

Or even simpler (no BoxFuture) Rust Playground

fn main() {
    let s = String::new();
    works(|_| &s);
    fails(|_| &s); // error: returning this value requires that `s` is borrowed for `'static`
    execute(&s);
}

fn fails(_: impl FnOnce(&()) -> &str) {}
fn works<'a>(_: impl FnOnce(&()) -> &'a str) {}

fn execute(s: &str) {
    works(|_| s);
    fails(|_| s); // error: returning this value requires that `'1` must outlive `'static`
}

transaction follows the failed pattern, so Op must be 'static.

3 Likes
6

transaction may have intentionally followed the failed pattern.

The future has a + Send bound, which means transaction() may send it to another thread. That creates this possibility:

  • An async function x has a local variable v and calls your function with an Op which captured it by reference. Op's 'a allows this.
  • You call transaction(). Assume it allowed your non-static lifetime.
  • It calls your callback to get your future
  • It passes the future to another thread
  • It awaits for the thread's completion
  • Something cancels. e.g. someone called x within a select!. There are many ways to cancel that aren't marked unsafe.
  • This drops x, which contains v
  • The thread executes your future, which invokes Op, which uses the now-dangling reference to v

The implied 'static prevents this from compiling.

This is one of the big reasons I love Rust! It's hard to prevent accidents like this in C++.

2 Likes
7

Does working pattern work because without lifetime annotation prevents lifetime of &str from being inferred 'static?
If so is it possible to achieve the same thing in my code like I did in the code above (that's not compiling though)?.

8

transaction requires your future to be 'static. You can't use annotations to relax transaction's requirement unless you fork its code, change its signature, and likely make other changes to it (e.g. drop threading). You can, however, make your future static by placing a requirement on op:

async fn execute(&self, op: Op<'static>) -> Result<()>

Here's an Example of what the 'static might be protecting you from. The cases seem similar - a background thread might be ultimately be the reason why transaction has that requirement.

4 Likes
9

transaction requires your future to be 'static .

I see.
One more question: Is lifetime 'c always inferred 'static? What are the cases it is inferred non-'static ?

Thanks for your patience...

10

That's not correct. 'c is not inferred as 'static here. It's hard to explain becuase I'm not sure what I've missed. I'll show you by going a little deep wrt my first code link.

use futures::future::{BoxFuture, FutureExt};
fn main() {
    let s = String::new();
    execute(&s);
}

// i.e. fn works<'b>(_f: impl for<'a> FnOnce(&'a ()) -> BoxFuture<'b, ()>) {}
fn works<'b>(_f: impl FnOnce(&()) -> BoxFuture<'b, ()>) {}

// i.e. fn take_cb(_f: impl for<'a> FnOnce(&'a ()) -> BoxFuture<'a, ()>) {}
fn fails(_f: impl FnOnce(&()) -> BoxFuture<'_, ()>) {}

fn execute<'life>(s: &'life str) {
    works(|_| async { s; }.boxed());
    fails(|_| async move { s; }.boxed()); // this emits the same error you've got
}

error: lifetime may not live long enough
  --> src/main.rs:15:15
   |
13 | fn execute<'life>(s: &'life str) {
   |            ----- lifetime `'life` defined here
14 |     works(|_| async { s; }.boxed());
15 |     fails(|_| async move { s; }.boxed());
   |               ^^^^^^^^^^^^^^^^^^^^^^^^^ returning this value requires that `'life` must outlive `'static`

'c is still an input lifetime that the caller passes and is probably non-static.
'static in the error msg refers to the captured lifetime, i.e. Op<'_> in your code and &'life str in my code.

11

There are 2 answers to this:

  • When something is calling your callback to get the future, the compiler can see how the future is used or stored, and can see the lifetime of the argument passed to your callback. This enables it to infer a 'c shorter than 'static. It can also infer a 'c equal to 'static if the future is passed to a context which requires 'static, e.g. a thread, assuming the argument also has a 'static lifetime.
  • When you're implementing the callback, the compiler is blind to the above. The for<'c> prevents the compiler from assuming any relationship between 'c and other lifetimes; it has to assume 'static since it's feasible that a caller needs 'static.
3 Likes
12

Thanks for your example code.
Hmm...I don't understand why works work, more specifically, why putting lifetime annotation after method name makes it work. What kind of difference does it make?

13

When you're implementing the callback, the compiler is blind to the above.

Is it correct to say that when you're implementing the callback, captured variables (in my code Op<'_>) must have 'static lifetime because the compiler must consider the worst case?

14

Yes

1 Like
15

Why works has allow_non_static_captures pattern:

async fn allow_non_static_captures<'capture_by_cb, 'capture_by_fut>(
    f: impl 'capture_by_cb + FnOnce(&/* 'input */()) -> BoxFuture<'capture_by_fut, ()>,
){
    f(&/* 'input */()).await;
}

async fn execute<'life>(s: &'life str) {
    allow_non_static_captures(|input| f(input, s)).await;
    // equivalent as
    // allow_non_static_captures(|_| {
    //     let _capture_by_cb = s;
    //     async move {
    //         let _capture_by_fut = _capture_by_cb;
    //     }
    //     .boxed()
    // }).await;
}

// Note: 'capture_by_cb has no lifetime connection with 'input
fn f<'input, 'capture_by_cb>(_: &'input (), s: &'capture_by_cb str) -> BoxFuture<'capture_by_cb, ()> {
    async move { let _capture_by_fut = s; }.boxed()
}

Why fails has only_static_captures pattern:

async fn only_static_captures<'capture_by_cb>(
    f: impl 'capture_by_cb + for<'input> FnOnce(&'input ()) -> BoxFuture<'input, ()>,
) {
    f(&/* 'input */()).await;
}

async fn execute<'life>(s: &'life str) {
    only_static_captures(move |input| f(input, s)).await;
}

// Note: 'capture_by_cb must outlives 'input, and because 'input can be any input lifetime,
//       'capture_by_cb should outlives 'static when 'input is 'static in the worst case.
fn f<'input, 'capture_by_cb: 'input>(_: &'input (), s: &'capture_by_cb str) -> BoxFuture<'input, ()> {
    async move { s; }.boxed()
}

Exactly.

4 Likes
16

I found an interesting detail: allow_non_static_captures works only when the return value/Future holds the capture-related lifetime, it won't work when the Future holds the input.

// Note: the input is in the Future this time
13 |     allow_non_static_captures(|a| async move { a; s; }.boxed()); // error: `'1` must outlive `'2`
   |                                -- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
   |                                ||
   |                                |return type of closure is Pin<Box<(dyn futures::Future<Output = ()> + std::marker::Send + '2)>>
   |                                has type `&'1 ()`

So the ideal (in my case) is to define three distinct lifetimes that have no lifetime connection with each other Rust Playground

fn ideal<'input, 'capture, 'fut>(_f: impl 'capture + FnOnce(&'input ()) -> BoxFuture<'fut, ()>) {}

Update: forget to mention that the downside of ideal is 'input becomes an outer lifetime and rarely useful because usually the argument in the callback is a temporarily referenced type which rarely satisfies 'input Rust Playground

// Note: 'input is not any input lifetime any more, it's a lifetime outside
//       the function, meaning you can't feed a lifetime inside the function to f
fn ideal<'input, 'capture, 'fut>(
    input: &'input (),
    f: impl 'capture + FnOnce(&'input ()) -> BoxFuture<'fut, ()>,
) {
    // f(&()); // Ok due to f(&'static ())
    
    // f(input) // Ok due to f(&'input ())
    
    let a = (); // local variable, thus &a can't be &'input ()
    // f(&a); // error: argument requires that `a` is borrowed for `'input`
}

The real ideal might be for<'input: 'fut> FnOnce(&'input ()) -> BoxFuture<'fut, ()>, an extended HRTB, which is said to be unneeded in most cases.

// Note: there is a lifetime bound in HRTB
fn ideal<'capture, 'fut>(
    f: impl 'capture + for<'input: 'fut> FnOnce(&'input ()) -> BoxFuture<'fut, ()>,
) {
    let a = ();
    f(&a);
}

Wrap up: lifetime contracts come up with all sorts of trade-offs. only_static_captures is the relatively easy, common and idiomatic pattern.

1 Like
17

What you might be missing here is that lifetimes (like bounds in general) look differently from the caller's and the callee's perspective.

A lifetime or trait bound is a capability that's required to be provided by the caller and can be relied on by the callee. Hence:

  • the stronger the bound, the fewer callers can satisfy it, but the more useful it is for the callee (it can do more stuff with it)
  • the weaker the bound, the easier it is to satisfy for the callers, and the less useful it is for the callee.

In addition, there's another thing at play when an API is expecting a callback: functions consume values that appear in their arguments and produce values that appear in their return type. Therefore, arguments of a callback that's passed to a higher-order function invert this relationship, since the callback that the caller passes will accept values from the callee. The technical term for this is that "function types are contravariant in their argument type".

Now, for<'a> T: 'a is the strongest bound there is. It looks like:

  • "it has to work for all lifetimes" from the point of view of the caller, and
  • "I can pass whatever I can" from the PoV of the callee.

Putting these together, when you have a signature like

fn higher_order(callback: impl for<'a> FnOnce(&'a T))

(for some type T), then the following happens:

  • higher_order specifies that it requires a callback that will work correctly when passed an argument type of arbitrarily short (or long) validity
  • which is very useful for the implementation of higher_order, because it can pass whatever reference it wants, including references to local variables; the callback essentially promises not to hold onto the reference for any longer than its own body is running.
  • OTOH, of course, the same fact severely limits what kind of callbacks the caller can pass in – for example, it will be prohibited from passing anything that only accepts a &'static reference, or one with a concrete lifetime parameter that may come from eg. an outer generic struct or function.

To sum it up: what's important is the direction of data flow between entities. There is no magic and the rules aren't ad-hoc; they are all in place they way they are because they are coherent and uphold memory safety like so.

6 Likes
18

Thank you guys for detailed answers and sorry for the late response...
I was totally missing the fact that A lifetime or trait bound is a capability that's required to be provided by the caller and can be relied on by the callee.

Regarding caller providing actual lifetime, I have another question but this may be off-topic from the original question...
In the following code (which does not compile), is the lifetime of 'a provided when the compiler sees the instantiation of Person struct or when it sees the method call person.get_name?
If it is the former then the 'a is bound to the lifetime of name. And get_name(&'a self) requires reference to self outlives 'a but person (which self referencese) does not outlive therefore it does not compile. But if this assumption is correct even if I removed the line println!("{}", val); it should not still compile, but it actually compiles.
So is the latter interpretation correct?

struct Person<'a> {
    name: &'a str,
}

impl<'a> Person<'a> {
    fn get_name(&'a self) -> &'a str {
        return self.name;
    }
}

fn main() {
    let val: &str;
    let name = String::from("john");
    {
        let person = Person {
            name: &name,
        };
        val = person.get_name();
    }
    println!("{}", val);
}

$ cargo run
   Compiling lifetime-sandbox v0.1.0 (/home/hitochan/developer/lifetime-sandbox)
error[E0597]: `person` does not live long enough
  --> src/main.rs:55:19
   |
52 |             let person = Person {
   |                 ------ binding `person` declared here
...
55 |             val = person.get_name();
   |                   ^^^^^^^^^^^^^^^^^ borrowed value does not live long enough
56 |         }
   |         - `person` dropped here while still borrowed
57 |         println!("{}", val);
   |                        --- borrow later used here
19

The lifetime 'a in your code corresponds to the lifetime of the instance of Person. The error appears when the get_name method is called, because it's borrowing self for its entire lifetime.

1 Like
20

@moy2010

Thanks for answering.

The lifetime 'a in your code corresponds to the lifetime of the instance of Person.

Does this apply only when you call a method where &self is annotated with 'a?

Because if I simply assign person.name to a variable that outlives person I can still access it even after person is dropped (in the following code).
This implies that 'a corresponds to the lifetime of name.

fn main() {
    let val: &str;
    let name = String::from("john");
    {
        let person = Person {
            name: &name,
        };
        val = person.name;
    }
    println!("{}", val);
}