Lifetimes: Adding a parameter inside a 3rd party closure

[taqtiqa-mark]

I'm new to Rust lifetimes and closures in practice (I've read but the necessary lesson/insight has clearly escaped me), but I do understand they are 'descriptive' not 'prescriptive' annotations. So errors are on me.

That said I have the following function:

fn get_pages() -> impl Stream<Item = Vec<usize>> {
    stream::iter(0..).then(move |i| get_page(i))
}

Which I am trying to extend to handle gracefully shutting down. I've resolved lifetime issues to the point I am stuck here:

fn get_pages<'a>(signal: &'a mut Signal<'a>) -> impl Stream<Item = Vec<usize>> + 'a {
    stream::iter(0..).then(move |i| get_page(i, signal))
}

Which generates this amazing (good) error message:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter 'a in function call due to conflicting requirements
   --> regatta/examples/05-pages-stream-4-ok-sig-2-c.rs:218:37
    |
218 |     stream::iter(0..).then(move |i| get_page(i, signal))
    |                                     ^^^^^^^^^^^^^^^^^^^
    |
note: first, the lifetime cannot outlive the lifetime `'_` as defined on the body at 218:28...
   --> regatta/examples/05-pages-stream-4-ok-sig-2-c.rs:218:28
    |
218 |     stream::iter(0..).then(move |i| get_page(i, signal))
    |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...so that closure can access `signal`
   --> regatta/examples/05-pages-stream-4-ok-sig-2-c.rs:218:49
    |
218 |     stream::iter(0..).then(move |i| get_page(i, signal))
    |                                                 ^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the function body at 217:14...
   --> regatta/examples/05-pages-stream-4-ok-sig-2-c.rs:217:14
    |
217 | fn get_pages<'a>(signal: &'a mut Signal<'a>) -> impl Stream<Item = Vec<usize>> + 'a {
    |              ^^
note: ...so that the expression is assignable
   --> regatta/examples/05-pages-stream-4-ok-sig-2-c.rs:218:49
    |
218 |     stream::iter(0..).then(move |i| get_page(i, signal))
    |                                                 ^^^^^^
    = note: expected `&mut Signal<'_>`
               found `&mut Signal<'a>`

error[E0308]: mismatched types
   --> regatta/examples/05-pages-stream-4-ok-sig-2-c.rs:176:55
    |
176 |     println!("First 10 pages:\n{:?}", get_n_pages(10, signal).await);
    |                                                       ^^^^^^
    |                                                       |
    |                                                       expected `&mut Signal<'_>`, found struct `Signal`
    |                                                       help: consider mutably borrowing here: `&mut signal`

Some errors have detailed explanations: E0308, E0495.
For more information about an error, try `rustc --explain E0308`.

The existing cases I find on the inter-tubes tend to deal with the case where you control the closure. My questions are:

1. Can this be fixed by changing lifetime annotation details alone?
2. If not, is there a proper/idiomatic Rust way of introducing a parameter into a closure such as this?

Unrelated to fixing the issue: When the compiler indicates: '_ as defined on the body at 218:28` I'm curious to know what expression would look like that it thinks is there.

Wider context is here:

Gracefully shutting down in Tokio: Using channel TX in task

I think this might be a better medium for this question than the Discord channel.... apologies for the cross post. While some useful direction came from Discord group, the question in case 3. below is still open, and I'd like to understand how the borrow checker can accept a function that used all three of the channel Sender functions - if there is a limit on what combinations of the three can be used in one function the docs don't yet set out those limits. The tokio::aync::oneshot::channel se…

[2e71828]

By using 'a twice in &'a mut Signal<'a>, you're asking the compiler to find a single lifetime 'a that can be used in both of those places. Because &mut T is invariant in T, the compiler doesn't have the option of picking a short lifetime for 'a. Instead, it tries to make the borrow extend over the entire lifespan of the Signal object, which is impossible.

One way to fix this is to use two different lifetime parameters (untested):

fn get_pages<'a,'b:'a>(signal: &'a mut Signal<'b>) -> impl Stream<Item = Vec<usize>> + 'a {
    stream::iter(0..).then(move |i| get_page(i, signal))
}

[alice]

Beyond what @2e71828 mentions, you are probably running in to the issue that futures returned in combinators such as StreamExt::then are generally prevented from borrowing things. This is because if the closure you passed to then is called a second time without first destroying the future returned by the first call, then both returned futures contain a mutable reference to signal, which is in violation with the rule that all mutable references must have exclusive access to the thing they point at.

The easiest way to fix this is to use the async-stream crate instead of then. This eliminates some function boundaries that the compiler cannot see through, and should make the compiler accept your code.

[taqtiqa-mark]

Thanks @2e71828 and @alice. Using your advice I was able to handle gracefully shutting down with this in place of the original:

use async_stream::stream;
fn get_pages<'a>(mut signal: &'a mut Signal<'a>) -> impl Stream<Item = Vec<usize>> + 'a
{
    stream! {
        for i in 0.. {
           yield get_page(i, &mut signal).await;
        }
    }
}

Hope that helps someone else?