Conflicting lifetime requirements in DIY Stream

I'm trying to write a Stream that drives some other async function, i.e. Future to completion. The Future depends on the state of the Stream, so I'm using BoxFuture.
Here's the exemplified code -

struct DiyStream<'a> {
    counter: usize,
    provider: Box<dyn Provider>,
    fut: Option<BoxFuture<'a, Vec<usize>>,
}

impl Stream for DiyStream<'_> {
    type Item = Vec<usize>;

    fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
           // codes to drive fut & other stuff is omitted 
            self.counter+=1;
            let fut = self.provider.get(self.counter);
            self.fut = Some(fut.boxed());
            Poll::Pending
        }
    }
}

#[async_trait]
pub trait Provider{
    async fn get(&self, n: usize) -> Ret;
}

But I'm getting the following error -

   Compiling playground v0.0.1 (/playground)
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
  --> src/lib.rs:37:23
   |
37 |             let fut = self.provider.get(self.counter);
   |                       ^^^^^^^^^^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 24:5...
  --> src/lib.rs:24:5
   |
24 |     fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...so that the reference type `&Pin<&mut DiyStream<'_>>` does not outlive the data it points at
  --> src/lib.rs:37:23
   |
37 |             let fut = self.provider.get(self.counter);
   |                       ^^^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'_` as defined on the impl at 20:27...
  --> src/lib.rs:20:27
   |
20 | impl Stream for DiyStream<'_> {
   |                           ^^
note: ...so that the expression is assignable
  --> src/lib.rs:38:24
   |
38 |             self.fut = Some(fut.boxed());
   |                        ^^^^^^^^^^^^^^^^^
   = note: expected `Option<Pin<Box<dyn futures::Future<Output = Option<Vec<usize>>> + std::marker::Send>>>`
              found `Option<Pin<Box<dyn futures::Future<Output = Option<Vec<usize>>> + std::marker::Send>>>`

The error is not unexpected, as both self.provider & poll_next() has anonymous lifetime.

Wrapping provider with Arc e.g. (provider: Arc<Box<dyn Provider + Send+Sync>>) solves the error but it also requires RwLock to get mutable reference, e.g. provider: Arc<RwLock<Box<dyn Provider + Send+Sync>>>.

Is there any other ergonomic way to achieve that without `Arc<RwLock>?

I'm not so fan of lock. Also if I'm not wrong, Future requires Send, not Sync, so RwLock seems unnecessary, yet a requirement.

Playground

The problem is that your struct is self-referential, that is, the fut field stores references into the provider field. To avoid this, you could store the provider inside the future like this:

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=425327179b43ffce59fb1c85fdf8ac04

If you construct the future in the constructor of your stream, you may even be able to avoid the enum entirely.

1 Like

The following example does something similar:

Thank you so much!!:star_struck: That's a cool piece of Rust & works like magic!:smiley: :heart: