Lifetime Annotation Problem

help
1

Hi guys,

I'm facing a problem with lifetimes which I can't seem to find a way around. I have the following code:

struct X;

struct Y<'a> {
    x: &'a mut X
}

struct Z<'a> {
    x: &'a mut X
}

impl<'a> futures::Stream for Z<'a> {
    type Item = Y<'a>;
    type Error=();
    fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
        Ok(Async::Ready(Some(Y { x: self.x })))
    }
}

There are three structs X, Y and Z. Y and Z carry a mutable reference to X. Z implements Stream from the futures crate. In Stream::poll(), Z is returning an instance of Y to which it's lending its own reference of X (see the expression x: self.x).

The compiler is complaining that it's unable to figure out a lifetime for the outgoing Y instance:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
  --> src\main.rs:39:30
   |
39 |         Ok(Async::Ready(Some(Y { x: self.x })))
   |                              ^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 38:5...
  --> src\main.rs:38:5
   |
38 | /     fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
39 | |         Ok(Async::Ready(Some(Y { x: self.x })))
40 | |     }
   | |_____^
note: ...so that reference does not outlive borrowed content
  --> src\main.rs:39:37
   |
39 |         Ok(Async::Ready(Some(Y { x: self.x })))
   |                                     ^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the impl at 35:1...
  --> src\main.rs:35:1
   |
35 | / impl<'a> futures::Stream for Z<'a> {
36 | |     type Item = Y<'a>;
37 | |     type Error=();
38 | |     fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
39 | |         Ok(Async::Ready(Some(Y { x: self.x })))
40 | |     }
41 | | }
   | |_^
note: ...so that types are compatible (expected futures::Stream, found futures::Stream)
  --> src\main.rs:38:65
   |
38 |       fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
   |  _________________________________________________________________^
39 | |         Ok(Async::Ready(Some(Y { x: self.x })))
40 | |     }
   | |_____^

error: aborting due to previous error

My question is two fold:

  1. Why can't the compiler assign lifetime 'a to the Y instance? It is clear from the code that whatever is the lifetime of self, it is gotta be longer than 'a. Why is the inference not happening then? Is it something to do with this being a mutable reference rather than immutable?
  2. How do I annotate to help the compiler here? I tried the following (annotating self with 'a):
impl<'a> futures::Stream for Z<'a> {
    type Item = Y<'a>;
    type Error=();
    fn poll(&'a mut self) -> Poll<Option<Self::Item>, Self::Error> {
        Ok(Async::Ready(Some(Y { x: self.x })))
    }
}

And the following (putting a bound of 'a' over 'b, the lifetime of self):

impl<'a> futures::Stream for Z<'a> {
    type Item = Y<'a>;
    type Error=();
    fn poll<'b: 'a>(&'b mut self) -> Poll<Option<Self::Item>, Self::Error> {
        Ok(Async::Ready(Some(Y { x: self.x })))
    }
}

But now the compiler complains about a mismatch with the trait declaration with the following errors respectively. This:

error[E0308]: method not compatible with trait
  --> src\main.rs:38:5
   |
38 | /     fn poll(&'a mut self) -> Poll<Option<Self::Item>, Self::Error> {
39 | |         Ok(Async::Ready(Some(Y { x: self.x })))
40 | |     }
   | |_____^ lifetime mismatch
   |
   = note: expected type `fn(&mut Z<'a>) -> std::result::Result<futures::Async<std::option::Option<Y<'_>>>, ()>`
              found type `fn(&'a mut Z<'a>) -> std::result::Result<futures::Async<std::option::Option<Y<'_>>>, ()>`

and this:

error[E0195]: lifetime parameters or bounds on method `poll` do not match the trait declaration
  --> src\main.rs:38:5
   |
38 | /     fn poll<'b: 'a>(&'b mut self) -> Poll<Option<Self::Item>, Self::Error> {
39 | |         Ok(Async::Ready(Some(Y { x: self.x })))
40 | |     }
   | |_____^ lifetimes do not match trait

It seems really hard to annotate this without deviating from the trait declaration. Any help will be much appreciated :slight_smile:

2

I have a growing feeling of dread that this is due to mutable references. The compiler is trying to save me from having mutable references with mutually overlapping lifetimes. Is this correct?

3

Yes, this is due to mutable references. This comes up quite often, usually in the context of an Iterator returning mutable references. You can also get a bit more color if you search for "streaming iterators", which is a way to tie the mutable borrow to the self borrow - that prevents holding (potentially) multiple mutable aliases to the same value.

The long term solution to this is ATC (associated type constructors) which would allow you to express the streaming nature without having multiple versions of a trait.

You can work around this by using unsafe or switching to something like RefCell holding the value and letting it enforce borrow rules at runtime.

4

Your suspicion is correct that this is a consequence of Rust's rule that something cannot be aliased and mutable at the same time.

Here is a simplified version of the signature of your poll method. &mut Z<'a> goes in and Y<'a> comes out.

fn poll<'a>(z: &mut Z<'a>) -> Y<'a> {
    unimplemented!()
}

Based on this signature, it would be perfectly valid for some caller to use poll as follows.

fn main() {
    let mut x = X;
    let mut z = Z { x: &mut x };
    let y = poll(&mut z) /* exclusive borrow of z ends here */;
    
    // *z.x and *y.x are both mutable, so they must not be aliased.
    impl X {
        fn mutate(&mut self) {}
    }
    z.x.mutate();
    y.x.mutate();
}

After the call to poll, the values of *z.x and *y.x are in scope and both mutable. That means whatever code takes the place of the unimplemented!() must not produce a situation in which *z.x and *y.x are the same object, as that would violate the rule that something cannot be simultaneously aliased and mutable.

The "cannot infer an appropriate lifetime for lifetime parameter 'a due to conflicting requirements" error is Rust enforcing that the implementation of poll does not make it possible for a caller correctly using the signature of poll to end up in a situation where a thing is aliased and mutable.

5

Brilliant!

Thanks guys :slight_smile: Not only did it get me unblocked, it also helped me understand lifetimes a little bit better.

@vitalyd I was gonna make this code thread-safe anyway later. So I can simply switch to using Mutex (rather than RefCell) and that should solve this problem as well, right?

6

Yup. You’d return an Arc<Mutex> from poll in that case rather than an Rc<RefCell>.

By the way, what’re the semantics you’re after here? Do you really want to yield the same value each time? Or is it a one-shot style of stream - yield the item once and then return None thereafter?

7

Yes. I'll be yielding the same reference to X in each call to poll(). There are some other fields in Y (not shown here) which will differ on each invocation. So, no, it's not one shot. Perhaps you were meaning to suggest an simpler alternative if it indeed was one shot, but it isn't unfortunately. :frowning:

8

Yeah, there could be a simpler solution for oneshot :slight_smile:. But that's moot ...

You might be able to arrange the design to avoid returning mutable references (or Arc<Mutex>) from poll. For example, instead of sharing data encapsulated in a mutex, send messages between threads (using built-in channels or some other queuing/communication mechanism); poll might then return some (readonly/immutable) data that would be used to generate a message to send to another thread. Or, poll returns the same data but the update/mutation (under a Mutex or via channels) is done by a different component. It's hard to say more without knowing details but I'd encourage you to explore alternative designs.

9

Absolutely. I was planning to make the code thread-safe later and only make it work for a single thread for now and I did plan on looking at Go-like sync via channels for multithreading rather than going with locks. So yes, I'm definitely going to step back and reassess the big picture here. Thanks for the suggestion :slight_smile:

1 Like