Lifetime issue with splitting Session to Reader and Writer

I'm trying to implement implement a trait that requires splitting a Session that can read/write into a Reader/Writer structs. I plan using a Mutex for synchronization.
The issue is with lifetimes which I can't get to do right. I tried multiple options (with Rc, Option, Weak for referencing the main struct) and nothing worked so it's about something I just don't understand:
Here is the simplified code, and below the error I get.
What's wrong and what should I change if at all possible to do?

// Wrapper of session 
pub struct SessionWrapper<'a> {
    session: Mutex<NoopRawMutex, Session<'a, TcpSocket<'a>>>,
}

impl<'a,'s> SessionWrapper<'s> 
where 
    's: 'a
{
    pub fn new(session: Session<'s, TcpSocket<'s>>) -> Self {
        Self {
            session: Mutex::new(session)
        }
    }
}

// Reader

pub struct SessionReader<'a> {
    session: &'a Mutex<NoopRawMutex, Session<'a, TcpSocket<'a>>>
}

impl<'a> embedded_io_async::ErrorType for SessionReader<'a> {
    type Error = TlsError;
}

impl<'a> embedded_io_async::Read for SessionReader<'a> {
    async fn read(&mut self, buf: &mut [u8]) -> Result<usize, Self::Error> {
        let mut session = self.session.lock().await;
        session.read(buf).await
    }
}

pub struct SessionWriter
{
}

impl embedded_io_async::ErrorType for SessionWriter
{
    type Error = TlsError;
}

impl embedded_io_async::Write for SessionWriter
{
    // async fn write(&mut self, _buf: &[u8]) -> Result<usize, Self::Error> {
    //     Ok(0)
    // }
    //
    // async fn flush(&mut self) -> Result<(), Self::Error> {
    //     Ok(())
    // }
}


// Implement picoserve Socket on SessionWrapper
impl <'s> picoserve::io::Socket for SessionWrapper<'s>{
    type Error = TlsError;
    type ReadHalf<'a> = SessionReader<'a> where 's: 'a;
    type WriteHalf<'a> = SessionWriter where 's : 'a;

    fn split(&mut self) -> (Self::ReadHalf<'_>, Self::WriteHalf<'_>) {
        let session = &self.session;
        (
            SessionReader{
                session
            },
            SessionWriter {},
        )
    }

    async fn shutdown<Timer: picoserve::Timer>(
        mut self,
        _timeouts: &picoserve::Timeouts<Timer::Duration>,
        _timer: &mut Timer,
    ) -> Result<(), picoserve::Error<Self::Error>> {
        Ok(())
    }
}`

Error

error: lifetime may not live long enough
   --> src/framework/web_config.rs:389:9
    |
382 |   impl <'s> picoserve::io::Socket for SessionWrapper<'s>{
    |         -- lifetime `'s` defined here
...
387 |       fn split(&mut self) -> (Self::ReadHalf<'_>, Self::WriteHalf<'_>) {
    |                - let's call the lifetime of this reference `'1`
388 |           let session = &self.session; // Mutably borrow the session inside the Option
389 | /         (
390 | |             SessionReader{
391 | |                 session
392 | |             },
393 | |             SessionWriter {},
394 | |         )
    | |_________^ method was supposed to return data with lifetime `'s` but it is returning data with lifetime `'1`
    |
    = note: requirement occurs because of the type `SessionReader<'_>`, which makes the generic argument `'_` invariant
    = note: the struct `SessionReader<'a>` is invariant over the parameter `'a`
    = help: see <https://doc.rust-lang.org/nomicon/subtyping.html> for more information about variance

When you use self to borrow something it owns, you don't get to say what arbitrary lifetime it has. It will be a lifetime related to the lifetime of the existing self loan, and that lifetime can only be as long as the current loan of self or shorter.

If you have some <'a> lifetime on a struct, trait, impl, it refers to a broad long-lived lifetime of something that could have been borrowed before your function has been called and can remain borrowed after your function returns. This makes it automatically incompatible with locally-scoped loans started inside functions, unless they are only re-borrowing from something already known to be borrowed for the scope of 'a.

Rust will never make anything live longer. It doesn't have ability to extend your lifetimes or prevent any object from being freed too early. It can only stop compilation if your code by itself on its own did not ensure that it's borrowing from data that lives long enough.

Your problem is that &self.session owned by self is only valid for as long as &mut self loan is, and therefore every struct that contains this reference to the session is also short-lived and bound by the ad-hoc short loan of &mut self implicitly created when the method is called.

You can't promise it will be 's, because that's unrelated. You're not borrowing from the place designated by 's. The <'a> on your struct is also unrelated. You're not borrowing fields borrowed for 'a, you're creating a completely different new loan when you borrow the owned Mutex.

Also avoid recycling the same lifetime label for multiple different loans in the same struct. This forces Rust to require they're equal. Rust can do that for shared loans by shortening them all to the most restrictive loan, which may limit too much what you can do with the struct. It's even worse when mixed with exclusive &mut, because it can require all of the objects to be borrowed at the same time from the same place.

It may be easiest to use Arc<Mutex<_>>, clone the Arc, and not use any lifetimes at all.

1 Like

In particular, this happens here:

pub struct SessionReader<'a> {
    session: &'a Mutex<NoopRawMutex, Session<'a, TcpSocket<'a>>>
}

You must use separate lifetime parameters for the borrow of the Mutex and the lifetimes that appear in the contents of the Mutex. Using a single parameter is guaranteed to get things unusably stuck.

pub struct SessionReader<'mutex, 'session> {
    session: &'mutex Mutex<NoopRawMutex, Session<'session, TcpSocket<'session>>>
}
1 Like

@kornel answered your question. But I'm also wondering whether the Session struct is defined to use the same lifetime param for its two lifetimes. If so, that may also be a problem. You haven't shown the Session struct.

@kornel @kpreid @ jumpnbrownweasel Thanks for the suggestions.

I tried Rc and Arc and had the same issue.
So I tried using Option and have Reader take ownership of Session from SessionWrapper, I thought this would be the most extreme, no kind of any reference from Reader to SessionWrapper at all, and it still doesn't work. Here is the code:

// Wrapper of session 
pub struct SessionWrapper<'a> {
    session: Option<Mutex<NoopRawMutex, Session<'a, TcpSocket<'a>>>>,
}

impl<'a,'s> SessionWrapper<'s> 
where 
    's: 'a
{
    pub fn new(session: Session<'s, TcpSocket<'s>>) -> Self {
        Self {
            session: Some(Mutex::new(session))
        }
    }
}

// Reader

pub struct SessionReader<'a> {
    session: Mutex<NoopRawMutex, Session<'a, TcpSocket<'a>>>
}

impl<'a> embedded_io_async::ErrorType for SessionReader<'a> {
    type Error = TlsError;
}

impl<'a> embedded_io_async::Read for SessionReader<'a> {
    async fn read(&mut self, buf: &mut [u8]) -> Result<usize, Self::Error> {
        let mut session = self.session.lock().await;
        session.read(buf).await
    }
}

pub struct SessionWriter
{
}

impl embedded_io_async::ErrorType for SessionWriter
{
    type Error = TlsError;
}

impl embedded_io_async::Write for SessionWriter
{
    async fn write(&mut self, _buf: &[u8]) -> Result<usize, Self::Error> {
        Ok(0)
    }

    async fn flush(&mut self) -> Result<(), Self::Error> {
        Ok(())
    }
}


// Implement picoserve Socket on SessionWrapper
impl <'s> picoserve::io::Socket for SessionWrapper<'s>{
    type Error = TlsError;
    type ReadHalf<'a> = SessionReader<'a> where 's: 'a;
    type WriteHalf<'a> = SessionWriter where 's : 'a;

    fn split(&mut self) -> (Self::ReadHalf<'_>, Self::WriteHalf<'_>) {
        (
            SessionReader{
                session: self.session.take().unwrap()
            },
            SessionWriter {},
        )
    }

    async fn shutdown<Timer: picoserve::Timer>(
        mut self,
        _timeouts: &picoserve::Timeouts<Timer::Duration>,
        _timer: &mut Timer,
    ) -> Result<(), picoserve::Error<Self::Error>> {
        Ok(())
    }
}

Error:

error: lifetime may not live long enough
   --> src/framework/web_config.rs:525:9
    |
519 |   impl <'s> picoserve::io::Socket for SessionWrapper<'s>{
    |         -- lifetime `'s` defined here
...
524 |       fn split(&mut self) -> (Self::ReadHalf<'_>, Self::WriteHalf<'_>) {
    |                - let's call the lifetime of this reference `'1`
525 | /         (
526 | |             SessionReader{
527 | |                 session: self.session.take().unwrap()
528 | |             },
529 | |             SessionWriter {},
530 | |         )
    | |_________^ method was supposed to return data with lifetime `'s` but it is returning data with lifetime `'1`
    |
    = note: requirement occurs because of the type `SessionReader<'_>`, which makes the generic argument `'_` invariant
    = note: the struct `SessionReader<'a>` is invariant over the parameter `'a`
    = help: see <https://doc.rust-lang.org/nomicon/subtyping.html> for more information about variance

I also tried to introduce two lifetimes to SessionReader, these two lifetimes then propogates up until Socket impl, and eventually it forces me to change the Socket Trait signature for ReadHalf to include two lifetimes, which I can't because I don't define the trait.

As for Session - this is the struct:

TcpSocket is the T there, and initially I tried using T in my struct with same constraints as in Session, when got to the Socket impl, I had to also add a lifetime to the constraint there, and the error ended up being the same.

Give this a shot:

 impl <'s> picoserve::io::Socket for SessionWrapper<'s>{
     type Error = TlsError;
-    type ReadHalf<'a> = SessionReader<'a> where 's: 'a;
+    type ReadHalf<'a> = SessionReader<'s> where 's: 'a;
+    //                                ^^
     type WriteHalf<'a> = SessionWriter where 's : 'a;

If I saw the attempt in code maybe I could suggest something here (or maybe not, don't know if it's worth your time).

This worked. I switched to use RC and used this and it compiles. Thanks!!!

Now I need to understand why. What does it mean to define a type as another type with lifetime?
Does it mean that we define the type alias with lifetime? Is it practically saying a type with a certain lifetime is equivalent to another type with another lifetime?

This lifetime thing is complex, something I still need to understand much better.

Is there a place that explains lifetimes real good?

Well, the short answer is this part of the error message:

note: the struct `SessionReader<'a>` is invariant over the parameter `'a`

Plus experience on my part, and... umm... in fact reading the error message closer, it gets the lifetimes backwards :frowning_face:.[1] Let's just look at the code ourselves then I suppose:

impl <'s> picoserve::io::Socket for SessionWrapper<'s> {
   //        +--- call this borrow '1
   //        v              vvvvvvvvvvvvvvvvvv should return `SessionReader<'1>`
   fn split(& mut self) -> (Self::ReadHalf<'_>, Self::WriteHalf<'_>) {
        (
            SessionReader{
                // Mutex<NoopRawMutex, Session<'s, TcpSocket<'s>>>
                //       vvvvvvvvvvvvvvvvvvv
                session: self.session.take().unwrap()
            },
            SessionWriter {},
        )
    }

Due to invariance, you can only create a SessionReader<'s>, but you said (via the GAT definition) that you would return a SessionReader<'1>. So I changed the definition to always be SessionReader<'s>.


It's instinctive that types that are parameterized by distinct type inputs (e.g. Vec<T> vs. Vec<U>) are different types. Types parameterized by distinct lifetimes are also distinct types. It's just less instinctive as variance often lets us pretend that this isn't the case, and lifetimes are often inferred and not explicit, etc.

So in the context of the implementation for SessionWrapper<'s>:

  • In the version that didn't compile, every ReadHalf<'a> was a different type (depending on 'a).
  • In the version that compiled, every ReadHalf<'a> was always the same type
    • and that's the only type that you could construct in the method, due to invariance
This part might be redundant with what I wrote above so I'll collapse it

You mean, this GAT situation?

impl <'s> picoserve::io::Socket for SessionWrapper<'s>{
    // We're definiing `<Self as Socket>::ReadHalf<'a> = SessionReader<'s>`,
    // that is, we have an alias parameterized by `'a` but we don't use `'a`
    // in the type that it aliases
    type ReadHalf<'a> = SessionReader<'s> where 's: 'a;

The trait allows you to define ReadHalf<'a> using the 'a lifetime, but you don't have to. It's there so you can borrow from *self in the method:

//        Return types can be dependent on the input lifetime
//        v-----------------------------vv-------------------vv
fn split(& mut self) -> (Self::ReadHalf<'_>, Self::WriteHalf<'_>)

But you don't have to. And in fact, you were already making use of this flexibility:

    type WriteHalf<'a> = SessionWriter where 's : 'a;

SessionWriter doesn't use 'a either.


Maybe think of it this way. You could have defined things like so:

pub struct SessionWrapper<T> {
    session: Option<Mutex<NoopRawMutex, T>>,
}
pub struct SessionReader<T> {
    session: Mutex<NoopRawMutex, T>
}

impl<'a> embedded_io_async::Read for SessionReader<Session<'a, TcpSocket<'a>>> {
// ...
}

impl<T> picoserve::io::Socket for SessionWrapper<T>
where
    SessionReader<T>: embedded_io_async::Read,
{
    type ReadHalf<'a> = SessionReader<T> where Self: 'a;
    // ...
}

Perhaps it's easier to see here that ReadHalf<'a> will be different for SessionWrapper<T> and SessionWrapper<U>, and that there's no 'a in the SessionReader<_> itself.

Even though the original involved a lifetime on the right side of the GAT, the solution was still about creating an alias that was the same for every input lifetime to the GAT. Maybe what I'm saying is, try thinking about it purely in terms of types and see if that removes some of the confusion around lifetimes.

I have some material here, but I don't know how well it really covers this case (other than mentioning invariance generally). I don't know of a comprehensive lifetime guide unfortunately. (It's a larger topic than one might think.)


  1. Oh look I've been here before. ↩ī¸Ž

2 Likes

Wow, thanks for the detailed answer.

It would take me some reading and learning to understand everything you explained, but I'm sure going to dive into this in details and use this as a basis for learning lifetimes better.

I find it the most complex area of Rust.

Thanks again!

1 Like