TcpStream BufReader + BufWriter lifetime issue


#1

I have program, where I need to read and write from a buffered TcpStream (through BufReader and BufWriter) from different places in the program. Hence, I tried to store the Reader and the Writer in the fields of a struct (very simplified example to make my point clear):

use std::net::TcpStream;
use std::io::{BufReader, BufWriter};

struct TestTcp {
    raw_stream: TcpStream,
    reader: BufReader<&TcpStream>,
    writer: BufWriter<&TcpStream>,
}

impl TestTcp {
    fn new(addr: &str) -> TestTcp {
        let stream = TcpStream::connect(addr).unwrap();
        let reader = BufReader::new(&stream);
        let writer = BufWriter::new(&stream);
        TestTcp {
            raw_stream: stream,
            reader: reader,
            writer: writer,
        }
    }
}

Note: I need to use references, since otherwise the BufReader would consume the stream and I can’t initialize the BufWriter afterwards.
Here, the compiler complains about missing lifetime specifiers.

So, I tried the following:

use std::net::TcpStream;
use std::io::{BufReader, BufWriter};

struct TestTcp<'a> {
    raw_stream: TcpStream,
    reader: BufReader<&'a TcpStream>,
    writer: BufWriter<&'a TcpStream>,
}

impl <'a>TestTcp<'a> {
    fn new(addr: &str) -> TestTcp<'a> {
        let stream = TcpStream::connect(addr).unwrap();
        let reader = BufReader::new(&stream);
        let writer = BufWriter::new(&stream);
        TestTcp {
            raw_stream: stream,
            reader: reader,
            writer: writer,
        }
    }
}

Now, the compiler complains that &stream does not live long enough - which makes sense, because we are initializing from a local variable.

Next try:

use std::net::TcpStream;
use std::io::{BufReader, BufWriter};

struct TestTcp<'a> {
    raw_stream: TcpStream,
    reader: Option<BufReader<&'a TcpStream>>,
    writer: Option<BufWriter<&'a TcpStream>>,
}

impl <'a>TestTcp<'a> {
    fn new(addr: &str) -> TestTcp<'a> {
        let stream = TcpStream::connect(addr).unwrap();
        let mut result = TestTcp {
            raw_stream: stream,
            reader: None,
            writer: None,
        };
        result.initialize_streams();
        result
    }

    fn initialize_streams(&mut self) {
        self.reader = Some(BufReader::new(&self.raw_stream));
        self.writer = Some(BufWriter::new(&self.raw_stream));
    }
}

Now, the compiler says: “cannot infer an appropriate lifetime for borrow expression due to conflicting requirements”…
My intuition is the following: since the references point to a struct’s field (and these have to live as long as the struct itself), the borrow should be valid. Is my reasoning correct? And if yes, is there some kind of hint I could give to the compiler to figure it out? Or some other kind of workaround?
It is not the first time, I ran into these kind of problems with references and lifetimes. So perhaps, someone has some general strategy or “best practice” advice to deal with these issues.


#2

I’m afraid you are trying to build a self-borrowing struct, which Rust currently does not handle well. As you have noticed, there is no syntax for telling the compiler that TestTcp::reader and TestTcp::writer borrow from TestTcp::raw_stream.

Typical workarounds revolve around storing the borrowed object on the heap and accessing it via an Rc or Arc. But fortunately, there is a cleaner option in your case: using TcpStream::try_clone, you can spawn a second handle to your TCP stream. You will then have enough handles to build both your BufReader and BufWriter, passing the clones by value.


#3

But in

fn initialize_streams(&mut self) {
   self.reader = Some(BufReader::new(&self.raw_stream));
   self.writer = Some(BufWriter::new(&self.raw_stream));
}

I am telling the compiler, that the reference points to a local field, isn’t it? Or do you mean something different?

Ah, great - I’ll try that out. Thx!


#4

When reasoning about Rust generics and lifetimes, I find it very helpful to keep in mind is that every generic parameter of a struct or method is provided by its user, and not by its implementor.

This is true, in particular, of the 'a lifetime parameter of the TestTcp struct in your code examples. Let’s have a look at the second version, since it is simpler and the third version does not really resolve its issues…

use std::net::TcpStream;
use std::io::{BufReader, BufWriter};

struct TestTcp<'a> {
    raw_stream: TcpStream,
    reader: BufReader<&'a TcpStream>,
    writer: BufWriter<&'a TcpStream>,
}

impl <'a>TestTcp<'a> {
    fn new(addr: &str) -> TestTcp<'a> {
        let stream = TcpStream::connect(addr).unwrap();
        let reader = BufReader::new(&stream);
        let writer = BufWriter::new(&stream);
        TestTcp {
            raw_stream: stream,
            reader: reader,
            writer: writer,
        }
    }
}

In this code, you are telling the compiler that for any lifetime 'a of the TestTcp user’s choosing, the inner references of the TestTcp struct must be valid. The compiler will analyze the code, and figure out that this is incorrect: the references are only valid for the lifetime of TestTcp’s raw_stream member, which is not infinite and may not match any choice of 'a from the user. So the code will be rejected by the borrow checker, with a “does not live long enough” message.

To resolve this, we would need to do either of two things:

  1. Modify the definition of TestTcp::new so that it forces the user to pick the right lifetime (impl Trait style)
  2. Modify the definition of TestTcp so that the lifetime is not user-provided

Most proposals which I have seen so far revolve around the second option. One of them is to introduce a new special lifetime specifier, 'self, which refers to the enclosing struct:

struct TestTcp {
    raw_stream: TcpStream,
    reader: BufReader<&'self TcpStream>,
    writer: BufWriter<&'self TcpStream>,
}

Another, notably used by the rental crate, is to be more specific and indicate which struct member is targeted by the reference:

struct TestTcp {
    raw_stream: TcpStream,
    reader: BufReader<&'raw_stream TcpStream>,
    writer: BufWriter<&'raw_stream TcpStream>,
}

In any case, lack of self-borrow support is a well-known issue, and the compiler and language teams have expressed multiple times that they are quite interested in tackling it after more pressing concerns such as non-lexical lifetimes have been taken care of.


#5

@HadrienG: Thanks for the great explanation. Yes, I also searched for something like &'self - both proposals seems worth considering in my opinion. The concept looks pretty similar to being able to use types defined within a structure (as when implementing a trait) via Self::mytype.

Is this possible already or just a proposal? Would be nice to see some example code as to grasp the idea.


#6

Come to think of it, I’m not sure if someone has actually proposed this variant. It just sprang to my mind as another possible solution to the problem while I was trying to enumerate what I’d already seen.

In practice, it would anyway probably be a bad idea, because the lifetimes which a function manipulates would not be easy to infer from its signature anymore. So implementation changes could break caller code, something which Rust is explicitly designed against.