Cannot return a boxed future due to lifetime issues


#1

I am trying to write an async verison of my DB Client using the tokio ecoystem. However, I am running into issues regarding lifetimes, and I would appreciate any help provided. I have tried a ton of permutations to get around this, without any luck.

Given the following:

pub struct AsyncDbConnection {
    connection: TcpStream, // use tokio_core::net::TcpStream;
    db_name: String
}

And given the following function:

    pub fn execute_request<Q>(&mut self, query: Q) -> IoFuture<DbData> where Q: ExecutableQuery {
        let proto_query = query.create_query();
        let request_bytes = get_request_bytes(&proto_query);

        let first_req = tokio_io::io::write_all(&self.connection, request_bytes);
        // for simplicity, just returning empty vecs 
        let second_req = first_req.and_then(|_| { future::ok(DbData { rows: Vec::new(), columns: Vec::new() })});

        second_req.boxed()

When I try to compile, I get the following error messages:

Compiling db_driver v0.1.0
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
–> src/connection/async.rs:45:49
|
45 | let first_req = tokio_io::io::write_all(&self.connection, request_bytes);
| ^^^^^^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 38:100…
–> src/connection/async.rs:38:101
|
38 | pub fn execute_request(&mut self, query: Q) -> IoFuture where Q: ExecutableQuery {
| ^
note: …so that reference does not outlive borrowed content
–> src/connection/async.rs:45:49
|
45 | let first_req = tokio_io::io::write_all(&self.connection, request_bytes);
| ^^^^^^^^^^^^^^^^
= note: but, the lifetime must be valid for the static lifetime…
note: …so that the type futures::AndThen<tokio_io::io::WriteAll<&tokio_core::net::TcpStream, std::vec::Vec<u8>>, futures::FutureResult<Db2Data, std::io::Error>, [closure@src/connection/async.rs:52:45: 52:118]> will meet its required lifetime bounds
–> src/connection/async.rs:54:20
|
54 | second_req.boxed()
| ^^^^^

error: aborting due to previous error


#2

I think the problem lies in the fact that the contents of a Tokio IoFuture, as any Box-like type, must by default have a potentially infinite ('static) lifetime, whereas what you’re boxing into it here does not.

More precisely, in the definition of first_req, you’re building an asynchrounous request to write some bytes into a TcpStream, which is none other than a member of the AsyncDbConnection object (AsyncDbConnection::connection). So you take a reference to that stream.

But the TcpStream may not live forever. For all the Rust compiler knows, it will only live as long as the host AsyncDbConnection object.

And from the execute_request function, you’re returning a future object associated with your work on that connection object, and this future object in turn may potentially live forever. Which means in particular that it may live longer than the AsyncDbConnection object. Which would lead to use after free.

As the Rust compiler cannot prove that this catastrophic scenario cannot happen, and must prove memory safety at all costs, it writes off your code as illegal.

I’m not yet proficient with the lifetime syntax, but I think replacing the &mut self parameter of execute_request with an &'static muit self would make the code legal in an inelegant way. It would mean that the execute_request method can only be called on an AsyncDbConnection object with a static (infinite) lifetime.

A more elegant solution would be to somehow specify that the future output by your function cannot live longer than the host AsyncDbConnection, but I’m not sure how to do this. Maybe some variant of this would work:

 pub fn execute_request<'a, Q>(&'a mut self, query: Q) -> IoFuture<'a + DbData>

But again, I’m not proficient enough with the lifetime syntax or the Tokio API to guarantee that.


#3

I’m not that versed in tokio, but based on what I’m seeing here, I think you’re supposed to move TcpStream (or rather, possibly your AsyncDbConnection struct if you need the db_name value later) into the Future (tokio_io::io::write_all), and it will then hand it back to you as part of the (resolved) Future::Item that’s implemented for WriteAll: https://github.com/tokio-rs/tokio-io/blob/master/src/write_all.rs#L63. Otherwise, I don’t see how you can satisfy the 'static lifetime requirement and borrow self.connection at the same time; given the WriteAll future may outlive the current stack, you pretty much have to move the state here.

I could be wrong, of course :slight_smile:


#4

If you want to keep access to the AsyncDbConnection object, a variant of @vitalyd’s solution would be to box it into an Arc and move a clone of that Arc into the WriteAll future. This ensures that the connection object will live as long as required by both its “native” scope and the asynchronous operation represented by the future object.


#5

I doubt that will work because presumably (I’m on my phone now so can’t browse easily) you need exclusive (mut) access to the AsyncWrite, and Arc won’t give you that; would need to wrap in Arc and Mutex (or similar) probably.


#6

Ah, yes, you’re probably right.


#7

I think due to the nature of the tokio::io functions (that return the connection itself as one of the return value), I will just have to make the execute_request self instead of &mut self.

I can’t seem to figure out a different way.


#8

Maybe you can define execute_request as a free function that internally constructs the AsyncDbConnection. You can make it consume self, but that doesn’t quite feel right since it’s always going to be a “one-shot” type of value, at which point you may as well hide it from the caller.

Why do you want to “figure out a different way”? Is there something that bothers you about moving a TcpStream around?

If you share the broader context of what you’re doing, perhaps someone can suggest a better design. It’s a bit unclear on what you’re trying to do beyond the obvious snippet you posted.


#9

I do not mind moving it around, so I will do that. The current documentation on tokio is more geared towards servers and less so towards clients (although yes, there is some), so I was not sure exactly how a client should act. Thank you for your input.


#10

I’d imagine that a client functions similarly to a server in terms of the core concepts. The client sends a request to the server, and gets back a Future. The mechanics of sending a request are an implementation detail, but should be somewhat similar for a networked client - there’s a Core (event loop) on which Tokio can multiplex I/O (and other types, but let’s leave those aside here) operations. One such I/O operation is your db query.

The TcpStream is essentially a connected socket here. While a query is in-flight (i.e. Future is unresolved), the TcpStream is unavailable for further operations (you could probably build a TcpStream/connection pool to allow multiple independent conversations with the server, if such a thing is needed, but let’s leave that aside as well). If a client wants to send another request, you’ll probably want to return them a Future but buffer it somewhere (with some limit to apply backpressure when needed) until the TcpStream is available again (the first Future resolves).

At least that’s the direction I’d explore, but not being intimately familiar with Tokio, there may be other better ways to structure Tokio-based async client libs.