Dealing with closing of a framed split Tokio connection

Hi, I have a server that obtains Tokio TCP connections. At some point the TcpStream is split into a writer and a reader. I am not sure about the behaviour expected when the connection is closed.

For example, assume that a new connection is split as follows:

let (writer, reader) = stream.framed(Codec::default()).split();

I assume that if the connection was closed and I try to write through writer, something like:

await!(writer.send(message))

an error will occur.

I also assume that if the connection was closed and all the data from the receive buffer was read,

await!(reader.next())

should return None at some point if the connection is closed.

My question is this: If I attempted to send a message using writer.send(), am I guaranteed to get a None result from a future await!(reader.next()) invocation?

Extra context for this question

                         +-----------------------------+
                         +--------+                    | Disconnect notification
   +---------------------|-----+  | Disconnect notification                  
   |              events |     |  |                    |
   |                     |     |  |   client1 handler  |  client2 handler 
   | Client senders:     v     |  |  +----------+      | +----------+     
   | - sender1                 |  |  |          |      | |          |     
   | - sender2                 |  +----Receiver |      +---Receiver |     
   |                           |     |          |        |          |     
   |                           |     |          |        |          |     
   +---------------------------+     +----------+        +----------+     
     Server loop

I am writing a server that deals with various events in a loop.
The server keeps a map of connected clients, and one event that should be handled is the disconnection of a client.

pub struct ChannelerState<A> {
    friends: HashMap<PublicKey, ConnectedFriend>,
    // ...
}

In case of a failed writer.send() attempt a client is removed. However, receiving messages from the client is done in a separate task. Therefore it is expected that a None will be returned from a reader.recv() call in the client handling loop. Eventually this event arrives back to the main server loop, notifying about disconnection of the client.

The server has two different sources for the disconnection event: one in the case of attempting to send a message (which happens in the server loop), and one in the case of attempting to receive a message (which originates outside the server loop, but eventually arrives at the server loop).

One can claim that there is no problem here. If a disconnection even arrives and the specific client is not present in the clients map, we can ignore the disconnection event and do nothing. (This will happen if first writer.send() failed, and then later reader.next() returned None, which sent a disconnection event to the server loop).

However, it is possible that by the time the disconnection event from the reader.next() arrives, the client has connected again. The disconnection event from the previous round will cause the new connection to be destroyed, which is a bug.

One idea I used for solving this issue is using generations for the connected clients. Every connected client is also given a generation number (u64).

pub struct ConnectedFriend {
    sender: mpsc::Sender<Vec<u8>>,
    generation: u64,
}

This value increases every time a new client is received, and is somewhat unlikely to repeat itself. Whenever a reader.next() returns None, a disconnection even is created, and it contains the client’s generation. This mitigates the problem described above: If the client disconnected and then connected again, and an old disconnection event arrives, the new client will ignore the old disconnection event, because the generation value at the disconnection event will not match the generation value of the new client.

I hoped to be able to simplify the solution by having a structure for the client as following:

pub struct ConnectedFriend {
    opt_sender: Option<mpsc::Sender<Vec<u8>>>,
}

If the sender ever fails, we remove it by setting connected_friend.opt_sender = None;. Subsequent sending attempts will be ignored. We wait for reader.next() to return None, and only at that point we remove the ConnectedFriend from the map.

My question is this: Can I be sure that eventually reader.next() will return None, even after a failed send attempt?

EDIT: If it is of any help, the real code is here:


It’s not finished yet. It contains an initial implementation for the generation values based solution.

This depends - do you anticipate (or plan to handle) clients in a half open state? That is, a client has shut down its input (your writer will detect this) but not its output (your reader will continue waiting for data).

@vitalyd: Thank you for your reply. I forgot about the case of half open state connection!
The connections I have are eventually proxied through a relay:

A -- Relay -- B

The relay should close the connection if anything fails. In that case I think that I can go with the simpler solution of having opt_sender: Option<mpsc::Sender<...>>, and empty it in case a send error occurs. Then wait for the receiver part to report that the connection was closed