Mio : TcpStream read/write on different thread

Hi,

I'm trying to use mio to receive event on my socket and to be able to close my application cleanly (avoid to read indefinitely on a read on the socket).

For now, my application is writing on the socket in a thread and reading on the socket on another thread. I added an event loop to read on the socket in the reading thread. But now, I can't write on the socket anymore. Write always returns a "WouldBlock" error.

Is it possible with mio TcpStream to write and read on different thread ?

Thanks

Why do you want to do this? An easier approach would be to have a single thread doing all I/O and then dispatch processing to a separate thread (if the processing is non-trivial). Connect the two threads with mpsc channels and you should be good to go.

1 Like

I want to do that because my client can send request but it can receive request as well (my client is a client AND a server for different requests). For now, I did exactly what you suggested, I added a channel between my 2 threads and transfer data that I want to send via that channel.

But without mio, I was doing send and receive on different thread. So I wanted to do the same thing. And one more thing, in my event loop, I register my stream only for readable (not writable) because writable is always raised, even if I have nothing to send and I guess it will use resource. So I have to reregister to add the writable interest and after the send, reregister to remove it. Should it work like this ?

Yeah, but you were (probably) doing that because you were using blocking IO and needed to dedicate threads to that. With an evented model, there shouldn’t be a need to split read/write like that.

Sounds about right. You should only have a writable interest registered if you have something to write, otherwise it’s removed from the interest set.

Another question - any reason you’re using mio instead of tokio?

I understand your point. You're right, with a blocking IO, I had no choice, I had to write on a different thread. But with mio, I didn't want to add complexity : a channel between threads, a queue to stack all messages to send, register/unregister writable event... I thought to be able to send messages directly from my first thread. But if it is not possible, fine, I will do what is needed.

Not really. For now, my application is using std::net::TcpStream and it is blocking IO. What I want now is to close my application cleanly. I didn't find a way to stop my threads since they are blocked on a read operation (read on a socket or read on a receiver, but both are blocking). Since I can't notify threads to stop, I searched a way to have events. I want to be able to set a "cancel event" in my first thread and wait the end of the other thread. So I found mio where I could define my own event with Registration/SetRediness. So I started to try that and working with that.

I guess you have already understook that I'm beginning with Rust :wink: Why should I use tokio instead of mio ? What is the difference ? I red that tokio is built on top of mio but since mio seems to have what I need, I didn't look further. What would be your recommendation ? Thanks for your help !

I don’t think you need a separate stack to buffer messages - the channel itself would serve that purpose.

It might be possible to get mio to work as you want by dup'ing the file descriptor of the socket and then having one of these descriptors attached to the event loop and the other for your writes. Not sure if/how well that would work out though - it would be a weird setup.

Going straight into mio (or tokio for that matter) while learning Rust is probably pretty adventurous :slight_smile:.

So tokio would raise the abstraction level for you - instead of worrying about blocking, interest (de|re)registration, edge vs level triggering and so on, you instead set up chains of futures that describe your processing pipeline. Tokio really is a complex topic on its own, and I won’t do it justice in a single post, but I urge you to look at its docs and examples to get a feel for it. You’ll likely have questions so feel free to come back to this forum with them.

You're right, using the channel directly should be fine. Thanks

Do you mean by duplicating the stream itself (mio::net::TcpStream) ? I tried to duplicate it and use one in my event loop and one in my other thread where I want to send, but the send always returned WouldBlock. I didn't find a fix to that other than sending in the same thread as the event loop when I receive an event writable on the socket.

And I will take a look to tokio, thank you !

I was thinking using something like nix::unistd::dup2 - Rust to get a RawFd and then using mio::net::TcpStream's impl of FromRawFd to construct a separate instance of it for the writes. I'm not entirely certain whether that would work though.