Network service - threading help


#1

Hi!

As a first trial project for Rust, I wanted to implement a (simple) form of a network service used at my workplace. It is basically a key-value store for multiple clients.

Wanted functionality: many clients can connect to the server, which stores key-value pairs. Any client can store a new entry, and if the value is new or has changed, all other connected clients are notified. A client can also explicitly ask for a certain key.

The basic code structure I wanted to have is to have a server thread (the main thread), and two threads for each connection: one handles reads from the client, and one writes back replies and notifications. I eventually managed to get request-reply on a single connection working with a Channel (read thread -> write thread) per connection, and a shared DB struct that contains the key-value map and is wrapped in an Arc<Mutex<>> to be able to access it from any thread.

But I’m not sure how to implement the notify feature. It requires that one thread can send messages to all other connections, which means that it has to somehow have either a a reference to all other connection handlers, or to the server (which then is a cyclic reference, that I think is a bad idea).

Another idea would be to have more channels, but since channels apparently cannot be shared between threads I have to create N^2 channels, where N is the number of connections…

Anybody have a good idea how to restructure my objects to get rid of this dilemma? Thanks!


#2

Hmm. It seems like you should be able to use one mpsc channel for this, right? Create the channel when you start the server and hand out a sender to each child connection. That way, a child connection can communicate back with the main server thread when something changes.

But I guess you then need a channel for each child connection to send data from the main server thread to the connection threads, unless you can find a crate with a mpmc channel, in which case you should be able to do it with one.

So this method seems like you need N+1 channels. With a mpmc channel, you could do it with just 1 I think. Does that sound right?


#3

syncbox might have something useful, although the documentation is a bit lacking so I’m not sure what its exact semantics are.

http://carllerche.github.io/syncbox/syncbox/struct.LinkedQueue.html

http://carllerche.github.io/syncbox/syncbox/struct.ArrayQueue.html


#4

Thanks for the pointers. syncbox looks interesting. I think I’ll try having a central “sender” thread that writes to all connections, and then gets messages via a channel.

I also realized that I can make life a bit easier by doing try_clone on the socket; reading and writing through the same handle does not need the same Rust object. (Is “object” the right word?)

The only caveat is to close the connection properly; i.e. when the reading thread closes the connection, the sender thread has to realize that and drop its socket handler. Probably when an error is returned.

Otherwise I have to say, I’m liking Rust quite a bit. I dug into Haskell a while ago, and while its elegance is quite breathtaking at first, trying to write real-world code can quickly become either very ugly or very deep comp-sci, which is not my forte. With Rust, I recognize many concepts from Haskell, and i’m still able to express many things in the more “traditional”, imperative, style that I’m used to.