I’m learning Rust by writing a memcached clone. The architecture I have in mind is:
- a single storage layer (basically a hashmap)
- a single protocol layer with a driver that drives the storage (performs storage operations using Cmd/Resp structures)
- a transport layer instantiated thread-per-client (manages the socket)
- driver and transport communicate via immutable Cmd/Resp structures
I was then going to let the driver and the transports communicate over async channels. Feeding requests to the driver is easy because a channel can have multiple senders. But returning responses to the transports is turning out to be tricky, because the driver needs to have all the sender channel endpoints. That’s not a problem with all transports are started upfront, along with the driver, because then I can just create all the channels and give the right endpoint to the right thread.
But if I want to be able to launch transports on demand I’m also creating channels on the fly, and then I need to deliver the sender endpoint to the driver. I can’t figure out how, though. I can’t access the driver once it has been moved into its own thread. Nor can I send a channel over a channel.
Is this kind of dynamic allocation of channels possible?
I haven’t yet looked at other options like MetalIO or other channel implementations as I was trying to make sure I understand what’s available in the standard library.