Tips for tokio ipc

I am working on a project that has 2 binaries in 1 workspace. Initially, both binaries will be running on the same machine, but I can imagine a future where that's not the case.

One of the binaries is a Discord bot for which I am using an async framework (serenity). Whenever a user interacts with the bot using a command, I need to communicate with the other binary.

Currently, I am using tmq, which is tokio-fied ZeroMQ. There is one issue with that crate: the server/client messaging is not Sync. So, I can't just spawn a tokio thread and wait for IPC messages from the other binary in there.

I worked around that by having a separate tokio thread with a LocalSet that communicates with the main thread using an unbounded channel.

As both binaries run async code, I find myself in some sort of channel hell. For one command in Discord where I need to get information from the other binary, I have to create a lot of channel communication details, like adding message enum variants, awaiting response messages, etc.

It gets convoluted very quickly and I was wondering if there is a better way.

What I ideally want is some form of bi-directional, client/server IPC channel between these two binaries, for which the server can run in it's own tokio thread and where I can send messages to from the client (Discord bot).

(Another requirement (which made me choose ZMQ in the first place) is that sometimes a message is published from the other binary to the Discord bot, where the Discord bot is a subscriber to and subsequently posts a message in a Discord channel. I can keep using ZMQ for this, though, but maybe it could be replaced with something better)

Does something like that exist? Or am I thinking too optimistic and do I simply need a lot of boilerplate to get IPC working?

Found out about NATS. Gonna give that a go, it seems ok enough for now.

Yes, Remoc has been developed exactly for providing Tokio-like channels between different processes, possibly running on different hosts.

See https://crates.io/crates/remoc.

2 Likes

I have looked around but didn't come across Remoc. It certainly seems exactly what I need!

Open an issue in the repo if you need some help. We are using it in production and it is working fine. However, I am aware that we are lacking examples and tutorials.

If you find it useful, would you be willing to publish your Discord bot as an real-world usage example of Remoc?

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.