Golang <-> rust interop: protobuf?

1

For doing golang / rust interop over a tcp connection, is protobuf the way to go?

If not, what else should I look at?

2

I would say this doesn't primarily depend on the languages involved, but on the kind, complexity, and amount of data you will need to transmit.

1 Like
3

The choice of language isn't too relevant here because TCP makes the problem mostly platform-agnostic. What you are really asking is "I've got a TCP connection, how can two programs use it to communicate?", which is a very big, question with no one-size-fits-all solution.

To get something more useful than "it depends", you would need to answer a couple questions:

  • is communication bidirectional?
  • is there one side which is obviously a client and another which is obviously a server, or do they have more of a peer-to-peer relationship?
  • are there multiple types of messages which must be handled differently?
  • will this communication take a request-response form or be something more complex like RPC with streaming?
  • do I know the form of these messages beforehand?

Once you have a better understanding for the problem you can start looking at the possible solutions and decide which works best for you. Some possible solutions are:

  • roll your own

    • this is the most fun!
    • the sky's the limit
    • you need to think about how to split the stream of bytes that is a TCP stream into discrete messages
    • you need to decide how a message is converted back and forth between Rust/Go types and bytes (JSON, Protobuf, hand-rolled serialisation code, etc.)
    • each message needs some sort of ID so you know which code to call to handle it
    • you need to decide how traffic will flow between the two ends
    • many books have been written about this

  • HTTP

    • you get routing and packets for free
    • there are a million HTTP frameworks out there with loads of documentation
    • you still need to choose the format for the messages... Most people just serialize to JSON
    • strictly request-response
    • you need to look at the code to know which routes accept what messages
    • you have HTTP status codes for error handling
    • easy to debug using MITM tools or postman or curl or whatever

  • gRPC

    • You get routing, packets, and message serialization for free
    • the entire communications layer is specified in a *.proto file and you can generate clients or servers for most languages
    • you can have more complex flows than request-response, with things like client-side, server-side, or bidirectional streaming
    • it's literally got "remote procedural call" in the name, which is the idea that I can have an object on one computer and when I call a method that'll actually invoke code on another machine and send the result back, all without me knowing/caring about the networking going on under the hood

I personally prefer gRPC, or some alternative RPC framework, because all I need to do is write down how my two sides will communicate then run a code generator and implement some trait/interface for the server. It means I spend less time thinking about how the two sides will communicate and more time on the business logic.

4 Likes
4

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.