Rust and Python - exchange commands and replies over the network

Is there a well established framework for communicating over the network between Rust and Python?

I have a device controlled from the network. There's a Rust server on the device (aarch64-unknown-linux-musl) that receives some Command from a TcpStream, does some work, and then sends back a Reply. On the controlling computer side, there's a thin Python program that sends these Commands to the device and waits for an answer.

Right now, I have a working solution using bincode and serde-generate. I define Command and Reply in Rust like so:

use serde::{Deserialize, Serialize};

#[derive(Debug, Clone, Serialize, Deserialize)]
enum Command {
    Version,                 // -> Reply::Str  or Reply::Error
    Sleep(f64),              // -> Reply::None or Reply::Error
    SetFrequency(f64),       // -> Reply::None or Reply::Error
    GetFrequency,            // -> Reply::Int  or Reply::Error
    Update(String, Vec<u8>), // -> Reply::None or Reply::Error
    // .. and more ...
}

#[derive(Debug, Clone, Serialize, Deserialize)]
enum Reply {
    None,
    Error,
    Bytes(Vec<u8>),
    Int(u64),
    Str(String),
    VecStr(Vec<String>),
    Bool(bool),
}

Then I use serde-generate to generate Python class definitions with type hints and (de)serialization code using the bincode format.

While this approach works, each Command variant is not tied to a particular Reply variant, so there's no static checking that things are set up the way they should be. Ideally, I would like a system to write down the definitions of Command, Reply and their relationship once, and then generate code for Rust and Python. Rust would keep everything in check with the type system, and Python would limp along with type hints.

Is this a common-enough use case that there is an already-established framework/library for it?

1 Like

You're looking for an RPC framework, which builds on top of serialization to add exactly the typed request-reply pairs you want. I don't have much personal experience with this area, but I hear Rust people talking about using gRPC and Cap'n Proto, and you should be able to find one or more Rust and Python libraries for each.

5 Likes

Thanks!

On the Rust side, tonic seems the most popular solution for gRPC, while for Cap'n Proto there is capnpc and friends. From a first look, it seems like either of them would address my needs, but...

Examples for both libraries are quite async heavy, which is something I haven't worked with yet. I couldn't find any example using a "simpler" blocking (non-async) workflow. Is RPC inherently tied to async?

1 Like

Again, I'm not familiar with those libraries, but broadly: async is valuable when you want to do things like issuing multiple RPCs simultaneously (to minimize total latency vs. doing them sequentially) or having complex policies about timeout, retry, or "stop trying to do this because it's obsoleted by a new user action". On the server side, it's an efficient way to serve very large throughput (measured by request rate, not bandwidth).

But you don't have to program in an async style if you don't want to; you can always use block_on() and spawn_blocking() or other such operations to adapt async interfaces to writing synchronous code.

4 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.