I have been working on a series of attempts to restructure the Python gRPC client of a specific toolkit to use Rust’s tonic while idl was generated by buf prost plugin, and then exposed Python bindings through PyO3. However, as a complete Rust beginner, I am seeking guidance and would greatly appreciate any experienced insights.
I’m dissatisfied with the current approach of communication between Rust and Python through bytes conversion due to the maintenance cost and unnecessary CPU overhead from serialization. I hope to eventually remove Python’s protobuf along with gRPC.
Currently, I am stuck on how to make code-generated Rust code of the IDLs that can directly bind to Python using PyO3.
My basic understanding of Rust is that the orphan rule stipulates that we can only implement either local traits or local types within our local crate. This means we can’t implement traits from another crate for types defined in a different crate. Possible solutions might include:
1. Defining local structures as wrapper types to contain the existing external structures from external crates, which are from my code-gened Rust IDL by protos.
2. Import the PyO3 crates while generating the IDL and somehow adding #[pyclass] macro in PyO3 to every type (may be nested or enum or using another external well-known prost_types).
Potential issues include the generation of a lot of handwritten bootstrap code, such as into and from implementations, or dealing with enums and prost types from another crate. Additionally, I need to ensure that every nested member field can be accessed in the Python environment.
As a Rust newbie, I am not sure if there are better approaches. Any suggestions or insights would be greatly appreciated.