Wrap Rust Library for Python (without changing Rust library)

help
1

Coming from C++ I would implement my C++ library (classes, structs, functions) in pure C++ and as a second step I would use pybind11 to wrap them so that they can be used in Python. For example:

// Vector3.hpp
struct Vector3 {
    Vector3(int x, int y, int z)
        : x(x), y(y), z(z) { }

    double length() const {
        return std::sqrt(x * x + y * y + z * z);
    }

    int x, y, z;
};

// PyBindings.cpp
#include <pybind11.h>
#include <Vector3.hpp>

namespace py = pybind11;

PYBIND11_MODULE(PyBindSample, m) {
    py::class_<Vector3>(m, "Vector3")
        .def(py::init<int, int, int>(), py::arg("x"), py::arg("y"), py::arg("z"))
        .def("length",      &Vector3::length)
        .def_readwrite("x", &Vector3::x)
        .def_readwrite("y", &Vector3::y)
        .def_readwrite("z", &Vector3::z);
}

Pybind's .def essentially points to a C++ class or function, meaning the underlying C++ library can exist without ever knowing about the Python bindings.

Looking at the blog post Calling Rust from Python using PyO3 it seems the C++ code above would look like this in Rust:

#[pyclass]
pub struct Vector3 {
    #[pyo3(get, set)]
    pub x: i32,
    #[pyo3(get, set)]
    pub y: i32,
    #[pyo3(get, set)]
    pub z: i32
}
#[pymethods]
impl Vector3 {
    #[new]
    pub fn new(x: i32, y: i32, z: i32) -> Vector3 {
        Vector3 { x, y, z }
    }
    pub fn length(&self) -> f64 {
        ((self.x*self.x + self.y*self.y + self.z*self.z) as f64).sqrt()
    }
}
#[pymodule]
fn rust(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<Vector3>()?;
    Ok(())
}

Here I had to insert multiple PyO3 annotations like #[pyo3(get, set)] into my Rust library. Is it possible to wrap an existing Rust library into Python without having to change any code within the underlying Rust library, similar to pybind's approach in C++ (where library & bindings are separated)? Perhaps PyO3 is not the right tool? If not, are there other tools that mimic pybind's approach?

3 Likes
2

Assuming that Vector3 is the type you don't want to change, I think you can wrap it into a newtype and implement the methods on the newtype. There might be a better way though.

1 Like
3

The way I would typically approach this is the same as what you did in C++... Introduce a new crate which wraps the underlying crate and exposes the corresponding pyo3 bindings.

2 Likes
4

Do you mind me asking how? Taking Vector3 as an example.

Original Library:

// vector3.rs
pub struct Vector3 {
    pub x: i32,
    pub y: i32,
    pub z: i32
}

impl Vector3 {
    pub fn new(x: i32, y: i32, z: i32) -> Vector3 {
        Vector3 { x, y, z }
    }
    pub fn length(&self) -> f64 {
        ((self.x*self.x + self.y*self.y + self.z*self.z) as f64).sqrt()
    }
    pub fn add(&self, other: &Vector3) -> Vector3 {
        Vector3 { x: self.x + other.x, y: self.y + other.y, z: self.z + other.z }
    }
}

Bindings:

// py_vector3.rs
use pyo3::prelude::*;
use crate::vector3;

#[pyclass]
pub struct Vector3 {
    pub obj: vector3::Vector3
}

#[pymethods]
impl Vector3 {
    #[new]
    pub fn new(x: i32, y: i32, z: i32) -> Vector3 {
        Vector3{ obj: vector3::Vector3{ x, y, z } }
    }
    pub fn length(&self) -> f64 {
        self.obj.length()
    }
    pub fn add(&self, other: &Vector3) -> Vector3 {
        Vector3{ obj: self.obj.add(&other.obj) }
    }
    pub fn x(&self) -> i32 {
        self.obj.x
    }
    pub fn y(&self) -> i32 {
        self.obj.y
    }
    pub fn z(&self) -> i32 {
        self.obj.z
    }
}
#[pymodule]
fn my_rust(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<Vector3>()?;
    Ok(())
}

Now, Python's Vector3 holds an instance of Rust's vector3::Vector3. This seems ok, but when we get to defining the functions, it's quite a big amount of boilerplate (almost re-implementing all the original functions). This is very different to the C++ pybind approach, where we don't have to create any dummy structs and don't need to implement any functions (we simply point to the original ones). Is something similar possible with PyO3?

5

You can use pyo3 as an optional dependency, and only enable it when building the extension module. Eg:

#[cfg_attr(feature= "pyo3", pyo3::pyclass)]
pub struct Vector3 {
    pub x: i32,
    pub y: i32,
    pub z: i32
}

// Methods that are only available when building the extension module.
#[cfg_attr(feature= "pyo3", pyo3::pymethods)]
impl Vector3 {
    #[new]
    pub fn new(x: i32, y: i32, z: i32) -> Vector3 {
        Vector3 { x, y, z }
   }
   
   // ... getters and setters for Vector3's fields here

}

// Methods accessible to both Rust and Python
impl Vector3 {
    pub fn length(&self) -> f64 {
        ((self.x*self.x + self.y*self.y + self.z*self.z) as f64).sqrt()
    }
}
#[#[cfg_attr(feature= "pyo3", pyo3::pymodule)]]
fn rust(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<Vector3>()?;
    Ok(())
}

Note that there's a bit of boiler plate because of cfg's and those field/method attributes not playing nice (see `pyo3(get, set)` attribute not found when using `cfg_attr` · Issue #1003 · PyO3/pyo3 · GitHub and `#[new]` doesn't play nice with `cfg_attr` · Issue #780 · PyO3/pyo3 · GitHub ). I recommend dealing with those issues by just writing some getters and setters.

6

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.