Implementing a pyclass containing Numpy arrays with Pyo3

[ckingsley]

Is it currently possible to return a Rust struct containing a numpy array to Python? Using Rust v1.51 and Pyo3 v0.13.1

A struct declaration like the following asks for a lifetime annotation for the PyArray:

#[pyclass]
pub struct PythonExternalStruct {
    #[pyo3(get)]
    pub pyarray: &PyArray<u32, Ix1>,
}

But adding a generic lifetime annotation to the struct gives a "#[pyclass] cannot have generic parameters" error. A static lifetime for the PyArray causes errors if I try to create an instance of the struct in the implementation of ToPyObject.

The issue with pyclass and lifetimes is known (How to wrap a struct with lifetime parameter? · Issue #502 · PyO3/pyo3 · GitHub) but wondering if anyone knows of a workaround. Or, is it only currently possible to do the conversion from Rust arrays/vectors to numpy on the Python side?

[RustyYato]

Don't use a reference, just store the data directly

#[pyclass]
pub struct PythonExternalStruct {
    #[pyo3(get)]
    pub pyarray: PyArray<u32, Ix1>,
}

If cloning is really expensive, then you could wrap the PyArray in an Arc

[mejrs]

@RustyYato

Don't use a reference, just store the data directly

You cannot ever instantiate this struct because there is no way to own a PyArray<...>...yes, PyO3's api is somewhat confusing here.

If you want to own one, you need a Py<PyArray> by wrapping it in the Py<...> smart pointer, like this:

#[pyclass]
pub struct PythonExternalStruct {
    #[pyo3(get)]
    pub pyarray: Py<PyArray<u32, Ix1>>
}

[ckingsley]

Wrapping the PyArray in a Py struct did the trick. Thanks!

BTW, when converting from a Rust vector to a PyArray, does it matter whether I use:
PyArray::from_vec(py, rust_vector).to_owned()
or
rust_vector.to_pyarray(py).to_owned()

The docs at numpy::array::PyArray - Rust indicate that the first approach using from_vec() doesn't allocate memory on the Python heap while the second approach using to_pyarray() does, but does the to_owned() method allocate memory and perform a deep copy?
I've tried both approaches and they both seem to work but I'm not sure what the advantage of one over the other is in this case.

thanks again

[mejrs]

does the to_owned() method allocate memory and perform a deep copy?

I don't really know how rust-numpy works, but that shouldn't be the case.

Python manages objects similarly to how Rc<...> works in Rust - nobody owns the inner object, cloning increments the reference count. Going from &PyXXX to Py<PyXXX> just increments the reference count.