Best practices for conversion between types defined in external crates

Hello,

First of all, I apologize if this is noise for most of you. I am new to rust but used c++ a fair amount ~5y ago. I have also never posted on any such forum, so any constructive feedback on how I pose my questions is greatly appreciated.

I am working on a project that requires conversion between types defined in two external crates. My specific example is nalgebra::DMatrix<f64> to ndarray::Array2<f64>, but the problem itself is quite a general one and it brings up the issue of the "orphan rule". Having had some past experience with messy inheritance in c++, I think I understand the rationale behind this rule, but it seems to create some unnecessary fluff. I followed the suggestion from the rust documentation of using a thin wrapper (e.g. a tuple struct) and implementing Deref trait. However,Deref has some serious limitations in terms of which method calls will actually be directly available (see this discussion on "receivers", etc), not to mention a general tendency to hide things that could be made more explicit/plain.

Here is an example of something that potentially "works":
Cargo.toml

name = "rs-nalgebra-to-ndarray-test"
version = "0.1.0"
edition = "2021"

[dependencies]
nalgebra = { version = "0.32" } 
ndarray = { version = "0.15" }

src/lib

pub mod matrix {
    // per the book, make thin wrapper (eg tuple struct)
    pub struct Matrix(nalgebra::DMatrix<f64>);

    // per the book, get access to underlying methods by implementing Deref
    // n.b. not usually a great idea; obfuscates/complicates things later
    impl std::ops::Deref for Matrix {
        type Target = nalgebra::DMatrix<f64>;

        fn deref(&self) -> &Self::Target {
            &self.0
        }
    }

    impl std::convert::From<nalgebra::DMatrix<f64>> for Matrix {
        fn from(value: nalgebra::DMatrix<f64>) -> Self {
            Self(value)
        }
    }

    impl std::convert::From<Matrix> for ndarray::Array2<f64> {
        fn from(value: Matrix) -> Self {
            let shape = value.shape();
            let data: std::vec::Vec<f64> = value.0.data.into();
            unsafe { Self::from_shape_vec_unchecked(shape, data) }
        }
    }
}

#[cfg(test)]
mod tests {
    use crate::matrix::Matrix;

    #[test]
    fn it_works() {
        let ex_nalg = nalgebra::DMatrix::<f64>::zeros(3, 2);
        println!("nalgebra matrix: {ex_nalg:?}");

        // the below will not compile for obvious reasons:
        // let try_ndarr: ndarray::Array2<f64> = ex_nalg.into();

        // so, left with trying to impl my own conversion on wrapped version
        // unfortunately, Deref is only called when of form obj.method(), not obj=Struct::method()...
        // so need trivial conversion to  wrapper first
        let ex_mat: Matrix = ex_nalg.into();

        // finally try to convert to ndarray
        let ex_ndarr: ndarray::Array2<f64> = ex_mat.into();
        println!("ndarray matrix: {ex_ndarr:?}");
    }
}

This seems to be an overly-complex solution to a problem that must be very common, so I think I have just missed something fundamental. Even though this "works", to get access to all the methods I want from nalgebra::DMatrix<T>, I have to call the innards of this wrapper rather than using the wrapper itself. Obviously, that is not the end of the world, but it is another thing that makes me think I am missing something.

Thanks in advance for any guidance on these issues.

Best,
MFB

You could use the derive_more crate to derive the From and Deref trait implementations and thus reduce the boilerplate. Other than that, I don't think you can do anything else to simplify type conversions.

1 Like

Are there invariants you're preserving in Matrix that don't apply to DMatrix<f64>? If not, I'd just make the field public

    pub struct Matrix(pub nalgebra::DMatrix<f64>);

and call the test like so

        let ex_ndarr: ndarray::Array2<f64> = Matrix(ex_nalg).into();

But even without the public field, you can

        let ex_ndarr: ndarray::Array2<f64> = Matrix::from(ex_nalg).into();

I'm not sure what role you feel the method receiver / function argument coercion play in this example. From and Into both operate on receivers only.

Moreover, function arguments don't get autoref'd, but they do get deref coerced. So this works.

    fn some_dmatrix_taking_thing(_: &nalgebra::DMatrix<f64>) {}
// ...
        let matrix = Matrix::from(ex_nalg.clone());
        some_dmatrix_taking_thing(&matrix);

@moy2010 , thanks very much for the suggestion! That will definitely help reduce some boilerplate.

@quinedot , Thanks for taking the time to even type out an example!

To answer your first question, no, there was no reason I did not make the field public. Thanks for spotting that!

I do like the from --> into one-liner. That condenses things, so thank you.

But I am realizing I did not explain why I mentioned coercion. The issue comes up when I try to call methods of the wrapped object when using the wrapper itself. For example, I would ideally call methods like matmul between wrapped instances of the matrix like this:

        let matrix_1 = Matrix::from(nalgebra::DMatrix::<f64>::from_element(3, 2, 1.0));
        let matrix_2 = Matrix::from(nalgebra::DMatrix::<f64>::from_element(3,3, 1.0));
        
        // the below will not compile
        // let matrix_prod = matrix_2 * matrix_1;
        
        // whereas the following obviously will
        let matrix_prod = matrix_2.0 * matrix_1.0;
        println!("product of matrices: {matrix_prod:?}");

see Permalink to the playground

I realize this is essentially asking for inheritance, which seems to be a no-no. But does it at least make more sense what I am asking?

In any case, the obvious answer here is to use the nalgebra type to do everything I need it for and invoke that from --> into one-liner you mentioned. It just seems like it creates a throw-away type definition that contributes no functionality in cases like this where I do not need to implement other traits of my own on top.

MFB

MFB

It's true Deref won't get you everything (it won't get you methods that take by value either). For those cases you can implement the desired traits (and methods).

(It can be a lot of boiler-plate.)

1 Like

Thanks, @quinedot . This is basically what I was on my way to doing but thought it seemed like a lot of boilerplate for a use-case people surely ran into often. Glad to know I was not just missing something fundamental from the docs.

I would just implement a function:

fn dmatrix_to_array2(dmatrix: DMatrix<f64>) -> Array2<f64>
1 Like

@tczajka I agree, a single function would avoid all the boilerplate of this particular example. Thank you! In general, I was just trying to adopt what I thought seemed to be the conventional "rust way" of using From/Into trait implementations.