CFFI down casting to Trait


#1

Using CFFi to wrap my Rust lib in Python (at the ABI level) I’m using Traits (at the library level) to define the application user API. Consequently, I also want to define as much as possible (…if possible indeed…) of the extern "C" interface at the library level, which obviously involves using the Trait types (only later to be implemented by the user application).

Say, with library code:

use T;
pub trait Trait {
   fn get(&self) -> T;
}

extern "C" fn model_get(mptr: *mut Trait) -> *const T {
    let m = unsafe {
        &mut *mptr
    };
    m.get()
}

and application code (in a different crate):

struct Model {...}
impl Trait for Model {...}

calling model_get with a with a Model self-object (through Python’s cffi) causes the following error:

TypeError: initializer for ctype 'Trait ’ must be a compatible pointer, not cdata 'Model '

If possible, what is the “cffi way” to ‘downcast’ a cdata _Model_ object to a ctype _Trait_ pointer? Thanks.


#2

Can you show the code for building the *Model?

Note that *mut Trait and *mut Model have different representations in memory. If you want to work on trait objects, then you shouldn’t be returning a *mut Model but instead a *mut Trait. e.g., Box::into_raw(Box::new(my_model) as Box<Trait>).

I think perhaps describing a bit more about the problem you’re trying to solve would help more. In particular, what is motivating the use of trait objects?


#3

I’m wondering if *mut Trait can be faithfully represented on the Python/C side at all. Would a struct with two pointers work?


#4

I assumed it would be treated as an opaque pointer.


#5

Library code:

use T;

pub trait Trait {
    fn get(&self) -> &T;
}

// Bindings

#[no_mangle]
pub extern "C" fn model_get(mptr: *mut Trait) -> *const T
 {
    let m = unsafe {
        &mut *mptr
    };
    m.get()
}

CFFI code:

try:
    from cffi import FFI
except ImportError:
    print "pip install cffi, included with PyPy"

ffi = FFI()
ffi.set_source("_trait",None)
ffi.cdef('''

typedef struct { int dummy; } Trait;
typedef struct { int dummy; } T;

const T* model_get(const Trait*);

''')

if __name__ == "__main__":
    ffi.compile()

Application code (Python):


def get(self):
    print("Lib:", self._lib, type(self._lib) )
    print("Obj:", self.obj,  type(self.obj) )
    return self._lib.model_get(self.obj)

with Python print output:

(Lib:, , )
(Obj:, , )

But then again, as @birkenfeld says, “I’m wondering…”


#6

I see. What about Box::into_raw(Box::new(Box::new(model) as Box<Trait>)).

Then your ffi function would have a *mut Box<Trait> parameter. Does that work?

(I’m spitballing here. No compiler.)


#7

I think it would work, yes.


#8

Obviously, this should be defined at the Rust application level, i.e. when Model (implementing the Trait) is defined. My (perhaps unrealistic…) point is if all the Rust API code can be defined at the library level, leaving the necessary casts at the Python CFFI level.


#9

I’m personally having a really difficult time groking the problem you’re trying to solve, but this may help clarify things: you’re using trait objects, and the representation of trait objects in memory is not stable, therefore, if they are part of your FFI, then they must be hidden behind an opaque pointer. I guess the only work around there is to define a C struct with representation identical to std::raw::TraitObject, but that seems like bad juju.

Otherwise, you should able to use: typedef struct trait trait and pass values of type *trait directly to model_get, assuming that values of type *trait are constructed with your Rust code. (But I don’t think you’ve shared that particular piece of code yet.)


#10

Thanks for all your help, much appreciated. Maybe I’m asking for the impossible considering that the “representation of trait objects in memory is not stable” I wouldn’t spent much further time on it.

Just for completion, since you say

here’s the application code side:

use T;

// #[derive(Debug,Clone,Copy)]
#[derive(Debug)]
pub struct Model {
    t : T,
}

impl Trait for Model {
    fn get(&self) -> &T {
        &self.t
    }
}

I think I’ll include the extern "C" definition into the application side eliminating the need for Trait types. Thanks again.


#11

OK, so to be more explicit, you have this FFI function:

pub extern "C" fn model_get(mptr: *mut Trait) -> *const T

Which presumably is called by Python somewhere, yes? How does the Python code get the mptr? In the code you’ve shown so far, this is a self.obj, but it doesn’t show how the value is defined or retrieved via the FFI.

I see a get method in your most recent comment, but that’s straight Rust code.

I would like to clarify that it should totally be possible to transport trait objects across an FFI, but you should probably be doing it with an opaque pointer. That’s all.


#12

Ah…, this probably completes it:

lib = ffi.dlopen("../../target/debug/libcx_app.so")
self.obj = lib.model_new(name)

where model_new is defined in the cffi file:

const Model* model_new  (const char*);

and in the Rust application code:

pub struct Model {
    t: T,
}

impl Model {
    pub fn new(name: &str) -> Model {
        Model {
            t: T::new(),
        }
    }
}

impl Trait for Model {
    fn get(&self) -> &T {
        &self.t
    }
}

// Bindings

#[no_mangle]
pub extern "C" fn model_new(name: *const libc::c_char) -> *const Model {
   // using a convenience 'cxstring' function
    let n = cxstring::c_char2string(name);
    let m = Box::new(Model::new(&n));
    Box::into_raw(m)
}

I guess your suggested

Box::into_raw(Box::new(Box::new(model) as Box))

should be substituted for Box::into_raw(m) in the extern "C" function.


#13

Yes, now you’ve changed the pointer type to *const Model, you don’t need the double boxing.

To summarize: *const Model is a “normal pointer” and can be passed to C as a pointer. *const Trait is a trait object pointer and cannot be passed to C, as its representation is unspecified. (The same holds for the *mut variants, btw.)

Therefore, as @BurntSushi wrote, when you want to pass a trait object you have to box the box, which gives you a “normal pointer” again.


#14

I’m almost there… Following your comments, with modified code

#[no_mangle]
// A '*const Model' can be passed into C.
pub extern "C" fn model_new(name: *const libc::c_char) -> *const Model {
    let n = cxstring::c_char2string(name);
    // Casting to 'Box' fails with: "error: wrong number of type arguments: expected 1, found 0 [E0243]"
    // let m = Model::new(&n) as Box;
    // Therefore:
    let m = Model::new(&n) as Box<Model>;
    // Don't need double boxing:
    // let t = Box::new(m);
    // Box::into_raw(Box::new(t))
    Box::into_raw(m)
}

… I receive the following error:

error: non-scalar cast: `cx::model::Model` as `Box`
... model.rs:105     let m = Model::new(&n) as Box;
                             ^___________________________

#15

You do need one Box::new().


#16

Indeed (sorry); this compiles:

#[no_mangle]
pub extern "C" fn model_new(name: *const libc::c_char) -> *const Model {
    let n = cxstring::c_char2string(name);
    let m = Model::new(&n);
    // Trivial cast:
    // let t = Box::new(m as Box<Model>);
    let t = Box::new(m);
    Box::into_raw(t)

However, calling it from Python gives the original error:

File "/home/cx/workspace/projects/CxABM/src/model/model.py", line 49, in get
return self._lib.model_get(self.obj)
TypeError: initializer for ctype 'ModelTrait *' must be a compatible pointer, not cdata 'Model *'

where ModelTrait is defined in the Python cffi file:

ffi.cdef('''
typedef struct { int dummy; } T;
typedef struct { int dummy; } ModelTrait;
const T* model_get(const ModelTrait*);

So the Model/Trait miss match still persists…
’’’)


#17

This doesn’t fit at all. You’re defining a model_new function in Rust, but calling model_get from Python?

Please show all relevant code. Put it into a github gist if it is too long for the forum.


#18

It does fit, let me explain. The extern “C” code is split over the Trait part and the Impl (i.e. application) part (both belonging to different crates/libraries).

Trait part:

CFFI (trait) code:

const T* model_get(const ModelTrait*)</code()>

refers to Rust (trait) code:

pub extern "C" fn model_get(mptr: *mut Trait) -> *const T {...}

and returns a T object that Model shall implement; hence the ModelTrait object – it is defined in the library code.

Impl part:

CFFI (impl) code:

const Model* model_new(name: *const …)

refers to Rust (impl) code:

pub extern "C" fn model_new(name: *const libc::c_char) -> *const Model {...}

and returns the Model object that implements the Trait – it is defined in the application code. In Python this is the self.obj object.

My perception is that you’re aiming at letting model_new return a Trait compatible pointer (stored in self.obj) that is acceptable to model_get… Hope this helps. (I might send/upload the complete example but it’s got some problem-irrelevant bells and whistles…).


#19

I can’t claim I understand it without more code, I can just repeat what we’ve been saying all along: you can’t have *mut Trait as an argument, or a return type, in a FFI function. It is not a simple pointer, and has an undefined layout. An in particular, you can’t cast it to *mut Model.


#20

Then we’ve reached a conclusion: the underlying assumption of CFFI type ModelTrait is that * mut Trait can be accessed as pointer. Apparently, using a * mut Trait argument or return type in an extern “C” function (for it to be generic) is not done. This doesn’t solve it but concludes it. Thanks for all your effort and patience.