Passing a trait object through an FFI as user_data

#1

I am struggling with passing a callback through a FFI, to a library
that I don’t control. The Rust version of the C interface looks like
this, generated by bindgen:

extern "C" {
    pub fn raptor_parser_set_statement_handler(
        parser: *mut raptor_parser,
        user_data: *mut ::std::os::raw::c_void,
        handler: raptor_statement_handler,
    );
}

The library expects a handler that looks like this:

void
  statement_handler(void* user_data, const raptor_statement* statement)
  {
    /* do something with the statement */
  }

So far I have a function like this as the statement handler.

extern "C" fn statement_handler(user_data:*mut c_void,
                                statement: *mut raptor_statement){

}

My thought was to pass a pointer to an object through user_data; this
seems to be the approach that the rustonomicon suggests:

https://doc.rust-lang.org/nomicon/ffi.html#targeting-callbacks-to-rust-objects

I have managed to achieve this with my own Parser struct, using this
piece of hieroglyphics.

let parser_ptr: *mut c_void = &mut parser as *mut _ as *mut c_void;

let parser:&mut Parser = &mut *(user_data as *mut Parser);

This actually (seemed) to work. But I really want to pass a
pointer to a ParserHandler which is a trait. Then, client users of
my library can implement the ParserHandler. ParserHandler will be
called multiple times.

I have found a wide variety of solutions (double Boxing for
example), but have failed to get them to work. Probably because I am
working at a “cut-and-paste” code level as I really don’t understand
much of what I am doing (I am experienced at neither Rust nor C).

Can anyone help with a solution.

#2

Remember that (as far as I know) C has no idea about traits, and in fact it doesn’t have any kind of polymorphism/inheritance etc. So, what really matters is the structure that the C library expects; ie the structure layout, because methods would probably be implemented on their side.

So from what I can collect, parser is a struct with an implementation of a parser, user_data is a pointer to a buffer/struct with data, and handler is a callback when it’s done running parser's function (Which takes the user data and turns it into a statement[?]). But what I’m not too sure about is parser, is it a struct with a configuration or a function that does the conversion?
Edit:
Take notice as well, that because of what I mentioned in the top half of this post that traits won’t really work, they only specify functions and not members in a struct, so there’s no guarantee that the implementors for your trait will structure their struct properly…

#3

No. raptor_parser is an object. raptor_parser_set_statement_handler sets the handler on this. user_data is what ever you want. It just gets passed through the C library. Data gets into the parser by another method raptor_parser_parse_chunk. The library then calls the callback function that you set here any number of times based on what ever is in the data you pass to it.

C doesn’t need to do anything at all with the trait object. I just need C to tell me which trait object it is talking about, so that I know which rust handler to call.

#4

Oh, so basically you’re choosing what to do with statement when you get into statement_handler based on what you got in user_data? This is pointing me to either an enum or a number to switch on. Oh and also, if you insist on using a trait object, then you’d have to do something a bit convoluted, trait objects are unsized, and therefore we store a pointer to them, but pointers to trait objects are the size of 2 pointers, meaning that you would have to: create your object, create a pointer/reference to it, cast that to a (usize, usize), take a pointer to that, and then pass the *mut (usize, usize) in user_data to then do the following: (untested)

let double_pointer: (usize, usize) = *(user_data as *mut (usize, usize));
let trait_object: &dyn Trait = double_pointer as *dyn mut Trait as &dyn Trait;

to get your trait object back
Edit:
Hmm, I can’t seem to get this working though, take a look at this, any thoughts?

#5

Okay, so if you just want to pass a trait object through your c api, you can do what’s described here to cast to and from a box like you mentioned.
Edit:
Here is a version with better comments!

#6

That seems to work well. And it seems to work with my library, but
alas only the first time the call back is called. The second time, it
core dumps.

This seems to make sense to me. The documentation on from_raw() says:

After calling this function, the raw pointer is owned by the
resulting Box. Specifically, the Box destructor will call the
destructor of T and free the allocated memory. Since the way Box
allocates and releases memory is unspecified, the only valid pointer
to pass to this function is the one taken from another Box via the
Box::into_raw function.

So, the first call is fine, but then the ParserHandler object gets
collected.

However, this doesn’t seem to be the case with your example:

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=80b04b696ce082e00cab919172588d7b

I can call Box::from_raw on the same pointer multiple times and it
works. Surely, it shouldn’t?

#7

Actually, because we’re Boxing up a reference, it doesn’t drop the data the bottom pointer is pointing to:

let b = 0;
let c: &usize = &b;
let d = Box::new(c);
let e: *mut &usize = Box::into_raw(d);

b still lives after d has been dropped/consumed because the content of d is a pointer
But I’m actually abit confused as to why c doesn’t get dropped, or trait_reference in my example, must be because of references being cloneable or something :confused:
Ah, I just realized that we’re dealing with an FFI, therefore rust can’t optimize at all to make sure that the reference isn’t removed, but in my example it can ensure that it lives.

#8

It’s because references are Copy so they don’t need to be dropped. Also Box::into_raw doesn’t drop the value inside the Box.

1 Like
#9

Yeah, sorry, my example wasn’t a complete one, but enough to get the point across.

#10

I think that should be UB that just happens to work in this case.

#11

This is still not working correctly in my life code. The trait object (Foo in your example, ParserHandler in mine) is not getting dropped as far as I can tell. However after the second Box::from_raw call, calling any trait method on results in a core dump.

Very strange indeed.

#12

I finally have a working version of this, following this example on Stackoverflow.

In this case, his first solution is similar to yours, but apparently it doesn’t work if the library stores a copy of the pointer (as mine does). I’d seem this before, but hadn’t understood it. The discussion here has helped a lot with that.

1 Like