What is the easiest way to send complicated struct with vec in FFI?

help
1

I'm trying to create a rust client for swapi.dev API that could be used in Android and IOS. (Kotlin & Swift)
I'm facing difficulties declaring FFI for the Swift. I understand that to send vector I need wrap it in a structure that would have pointer and usize.
What is the best way (without tons of boiler plate code) to transform a structure that contains a lot of nested vectors into "C" supported struct?

  • Is there any possibility to set custom translation for vector in the cbindgen? (I have not found it in doc.)
  • Any way to do it with code-generation?

For Kotlin I use rust_swig so I don't need to wrap it manually.

This is struct I need to send from Rust to Swift:

#[repr(C)]
#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct ResponsePeople {
    pub count: i64,
    pub next: String,
    pub results: Vec<People>,
}

#[repr(C)]
#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct People {
    pub name: String,
    pub height: String,
    pub mass: String,
    #[serde(rename = "hair_color")]
    pub hair_color: String,
    #[serde(rename = "skin_color")]
    pub skin_color: String,
    #[serde(rename = "eye_color")]
    pub eye_color: String,
    #[serde(rename = "birth_year")]
    pub birth_year: String,
    pub gender: String,
    pub homeworld: String,
    pub films: Vec<String>,
    pub species: Vec<String>,
    pub vehicles: Vec<String>,
    pub starships: Vec<String>,
    pub created: String,
    pub edited: String,
    pub url: String,
}

What I did so far, for each DTO I created a separate native DTO which I use to send in FFI. for e.g.:

#[repr(C)]
pub struct PeopleNativeWrapper {
    array: *mut PeopleNative,
    length: usize,
}

#[repr(C)]
pub struct PeopleNative {
    pub name: *const c_char,
    pub height: *const c_char,
    pub mass: *const c_char,
}

But it is a lot of boiler plate code. I need not only wrap vec but also to translate all types to C compatible:
String nama => pub name: *const c_char,;

2

I am personally bad at FFI. I prefer this raw copy between simple structures than complex annotations and understand of both languages FFI interfaces.

So the easy way I found is just transfer bytes between FFI and use some binary serialization format with automatic code generation like flatbuffer.

1 Like
3

When you're trying to share types that aren't #[repr(C)] with another language, the easiest way is to not give it direct access to the fields. Instead, you pass Swift an opaque pointer to your struct and provide getter functions that hand out pointers to the things it wants to access.

With fields that have a dynamic length (e.g. Vec and String) as well as a pointer to the field you'll also need to tell swift about the item's length. There are a bunch of ways you could do this, for example by returning some #[repr(C)] struct containing a char* pointer and size_t length (I guess that'd be the C equivalent of a slice), accepting "out pointers" (e.g. void person_get_name(const People *person, const char **name, size_t *length)), etc.

The idea is you can avoid writing loads of boilerplate and needing to keep the native DTOs in sync with the Rust types. You also only pay for what you need, so if you only need access to 5 fields you'll only need to define 5 getters. It's also less intrusive than trying to make your Rust types FFI-friendly.

2 Likes
4

I was thinking to use ProBuf, I saw that Swift supports it. But I was afraid that it will be relative slow comparing to the FFI.
I have not heard about FlatBuffers before. Can you please share more experience:

  • How it is in terms of speed and boilerplate code?
  • You can reuse schema in all 3 platforms (Swift, Kotlin, Rust)?
5

I tried something like that as well but still had to create a lot of boilerplate code.
I assume you suggesting something like this, (I tried this for networking client):

//Create client
#[no_mangle]
pub extern "C" fn create_swapi_client() -> *mut SwapiClient {
    Box::into_raw(Box::new(SwapiClient::new()))
}

//Free client
#[no_mangle]
pub unsafe extern "C" fn free_swapi_client(client: *mut SwapiClient) {
    Box::from_raw(client);
}

//Load name
#[no_mangle]
pub unsafe extern "C" fn load_name(client: *mut SwapiClient, c_callback: fn(*mut c_char)) {
    let local_client = client.as_ref().unwrap();
    //code code code
    local_client.loadName(1, callback);
}

So the same with Person, just to expose pointer and that for each field create a separate method?

I have some concerns here:

  • if I would have 100+ getter how this will affect performance? (basically they are all static, I assume)

Is the next example correct?
---> I'm getting a DTO with vector inside, I'll expose pointer to this DTO than will have a func in rust that will take it and return a pointer to vec that is inside of it (here I'll already need wrapper with size?), and then after I got a list of an element to swift to get any field I'll need to use create a method that accepts a pointer to element of a list and returns specific field, correct?

6

What I'm referring to is just a plain ol' extern "C" function, so unless you're doing this in a really hot loop you'd barely notice. .The function call can't be optimised/inlined, so retrieving a field would cost about the same as calling a virtual method.

Compare that with something like serialization (e.g. to JSON or Protobufs) which would need to serialize the entire object and all its fields (and all their fields, ...) and allocate a new buffer every time you want to pass a Rust object to Swift. You'd have pretty much the same problem with a DTO, except the DTO is copying each field to its #[repr(C)] equivalent.

By using getters there's no extra copying because you're working with a pointer to the original Rust object.

To put things in perspective... At my day job I work on a CAD/CAM program and because the CAD engine we use under the hood is a 3rd party native DLL, every property access (e.g. some_point.x) goes via some getter. I've never noticed any performance issues with this, instead most of my performance issues come about because I've used the wrong algorithm (e.g. a linear scan of 50,000 elements instead of using a cache/lookup table) or because I make unnecessary intermediate lists instead of using iterators.

I have a feeling you could use code generation to write the getters for you (e.g. a proc macro).

Pretty much. This is what I came up with.

If I needed to do it for more than half a dozen fields I'd probably write some code to write the boilerplate for me (e.g. a macro_rules macro or even a procedural macro if I felt that way inclined).

7

Thanks, great example! if I understood correctly, you declare UnsafePointer in Swift and pass it to Rust to populate data? I'll try to test this approach as well

In general, I would say that from a mobile perspective I see a few limitations:

  • performance
  • size
  • cross-platform reusability - can I use the same approach with JNI Env (Java/Kotlin)
  • start-up time - when methods are loaded etc.

As I understood performance should not be a problem.
Size- I was comparing with a native solution. I have not checked IOS but on Android diff is pretty small for a very simple app (6 MB vs 7 MB) I think we pay only for base codebase which will not change for other functions.
About others I'm not sure, maybe you have some insights about it as well?

8

I only have experimental use, I am an amateur in FFI world. Unfortunately is still a solution without a problem solve :slight_smile:

In theory, is very good since you define your schema in a DSL, generate the code for all platform, it is fast with zero memory copy access.

The DSL support complex types [1], so you can define almost any data structure. Once the code is generated, you just use it. No skill needed to understand each type in each language, and then how to map each type for each pair of languages.

In reality, it is a bit painful to work directly with their data struct. Rust plays nice. JVM languages are horrible since its memory layout is incompatible. For instance, in C# you can not just add a Vec, but needs to start_array in flatbuffer structure, append all structs using flatbuffer constructors, then complete the array.

[1] - FlatBuffers: Writing a schema

1 Like
9

Can you please share some code snippets? Just curious how bad it is :frowning:

10

From examples

Schema

C#

Rust

My personal test

Schema
https://github.com/sisso/test-unity3d-rust/blob/master/data/schema.fbs

C#

Rust

It is doable for transfer, but not in spread in your main code without proper wrappers.

1 Like
11

seems it still will require some boilerplate code.. anyway will test it. Thanks!

12

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.