Rust FFI with Callback memory issue

1

I am just learning Rust therefore I apologize if this is a very basic question. I want to create a library that I can use across multiple languages and operating systems. This library will need to have the ability to use callbacks to pass information back to the main application so it does not hang while the Rust library does its thing. When I spawn a new thread, and pass in the reference to the callback function it appears that I may have a memory leak.
I created a sample project where the Rust code looks like this:

use std::thread;
use std::time::Duration;


pub fn do_logic() -> i32 {
    thread::sleep(Duration::from_millis(50)); 
    42
}

#[no_mangle]
#[cfg(windows)]
pub extern fn my_rust_function(callback: extern "stdcall" fn(i32)) {
    thread::spawn(move || {
        let val = do_logic();
        callback(val);
    });
}

#[no_mangle]
#[cfg(not(windows))]
pub extern fn my_rust_function(callback: extern "C" fn(i32)) {
    thread::spawn(move || {
        let val = do_logic();
        callback(val);
    });
}

For my logic I am just pausing for a short period of time and then return a value back. When the value is returned I use the callback function to return the value to the main application. In the main application, using C# or Java, if I call the my_rust_function each time the callback is called the memory usage continuously goes up. If I change the code in the my_rust_function to this:

let val = do_logic();
callback(val);

where the thread::spawn line is removed then the memory usage stays constant. As an example of how I am testing this code, here is the sample C# code that uses it:

public sealed partial class MainPage : Page
{
    public delegate void PrintFn(int x);

    [DllImport("rust_forum_example.dll")]
    private static extern Int32 my_rust_function(PrintFn fn);

    int cnt = 0;

    public MainPage()
    {
        this.InitializeComponent();
        callRust();
    }

    public void callRust()
    {
        my_rust_function(Print);
    }

    public void Print(int x)
    {
        Debug.WriteLine($"C#: {cnt} - {x}");
        cnt++;
        if (cnt < 1000)
        {
             callRust();
        }
    }
}

I understand that when I spawn the new thread, callback is moved into the thread therefore the thread now owns it. Could this be causing a memory leak? If so how would this be fixed?

2

The memory usage is going up because of all the threads that are being created. You probably want to rethink the design. In particular, make the Java/C# app invoke the Rust code on its background thread if you don't want it to block the UI loop (or whatever); the Rust code can then be single threaded (or if it wants to use multiple threads internally, it can but this fact won't leak out in any way).

3

Thank you for the reply. I can try that however I would prefer to have the threads handled in the Rust library so I only need to implement once rather than in every project the library is used in.

4

IMO, the threads in the Rust lib should only be needed if the Rust code, on its own, wants to use threads. In your case, it's only using threads because of the global knowledge that some code is calling into Rust where hogging the CPU will cause issues. In addition, the Rust threads are left detached because nobody is join()'ing on them. I think, at the least, you'd want to use a threadpool to service the work, and not spawn a new thread per call.

As for reuse, you can encapsulate the background thread offload in a C# or Java library, and reuse that from other C#/Java code. That also gives you flexibility to determine the appropriate threading model, threadpool size, etc that should be used, and not leave it up to the Rust lib to guess.

1 Like
5

100% agree with vitalyd on this. And if my C# knowledge is still up to date, offloading a CPU task to the thread pool is as easy as:

var task = Task.Run(() => my_rust_function());

And the benefit of this approach is that you then get a C# native interface for managing the asynchronous Task - you can await it, block on it (if appropriate), compose it with other Tasks...