【Rust】Exception: User-mode data execution prevention (DEP) violation at location when calling Rust FFI library function on x86 Windows and Linux

help
1

I'm using Rust to call a Rust-written FFI library function. The library function runs fine on macOS Pro with an M1 processor. However, when running on x86-based Windows and Linux virtual machines, I encounter an error whenever any method within the library function is called. The error message is as follows:

Exception: Exception 0xc0000005 encountered at address 0x7fffd7609220: User-mode data execution prevention (DEP) violation at location

Problematic code repository: https://github.com/dingdaoyi/yanbing-edge

Code snippets:

Library function:

fn initialize(&mut self, device_list: Vec<Device>, sender: mpsc::Sender<PointEvent>, handle: Handle) -> Result<(), String> {
    ModbusTcpProtocol::init_log();
    println!("协议包含数据:{:?}", device_list);
    self.sender = Some(sender);
    self.device_list = device_list;
    self.handle = Some(handle);
    self.init_modbus();
    self.schedule_event();
    Ok(())
}

Main function invocation:

pub async fn load_protocol(
    &self,
    config: &ProtocolConfig,
    sender: mpsc::Sender<PointEvent>,
    device_list: Vec<Device>,
) -> Result<()> {
    // Load the protocol library
    let lib_path = Path::new(&self.lib_path);
    let protocol_path = lib_path.join(get_library_filename(&config.path));
    let lib = unsafe { Library::new(protocol_path) }?;

    // Get the create_protocol function symbol
    type CreateProtocolFn = extern "C" fn() -> *mut dyn Protocol;
    let constructor: Symbol<CreateProtocolFn> = unsafe { lib.get(b"create_protocol")? };

    // Call the function to obtain a raw pointer to the Protocol trait instance
    let boxed_raw = constructor();

    // Construct a Box from the raw pointer
    let mut protocol_box = unsafe { Box::from_raw(boxed_raw) };
    let protocol_box1 = unsafe { Box::from_raw(boxed_raw) };
    tokio::task::spawn(async move {
        let handle = tokio::runtime::Handle::current();
        protocol_box.initialize(device_list, sender, handle).unwrap();
    });
    self.add_protocol(config.id, protocol_box1)
}

Please help me solve this issue. Thank you!

2

Also posted on Stack Overflow.

3

AFAIK trait objects are not supposed to be FFI safe, so here you could potentially get junk.

You're creating two boxes that point to the same allocation. This is UB, Box's allocation is supposed to be unique to that Box. Moreover when one of them will go out of scope it will free the allocation, leaving the other with a dangling pointer, resulting in use-after-free and/or double-free.

4 Likes
4

This is another issue that I have mentioned in the code. Currently, the initialize method needs to block indefinitely, so I used tokio::task::spawn. However, the problem I'm asking about occurs in the FFI code, specifically when calling self.init_modbus(). It throws an error regardless of what's inside the method. This issue doesn't occur in my development environment on macOS, where it runs fine.

5

I have just started learning Rust, and I'm not familiar with some of its mechanisms. If you have time, I would appreciate it if you could take a look at my code and help me solve the issue of not releasing the lock when initialize blocks. I would greatly appreciate your assistance, and thank you in advance!

6

If the error is indeed the fact that trait objects are not FFI safe then this sounds expected.

Probably it just happens to work on that OS/machine, even though the code contains Undefined Behaviour and is thus not guaranteed to work.

Then I would suggest you to avoid playing around with unsafe and dynamic libraries, they're both very very tricky to get right. If you really really want to use them I suggest you to read the Rustonomicon and to stick to only #repr(C) types at the FFI boundary.

1 Like
7

My initial idea was to achieve dynamic loading of JAR files, just like in Java, to implement protocol, rule, and northbound data flow extensions. It seems that this approach is not feasible. I started with the most challenging path right from the beginning. :rofl: Now I have to see if I can achieve the desired result using features.