Async Rust - can't send type containing unsafe mutable pointer in library

[Kimimaru]

Hi everyone, I'm fairly new to Rust and especially async Rust. Some context: I'm writing a program that takes in a string and converts it to instructions to press buttons/axes on a virtual input device. For example, up'1 would hold up on a controller's joystick for 1 second. I want to run an async method executing the instructions on said device. For creating and managing the virtual devices, I'm using evdev-rs.

So back to my problem. I intend to run the instructions through an input handler like this:

let mut v_joystick_manager = VirtualJoystickManager::new(1);

// Create a virtual joystick
_ = v_joystick_manager.create_joystick(0, "TRBot Joystick 0", 0x378, 0x3, 3);

let mut join_set = JoinSet::new();

let input_handler = input_handler::InputHandler::new(Arc::new(v_joystick_manager));

join_set.spawn(async move { input_handler.execute_instructions(&instructions) });
// ^ Compiler error occurs here!

The compiler is complaining because the input handler isn't safe to send because it has a VirtualJoystickManager which contains a VirtualJoystick, which in turn contains a UinputDevice from evdev-rs. That definition for that is as follows:

pub struct UInputDevice {
    raw: *mut raw::libevdev_uinput, <---- Compiler complains about this pointer!
}

unsafe impl Send for UInputDevice {}

The exact error I get is:

error[E0277]: `*mut evdev_sys::libevdev_uinput` cannot be shared between threads safely
   --> src/main.rs:164:22
    |
164 |             join_set.spawn(async move { input_handler.execute_instructions(&instructions) });
    |                      ^^^^^ `*mut evdev_sys::libevdev_uinput` cannot be shared between threads safely
    |
    = help: within `VirtualJoystickManager`, the trait `Sync` is not implemented for `*mut evdev_sys::libevdev_uinput`
note: required because it appears within the type `UInputDevice`

I've tried modifying evdev-rs to implement Sync, but it still complains. I've also tried wrapping the spawn in an unsafe block, but it still throws the same error.

I'm okay with breaking the thread safety of the virtual devices since it's intended for multiple different threads to access the devices and write different states to them. My program would not function as well otherwise.

What are my options here to get this working? Please let me know if you need more details. Thanks so much in advance!

[quinedot]

@Kimimaru

I've tried modifying evdev-rs to implement Sync, but it still complains.

What's the full error when you do that?

[Kimimaru]

The full stack trace is here:

error[E0277]: `*mut evdev_sys::libevdev_uinput` cannot be shared between threads safely
    --> src/main.rs:164:22
     |
164  |             join_set.spawn(async move { input_handler.execute_instructions(&instructions) });
     |                      ^^^^^ `*mut evdev_sys::libevdev_uinput` cannot be shared between threads safely
     |
     = help: within `VirtualJoystick`, the trait `Sync` is not implemented for `*mut evdev_sys::libevdev_uinput`
note: required because it appears within the type `UInputDevice`
    --> /redacted/evdev-rs/src/uinput.rs:11:12
     |
11   | pub struct UInputDevice {
     |            ^^^^^^^^^^^^
note: required because it appears within the type `Option<UInputDevice>`
    --> /redacted/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:572:10
     |
572  | pub enum Option<T> {
     |          ^^^^^^
note: required because it appears within the type `VirtualJoystick`
    --> /redacted/src/virtual_joystick.rs:195:12
     |
195  | pub struct VirtualJoystick {
     |            ^^^^^^^^^^^^^^^
     = note: required for `&VirtualJoystick` to implement `Send`
note: required because it's used within this `async` fn body
    --> /redacted/src/instruction_handler.rs:43:3
     |
43   |   ) {
     |  ___^
44   | |     let ev_key = int_to_ev_key(button_instruction_data.button_value as u32).unwrap();
45   | |
46   | |     _ = virtual_joystick.press_release_button(ev_key, button_instruction_data.start_press_val);
...    |
59   | | }
     | |_^
note: required because it's used within this `async` fn body
    --> /redacted/src/input_handler.rs:50:77
     |
50   |       async fn process_instruction(&self, instruction_type: &InstructionType) {
     |  _____________________________________________________________________________^
51   | |         match instruction_type {
52   | |             InstructionType::Button(btn_data) => {
53   | |                 process_button(
...    |
95   | |     }
     | |_____^
     = note: required for `Unique<impl Future<Output = ()>>` to implement `Send`
note: required because it appears within the type `Box<impl Future<Output = ()>>`
    --> /redacted/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:231:12
     |
231  | pub struct Box<
     |            ^^^
note: required because it appears within the type `Pin<Box<impl Future<Output = ()>>>`
    --> /redacted/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/pin.rs:1089:12
     |
1089 | pub struct Pin<Ptr> {
     |            ^^^
note: required because it's used within this `async` fn body
    --> /redacted/src/input_handler.rs:41:83
     |
41   |       pub async fn execute_instructions(&self, instructions: &Vec<InstructionType>) {
     |  ___________________________________________________________________________________^
42   | |         println!("{} instruction(s) to execute!", instructions.len());
43   | |
44   | |         for instruction in instructions {
...    |
48   | |     }
     | |_____^
note: required by a bound in `JoinSet::<T>::spawn`
    --> /redacted/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.45.0/src/task/join_set.rs:136:12
     |
132  |     pub fn spawn<F>(&mut self, task: F) -> AbortHandle
     |            ----- required by a bound in this associated function
...
136  |         T: Send,
     |            ^^^^ required by this bound in `JoinSet::<T>::spawn`

I implemented unsafe impl Sync and unsafe impl Send for InputHandler, VirtualJoystick, VirtualJoystickManager, and UInputDevice.

It's worth mentioning that VirtualJoystickManager has a Vec<Option<VirtualJoystick>> containing the virtual joysticks. And it appears Option<T> doesn't implement Sync/Send either.

The async runtime I'm using is tokio.

[Kimimaru]

Due to the raw pointer I probably won't be able to run this outside of an unsafe context, which is okay, but even the unsafe context is throwing the above error.

let input_handler = input_handler::InputHandler::new(Arc::new(v_joystick_manager));

unsafe {
    join_set.spawn(async move { input_handler.execute_instructions(&instructions) });
}

I'm not sure if I'm doing that correctly. I haven't written unsafe code in Rust before.

[nerditation]

@Kimimaru

I probably won't be able to run this outside of an unsafe context,

unsafe is not a runtime concept, it is checked at compile time. there is no such thing as "running inside" an unsafe context.

you only need unsafe when calling unsafe functions, and when dereferencing raw pointers directly. unsafe cannot make arbitrary compile error go away.

@Kimimaru

join_set.spawn(async move { input_handler.execute_instructions(&instructions) });

is the execute_instructions() method async? if it was, then this code probably is NOT doing what you want. either add .await(), or remove the async move block:

// option 1: add `.await`
join_set.spawn(async move { input_handler.execute_instructions(&instructions).await });

// option 2: remove the enclosing`async` block
join_set.spawn(input_handler.execute_instructions(&instructions));

@Kimimaru

unsafe impl Send for UInputDevice {}

this should be enough to make the code compile, if the UInputDevice is the only type that contains non-Send fields like raw pointers. but I can't be sure without seeing the full code of the execute_instructions() method.

@Kimimaru

I'm okay with breaking the thread safety of the virtual devices

thread safety is not a "small" problem, you cannot just choose to "give up" thread safety without impact memory safety of the program.

there's a reason the Send trait requires unsafe to manually implement for a type. you must understand the implication of the unsafe conditions.

@Kimimaru

it's intended for multiple different threads to access the devices and write different states to them

ffi bindings inevitably use raw pointers, however, you should create safe wrappers or use safe constructs for those types that are designed to be accessed from multiple threads, and the application should use these safe types instead of raw ffi types.

[Kimimaru]

Thanks for clarifying unsafe and the terminology. However I still can't get the code to compile. UInputDevice already implements Send, and I made it implement Sync as well, but I'm getting the same exact errors from the compiler, as if I didn't change anything, and it's puzzling me.

If I run input_handler.execute_instructions(&instructions).await; on its own everything works, but the second I put it in the spawn is what causes the compiler errors.

Going by this thread, which has a similar problem I'm having, the last suggested solution is to, again, implement Send and Sync or deal with a Mutex, the latter of which I'd rather not have to do - I'm hoping for doing everything through async tasks rather than threads.

Here is the full code for the input handler (which I've slightly altered since the post). That code compiles and builds. The intention is to run execute_instructions on its own task each time the user inputs a string, which allows others to continue entering strings and thus executing more instructions while the existing sets of instructions are still going. This is why I need to spawn the task instead of awaiting it right there.

[kpreid]

Could you try to minimize this problem? That means, edit a copy of the code to remove uses of other libraries (e.g. replacing UInputDevice from evdev_sys, or some other struct containing it, with a struct you declare yourself that contains a raw pointer and the same unsafe impls) and remove unnecessary parts until you have a short program (that can probably be run on the Rust Playground) that demonstrates the problem.

This process, while tedious, will either lead you to understanding of the problem, or produce something that we can more readily understand and experiment with in order to help you, and will almost certainly result in full understanding of the problem.

[Kimimaru]

Thanks all, I managed to solve it! For reference I put this code in the playground.

It compiles whereas my own code wasn't. I tried building after implementing Sync in UInputDevice, and while my IDE was saying there were issues, cargo said otherwise (there were new compiler errors but not related to this). Lesson learned. Thanks for all the help!