Maintaining lifetimes after FFI clobbers them


#1

Ah yes, the rare “borrow checker is too permissive” topic.

I have a function that passes in references to FnMuts, then internally sends them across an FFI boundary to get registered as a callback from a C function, and returns a handle. When the handle falls out of scope on Rust’s end, its Drop trait unregisters the callbacks. This works (yay) but rustc is not preventing what I think are invalid borrows. For instance, I can do this:

fn this_does_not_seem_safe() -> DeviceCallbackHandle {
    let mut on_device_connect = |device_info: DeviceInfo| {
        println!("{} connected", device_info.uri);
    };

    let mut on_device_disconnect = |device_info: DeviceInfo| {
        println!("{} disconnected", device_info.uri);
    };

    let mut on_device_state_change = |device_info: DeviceInfo, state: DeviceState| {
        println!("{} changed state: {:?}", device_info.uri, state);
    };

    register_device_callbacks(&mut on_device_connect, &mut on_device_disconnect, &mut on_device_state_change).unwrap()
}

Unless I’m mistaken, the closures fall out of scope at the end of the function, even though C code still references it. To convince rustc that this is the case, I think I need to tie the lifetime of the DeviceCallbackHandle to the lifetimes of the closures, since the handle will unregister the callbacks when it drops, dropping the closure references from C code, but I don’t know how.

I tried adjusting the function signature to add two lifetimes like this:

// Previously there were no lifetime annotations at all
pub fn register_device_callbacks<'cb, 'handle, F1, F2, F3>(
    on_device_connected: &'cb mut F1,
    on_device_disconnected: &'cb mut F2,
    on_device_state_changed: &'cb mut F3
) -> Result<DeviceCallbackHandle<'handle>, Status>
    where F1: FnMut(DeviceInfo), F2: FnMut(DeviceInfo), F3: FnMut(DeviceInfo, DeviceState), 'cb: 'handle {

    // Some code

    let closures = Box::new(ClosureStruct {
        on_device_connected,
        on_device_disconnected,
        on_device_state_changed,
    });

    unsafe {
        oniRegisterDeviceCallbacks(
            &mut callbacks, // a struct of `extern "C"` wrapper functions, not shown
            Box::into_raw(closures) as *mut _,
            &mut callbacks_handle, // just ptr::null_mut()
        )
    }

    // Some more code

     Ok(DeviceCallbackHandle {
        callbacks_handle,
        _closures_lifetime: PhantomData,
    })
}

But this backfired. rustc now believes that the handle lasts for longer than it really does, so the handle can never be valid.

fn main() {
    let mut on_device_connect = // closure body
    let mut on_device_disconnect = // closure body
    let mut on_device_state_change = // closure body

    if let Ok(handle) = register_device_callbacks(&mut on_device_connect, &mut on_device_disconnect, &mut on_device_state_change) {
        println!("Got handle {:?}", handle);
        loop { thread::sleep(Duration::from_millis(100)) }
    }
} // error! closures dropped here while still borrowed. really?

('cb : 'handle does mean "'cb matches or outlives 'handle", right?) Does anyone have a clue what the correct annotation is?


#2

The base code works;
https://play.rust-lang.org/?gist=e0e48b69bccf32a4cce8e973d04a09fb&version=stable&mode=debug&edition=2015

Need to know more detail to be helpful. (of ClosureStruct, callbacks, oniRegisterDeviceCallbacks)


#3

It does mean that, but I think you don’t need 'handle at all - register_device_callbacks() should return a DeviceCallbackHandle<'cb>. This extends the mutable borrow of the closures into this struct, but more importantly, it won’t allow this struct to outlive any of the closures.


#4

Thanks, I should have linked the code before:
register_device_callbacks
the other structs


#5

Looks to me like a compiler bug.
Stripped down version;
https://play.rust-lang.org/?gist=f0fc23d2b51cf2be2944be11bbdace2d&version=stable&mode=debug&edition=2015


#6

There’s a known issue about an expression terminating a block causing the borrow to be extended into the outer scope. It might be https://github.com/rust-lang/rust/issues/22449 or one of the linked issues.

The workaround is usually to introduce a binding and then work with that.


#7

Yikes, thanks both of you. I guess I see the issue now. Coincidentally, while working on something else I changed the signature of the main function in my last code sample to return fn main() -> Result<(), Status> and used ? for early returns instead of if let, and that change also fixed the issue when I added lifetimes back in.

Amended playgoround: https://play.rust-lang.org/?gist=b6e9873f45bd9d41cd27c38e9df92da9&version=stable&mode=debug&edition=2015 where rustc permits the main function to work, also correctly fails to compile the toy example that intentionally drops the closures early.