A scope guard whose lifetime is bounded but doesn't hold a reference?

I'm trying to write a function that takes a reference to some object, &'a thing, and returns a scope-guard object, whose lifetime needs to be bounded by the lifetime 'a; however, it should not be considered to capture a reference to thing. It just needs to run its Drop impl before thing ceases to exist. With the most straightforward implementation of a lifetime-bounded scope guard I can think of, the compiler thinks the guard does capture a reference to the thing, and so I get E0502 errors ("cannot borrow X as mutable because it is already borrowed as mutable"). What's the best way to code it so that the lifetime is bounded as desired but the reference isn't captured?

The concrete code requires some background, because I am, in fact, trying to do some black magic. I'm writing a program for a #[no_std] environment. It does have a debug console, and so I'd like to log progress messages there, and panic messages as well. A global heap allocator is available, but memory is limited, and unwinding and threads are not available. Crucially, the debug console is strictly line-oriented. If you feed it a message in chunks of less than a full line it will mangle the message.

So to deal with the line-oriented console I have copied just barely enough of std::io into my crate that I can use LineWriter. Memory is limited, so I want to create the LineWriter only once, in my startup glue. I pass a mutable reference to it to the main program. The challenge is to make the same LineWriter also available to the #[panic_handler] function. Now, if the panic handler has been called, the main program isn't going to execute any more code, so it is safe for the panic handler to steal the main program's &mut reference; the dynamic lifetime of that reference has ended, even though the compiler cannot prove it.

Version 0 of the code I wrote for this, which did work, looked like this:

static PANIC_CONSOLE: Mutex<
    UnsafeCell<Option<&'static mut ConsoleWriter<'static>>>,
> = Mutex::new(UnsafeCell::new(None));

unsafe fn set_console<'a, 'b: 'a>(writer: &'a mut ConsoleWriter<'b>) {
    unsafe fn duplicate<'a, 'b: 'a>(
        p: &'a mut ConsoleWriter<'b>,
    ) -> &'static mut ConsoleWriter<'static> {
        mem::transmute(ptr::from_ref(&p).read())
    }

    unsafe {
        let guard = PANIC_CONSOLE.lock();
        *guard.get() = Some(duplicate(writer));
    }
}

unsafe fn clear_console() {
    unsafe {
        let guard = PANIC_CONSOLE.lock();
        *guard.get() = None;
    }
}

unsafe fn get_console() -> Option<&'static mut ConsoleWriter<'static>> {
    let guard = PANIC_CONSOLE.lock();
    unsafe { &mut *guard.get() }.take()
}

where Mutex is a fake mutex that exists to satisfy the Sync bound on mutable statics. (Remember, this is a strictly single-threaded environment.) The panic handler called get_console, and the startup glue called set_console immediately after creating the ConsoleWriter and clear_console right before it went out of scope. (The environment does have a meaningful notion of executing one program after another; it's normal for programs to finish their work and exit.)

This works, as I said, but it is an API with sharp edges. If the startup glue happens to forget to call clear_console at the right time, and a panic happens after the ConsoleWriter goes out of scope, which is a possibility I can't rule out for certain, the panic handler will scribble on deallocated stack memory, with unpredictable consequences. It occurred to me that I could eliminate that problem by having set_console return a scope guard that clears the cell on drop:

struct CellGuard<'a> {
    closure: fn(),
    lifetime: PhantomData<&'a ()>,
}

impl<'a> CellGuard<'a> {
    fn new(f: fn()) -> Self {
        Self { closure: f, lifetime: PhantomData }
    }
}

impl<'a> Drop for CellGuard<'a> {
    fn drop(&mut self) {
        (self.closure)()
    }
}

static PANIC_CONSOLE: Mutex<
    UnsafeCell<Option<&'static mut ConsoleWriter<'static>>>,
> = Mutex::new(UnsafeCell::new(None));

unsafe fn set_console<'a, 'b: 'a>(writer: &'a mut ConsoleWriter<'b>) {
    unsafe fn duplicate<'a, 'b: 'a>(
        p: &'a mut ConsoleWriter<'b>,
    ) -> &'static mut ConsoleWriter<'static> {
        mem::transmute(ptr::from_ref(&p).read())
    }

    unsafe {
        let mguard = PANIC_CONSOLE.lock();
        *mguard.get() = Some(duplicate(writer));
    }
    CellGuard::new(|| unsafe {
        let mguard = PANIC_CONSOLE.lock();
        *mguard.get() = None;
    })
}

clear_console goes away, get_console is unchanged. It's still an unsafe API because of the shenanigans with stealing a mutable reference, but one way to misuse it has been eliminated. Strictly better, right?

Except now I get E0502 on the startup glue:

error[E0502]: cannot borrow `console` as mutable because it is already borrowed as mutable
  --> src/glue/entry.rs:44:25
   |
30 |     let _panic_console_guard = panic::set_console(&mut console);
   |                                                    ----------- first mutable borrow occurs here
...
44 |     crate::main(&argv, &envp, &mut console);
   |                                ^^^^^^^^^^^ second mutable borrow occurs here
...
71 | }
   | - first mutable borrow might be used here, when `_panic_console_guard` is dropped and runs the `Drop` code for type `glue::panic::CellGuard`

So now I'm stuck. Apparently giving a CellGuard a lifetime derived from the &console reference means the compiler thinks CellGuard holds an actual copy of the reference, even though it does not? (Pay no attention to what's in the UnsafeCell :wink:) Surely there's some way to separate the lifetime from the reference itself?

(I'm open to alternative ideas, but I really have put a lot of thought into this and all the other possibilities I can think of are ruled out by one of the environmental limitations that I have listed, and there's more of them, so be prepared for me to come back at you with "that won't work because ...".)

That's not how it works though. An object is not considered to "capture" some reference, that's just how it may be presented for easier understanding of the common cases.

What actually happens is that your thing is borrowed for some lifetime 'a, and can't be used (mutably) anymore in that lifetime. The actual duration of the lifetime is then determined by its usages, such that it satisfies all the related constraints (otherwise it's a compile error). So what you want to do is to create something bounded by 'a, meaning it should generate one such constraint, but not actually generate a constraint. This is a contradiction!


I'm not sure if this will solve your issue, but instead of returning only the guard with lifetime 'a you could try also returning the &'a thing. This should allow you to continue accessing the thing while the guard exists, while still ensuring the guard doesn't outlive the initial thing borrow.

5 Likes

That's how it should work though!

What you're saying is that the borrow checker is ignoring a very real difference between &'a T and PhantomData<&'a ()>, namely that the former actually holds a reference to a concrete object with lifetime 'a and the latter doesn't. As a conservative approximation that's fine, but it means there are things that are legitimate but not possible.

... Presumably if we were to change Rust so that the borrow checker did pay attention to the difference, PhantomData would have to be grandfathered to behave as if it holds a reference to a concrete object, because that's how it gets used most of the time, e.g. in the example in the manual

struct Slice<'a, T> {
    start: *const T,
    end: *const T,
    phantom: PhantomData<&'a T>,
}

the overall object does hold references to the concrete object with lifetime 'a even though the PhantomData itself doesn't. But I see no reason why it would be unsound to have a way to capture the lifetime without the borrow -- to have a way to ensure object A is dropped before object B, but place no other constraints on usage of B.


That said, I've realized that my original analysis was incorrect, and it is not necessarily safe for my panic handler to "steal the main program's &mut reference", because even in this single-threaded context, and even though the main program doesn't get to execute any more code after the panic handler is called, I might still have reentrancy issues to worry about. Specifically, the panic might originate inside the ConsoleWriter, and so trying to reuse it without any form of locking is incorrect. That puts my problem back into territory covered by a standard (non-recursive) Mutex.

It is already documented to behave like this.

Zero-sized type used to mark things that “act like” they own a T.

Adding a PhantomData<T> field to your type tells the compiler that your type acts as though it stores a value of type T, even though it doesn’t really.

I'm not saying it would be unsound, just that this would not be natural (if even possible) to express in the current borrow checker.

But your code that was failing to compile didn't satisfy this. If you allow to pass a &mut B to some function (which is what you where doing) then that function can very easily drop the object B before A is dropped, it just needs to replace it with something else. So I don't see how this restriction would work for you unless you actually cared about the variable holding B rather than the object B.

I regret to say I don't understand the distinction you're making. Can you give a concrete example of code that would "replace B with something else", please?

So, you want CellGuard to be forced to drop before &'a mut ConsoleWriter<'b> expires, but still have the ConsoleWriter<'b> be usable, right? Why not just hold onto the &'a mut ConsoleWriter<'b>[1] and implement DerefMut<Target = ConsoleWriter<'b>>? Consumers can use the ConsoleWriter by going through the CellGuard, similar to going through a MutexGuard.

A sketch:


I'm including these parts in case it helps clear anything up. There are no more practical suggestions.

Rust lifetimes ('_ things) are not directly about the liveness scope of values. It's an unfortunate clash of terminology. They're primarily about the duration of borrows. There's a relation with liveness, because it's an error to destruct something that's borrowed, but they are not the same thing. (It's also an error to move something that's borrowed, for example, even though no destructor need be ran in that case.)

To the borrow checker, the (possible) drop points are places where a value may be exclusively used, and aren't lifetimes ('_ things) associated with the value.


It's also true there is also no way to relate lifetimes besides equality or outlive bounds, and with an API like

fn example<'x>(input: &'x mut X) -> Something<'x>
// Or
// fn example<'x: 'y, 'y>(input: &'x mut X) -> Something<'y>

the Something<'_> being alive means 'x must be alive, which in turn means the X is still exclusively borrowed. That's what the API means, regardless of what holds the lifetime (e.g. PhantomData), or what terminology the diagnostics use. You have to exclusively borrow X to even make the call, and the lifetime 'x of that borrow has to be equal to/outlive the lifetime in the type of the Something<'_>.

Rust would need a new type of function API, with some sort of... dependent lifetimes or something. Some new way of relating lifetimes without equality or outlives.


  1. or "equivalent" ↩︎