Pinning a static future, safely, without allocating

I'd like to be able to pin a single, known future in static memory, and be able to poll it from a "notified" function that's handled by a external (async, but not rust async) executor-like loop of seL4 microkit.

In addition, I'd rather not have to allocate memory, as all the existing C code in this model don't need memory allocation at runtime.

I believe that on stable rust, my two options seem to be Box::pin (heap) or the pin! macro (stack). As I don't control the stack¹, I can't pin!, and I'd rather avoid heap allocation when it's a single known future.

What follows is the reasoning of the code, which is what I'm asking for feedback on, and if there's a better way to do it.

  • To be able to store the future in a static, we need TAIT nightly feature, which seems(?) to require another mod future module to do the type-inference
  • To be able to store the future in statics, we need const_async_blocks.
  • Since we can't directly create an owned-pinned value, we need a second static for storing the future, and to use Pin::static_mut we need a 'static lifetime, so we can't put static mut STORAGE inside of a RefCell etc. So there's an unsafe block that takes it to a pointer and back (because otherwise rustc complains about &mut STORAGE references).

Is there a way to do this without nightly? Some better feature I'm just missing?

The "working" implementation that you can just run is here. Following is a simple implementation for the rust playground that builds.

¹ Whilst I could get this working by rewriting the C code, there's a fair amount of resistance to this, not to mention that the existing C code has been formally verified and that work would have to be redone.


As a slightly secondary question, in the repo I need -Z build-std — as otherwise the linker complains about a missing rust_eh_personality symbol. Is that the appropriate fix?

#![no_std]
#![feature(type_alias_impl_trait, const_async_blocks)]
#![feature(thread_local)]
#![deny(clippy::undocumented_unsafe_blocks)]

use core::{cell::RefCell, convert::Infallible, future::Future, pin::Pin};

async fn mainer() -> ! {
    /* Pretend this actually does something useful */
    loop {}
}

#[thread_local]
static FUTURE: RefCell<Pin<&mut dyn Future<Output = Infallible>>> = {
    static mut STORAGE: future::TheFuture = future::run_main();

    // SAFETY:
    // - The mut STORAGE is only ever &mut referenced once, by us, and we hide
    //   it within this block, so aliasing is OK.
    // - It is a valid pointer, because we made the source itself.
    RefCell::new(Pin::static_mut(unsafe {
        (&raw mut STORAGE)
            .as_mut()
            .expect("ptr is valid + this is the only reference")
    }))
};

fn poll_task(_: Pin<&mut dyn Future<Output = Infallible>>) {
    /* pretend this actually polls */
}

#[no_mangle]
pub extern "C" fn notified(/* some arguments */) {
    poll_task(FUTURE.borrow_mut().as_mut());
}

mod future {
    use super::*;

    pub type TheFuture = impl Future<Output = Infallible>;

    pub const fn run_main() -> TheFuture {
        async {
            mainer().await;
        }
    }
}

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.36s

Not necessarily; you can add an Option so that you can use None for the static initialization, and only create the future when you first need to poll it, if you wish. That might be more trouble than it’s worth.

I can’t think of a way to not use TAIT (or Return Type Notation), though.

Note that this is unsound in the presence of multiple threads (you haven’t said whether this is an always-single-threaded environment) because you have a non-thread-local static mut whose access is being guarded by thread-local RefCells which don’t know about each other.

Even without that consideration, I think that if you’re going to set up a pinned static with unsafe code, it makes much more sense to not bother with the second static and just assert the pinnedness as needed — this gets you the same result with less complexity and no lurking thread hazard.

This would look like:

fn poll_task() {
    #[thread_local]
    static FUTURE: RefCell<future::TheFuture> = RefCell::new(future::run_main());

    let future_mut = &mut *FUTURE.borrow_mut();

    // SAFETY:
    // * This code does not move the cell or its contents.
    // * It is impossible for any other code to access it.
    // Therefore, we may soundly declare that it is pinned.
    let future_pin = unsafe { Pin::new_unchecked(future_mut) };

    let outcome = future_pin.poll(&mut task::Context::from_waker(task::Waker::noop()));
    assert!(outcome.is_pending());
}

It’s completely fine to construct Pin repeatedly, because Pin doesn't make a value pinned, so by dropping it we are not losing pinnedness. Rather, Pin claims pinnedness, and it is unsound to create Pin in situations where the claim would be false, but in this case it is true.

Ideally, Rust would let you just get a pinned (but not &'static) reference to the thread-local static, which would then let you use a pin-projectable cell like pin-cell to do this with zero new unsafe code.

Another angle would be if there was a cell type like pin-cell that also used atomic refcount updates (like atomic_refcell and atomicell say they do — I haven't tried them) so that you could outright put it in a non-thread-local static and use Pin::static_ref() to get pinning from that. That would be possible today on stable Rust, but I don’t know that anyone has written such a cell type already.

2 Likes

Yeah, unless TAIT gets stabilised before const-async then maybe it could. TBH, it's not actually that terrible — the full code example has init() and notified(), so there's already a convenient place for it.

Ah. That is very slightly confusing that statics inside static blocks inside of thread_local aren't themselves, thread_local. Yes, this is an always single-threaded environment.

My concern with this is that it then requires that as a "user" of the static, you write the unsafe. Which is probably fine, but requires you to be convinced that it is perfectly OK at the use site. Something which honestly, I'm not too comfortable with (whilst it's not relevant for my code, I'm not particularly certain on the precise "safe" semantics of pinning and options etc).

This wouldn't make everything else stable rust, and besides, having an atomic is a bit overkill when everything is single threaded :).

That'd be nice :slight_smile:

... and it turns out that this isn't allowed: E0625

error[E0625]: thread-local statics cannot be accessed at compile-time

The general rule that applies here (which, yes, is surprising) is that a static item is a single place no matter where you declare it. "inside static blocks inside of thread_local" has the same behavior as "inside a function"; there is always only one static place.

Also, note there isn’t really such a thing as a “static block”; the initializer of a static item is just another a const context.

The general solution to this problem is to take the unsafe code pattern and put it in a function, or — if a function is not feasible — a macro. That is exactly what pin! is doing. Here is an attempt:

macro_rules! declare_pin_thread_local_cell {
    (_: $type:ty = $initializer:expr; fn $func:ident;) => {
        fn $func<R>(f: impl FnOnce(::core::pin::Pin<&mut $type>) -> R) -> R {
            // We declare this constant in the outer scope so that RAW_STATIC cannot be
            // mentioned by $initializer.
            const INITIALIZER: $type = $initializer;

            let mut_ref = &mut *{
                #[thread_local]
                static RAW_STATIC: ::core::cell::RefCell<$type> =
                    ::core::cell::RefCell::new(INITIALIZER);
                RAW_STATIC.borrow_mut()
            };

            // SAFETY:
            // * This code does not move RAW_STATIC.
            // * It is impossible for any other code to access RAW_STATIC,
            //   because RAW_STATIC is enclosed in a block with no outside code.
            // Therefore, we may soundly declare that it is pinned.
            f(unsafe { ::core::pin::Pin::new_unchecked(mut_ref) })
        }
    };
}

declare_pin_thread_local_cell!(
    _: future::TheFuture = future::run_main();
    fn with_future;
);

fn poll_task() {
    with_future(|f| {
        let outcome = f.poll(&mut task::Context::from_waker(task::Waker::noop()));
        assert!(outcome.is_pending());
    });
}
1 Like