Pin projection with Arc (and other std memory owning types)

Greetings everyone,
I am quite stuck trying to create a particular async tasks queue. The main issue is related to project a struct containing a pinned arc future in order to forward the Future::poll function. I am not sure if I am trying to do something very wrong or if we still miss some pieces from std when working with pinned objects.

I have the following:

#[derive(Debug)]
struct QueuedFuture<F> {
    future: F,
    waker: AtomicWaker,
}

#[derive(Debug)]
pub struct Queued<'a, F> {
    inner: Pin<Arc<QueuedFuture<F>>>,
    _phantom: PhantomData<&'a ()>,
}

impl<'a, F> Future for Queued<'a, F>
where
    F: Future,
{
    type Output = F::Output;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        todo!()
    }
}

Playground

The main idea is that I want to be able to get a Queued from QueuedFuture::enqueue and then either:

  • queued.await
  • queued.detach()

In the first case when a future is extracted from the queue (a unbounded channel, to be precise), it must be left alone in order to let the original function await on it. In the second case the original function will consume the future, therefore the queue will await the future. This is interesting because there are some operations that return a result, in other cases it is just a matter of executing the async function when there is a valid situation.

Now, here my problem: if I didn't need the Arc, I would have used pin-project to project from Pin<&mut Queued> to a Pin<&mut QueuedFuture> and then call queued_future.poll(cx).

However, just because I need to keep the future shared between Queued and the queue (ThrottledQueue in the playground), I need to use an Arc. What I am missing is Arc::get_pin_mut(self: &mut Pin<Self>) -> Option<Pin<&mut T>>, and I am not sure if I can actually write it and if the function would be sound.

Finally, my two questions:

  • Is an hypothetical Arc::get_pin_mut function sound? I don't see why it shouldn't be, but most people in this forum are smarter than me, therefore I want to ask :wink:
  • If the function is sound, is it possible to implement outside std without footguns? The reason I am having troubles about this is that Pin::as_mut is pretty similar to what I would write, but it dereferences the pointer internally, and I cannot do direcly with the Pin interface. I came up with this, but I really think that there are too many unsafe shenanigans... I expect a shoot in my feet.
fn get_pin_mut<T>(this: &mut Pin<Arc<T>>) -> Option<Pin<&mut T>> {
    unsafe {
        let this_ptr = this as *const Pin<Arc<T>>;
        // Is this needed to avoid mut ref aliasing?
        drop(this);

        let pinned = ptr::read(this_ptr);
        let arc = Pin::into_inner_unchecked(pinned);
        let result = Arc::get_mut(&mut arc).map(|inner| Pin::new_unchecked(inner));

        // Don't drop the Arc
        mem::forget(arc);
        result
    }
}

What do you think? Am I totally wrong with my assumptions? Should this function exist? Do you have a better solution? I would really like a safe (as without unsafe) implementation, but I did not came up with anything for now.

In this particular case, you don't even really need a pin-projection. You can just mark your struct unpin and use new_unchecked to create a pinned pointer. This is because the future is not stored directly in the struct, but on the heap, so moving the struct doesn't move the future. Of course this assumes that you're not moving the future out of the Arc.

Your main problem here is actually that you can't create mutable references into an Arc unless you verify that there is only one Arc, but in that case you should just use a Box.

1 Like

This is what I initially thought, but... the fact that Arc::pin creates a Pin<Arc<T>> and not a Arc<Pin<T>> made me think. I know that I could just use an Arc<T> because I don't have T on the stack, but the whole reason behind Pin is that mem::replace exists (and lots of other things, obviously). You are saying that I can avoid pinning Queued::inner until I have to poll? Maybe it is just me being too paranoid when try to build this kind of abstraction. I will give a try to see if, without a single unsafe, I can avoid a premature pin.

But that's the point of using Arc::get_mut! I want to get the mutable future only when I am the unique owner, the fact is that I don't know who the unique owner will be a priori.

Thank you @alice, I think you are right that is more simple than I thought. I just had to add the bound for the inner future F: Unpin, which should be fine most of the time... right?

Just for the sake of information, this what I did:

#[derive(Debug)]
pub struct Queued<'a, F> {
    inner: Arc<QueuedFuture<F>>,
    _phantom: PhantomData<&'a ()>,
}

impl<'a, F> Future for Queued<'a, F>
where
    F: Future + Unpin,
{
    type Output = F::Output;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let queued_future = &mut self.get_mut().inner;

        match Arc::get_mut(queued_future) {
            Some(QueuedFuture { future, .. }) => {
                let future = Pin::new(future);
                future.poll(cx)
            }
            None => {
                queued_future.waker.register(cx.waker());
                Poll::Pending
            }
        }
    }
}

If anyone has more thoughts or comments feel free to continue discussing down here :blush: