The trait `Unpin` is not implemented for `GenFuture` (error when using `join_all`)


#1

I’m playing with futures-preview and Rust nightly and I can’t figure out one thing. I’m trying to run an async function multiple times and join the result with the join_all function, so essentially something like that:

async fn foo() {
}

// ... somewhere in the code
let mut futures = Vec::new();
futures.push(foo());
await!(join_all(futures));

(the full code is here)

I get the following error:

error[E0277]: the trait bound `std::future::GenFuture<[static generator@src/main.rs:121:31: 139:2 {}]>: std::marker::Unpin` is not satisfied in `impl futures::Future`
   --> src/main.rs:151:13
    |
151 |             await!(join_all(futures));
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^ within `impl futures::Future`, the trait `std::marker::Unpin` is not implemented for `std::future::GenFuture<[static generator@src/main.rs:121:31: 139:2 {}]>`
    |
    = help: the following implementations were found:
              <std::future::GenFuture<T> as std::marker::Unpin>
    = note: required because it appears within the type `impl futures::Future`
    = note: required because it appears within the type `impl futures::Future`
    = note: required because of the requirements on the impl of `futures::Future` for `futures::future::JoinAll<impl futures::Future>`

I still don’t fully understand Pin and Unpin, so I might be missing something, but is there any way to make this code work?

From what I understand std::future::GenFuture is explicitly marked as !Unpin because async/await futures are immovable and in turn join_all requires an Unpin future, so it seems that what I’m trying to do is not doable. But that raises another question - if a future returned by running an async function is always !Unpin, is there any good way to implement something like join_all with !Unpin futures? I’d try it myself, but I’m worried that I’ll waste a lot of time for something that’s not possible.


#2

I’ve read a bit more about the subject and I think I have a bit better understanding on how various pieces work, but it doesn’t make any sense when put together, maybe someone more experienced will shed some more light onto it. As far as I understand, a pinned pointer is a pointer that either can’t be moved or implements Unpin (ie. is safe to be moved). Which also means that everything can be pinned, but after it’s been pinned it can’t be moved unless it’s safe.

Now, I also think that the reason that GenFuture is !Unpin is that the structure generated by compiler when using async can’t be moved. And OK, that’s reasonable, and then it also makes sense that the poll function is implemented only on pinned future.

Now I’m trying to change how the join_all works and in order to start from a simpler code, I tried to clean up the poll implementation. With the following implementation:

impl<F> Future for JoinAll<F>
where
    F: Future
{
    type Output = Vec<F::Output>;

    fn poll(
        mut self: Pin<&mut Self>,
        lw: &::std::task::LocalWaker,
    ) -> Poll<Self::Output> {
        let mut all_done = true;

        for elem in self.as_mut().elems.iter_mut() {
            match elem {
                ElemState::Pending(ref t) => {
                    Pin::new(t);
                },
                ElemState::Done(ref mut _v) => (),
            };
        }

        Poll::Pending
    }
}

I get the error:

or[E0277]: the trait bound `F: std::marker::Unpin` is not satisfied
  --> src/join_all.rs:91:21
   |
91 |                     Pin::new(t);
   |                     ^^^^^^^^ the trait `std::marker::Unpin` is not implemented for `F`
   |
   = help: consider adding a `where F: std::marker::Unpin` bound
   = note: required by `<std::pin::Pin<P>>::new`
error: aborting due to previous error

And here my understanding finishes, ie. I’m not sure how could I call poll on the GenFuture at all if Pin::new requires a rerference to be Unpin. I’m either missing something or it requires unsafe code maybe?


#3

The key here is that Pin<Box<F>>: Future + Unpin where F: Future, basically you put the GenFuture into a pinned heap allocation so that even though the Pin<Box> you pass into the JoinAll may be moved if the storage is reallocated the underlying heap allocation isn’t. There’s a combinator for this so it should be as simple as:

let mut futures = Vec::new();
futures.push(foo().boxed());
await!(join_all(futures));

One interesting note is that JoinAll should never need to actually reallocate the underlying storage. So it should be possible to change it to store a Pin<Box<[F]>> instead and use a little unsafe code to pin project into that field, I’ve been meaning to take a look at doing that at some point. In that case you would just have a !Unpin JoinAll instance which await! can handle, and you’ll avoid the overhead of the extra heap allocations. (EDIT: Although commonly you will be wanting to join multiple types of futures so will end up boxing them anyway, hopefully there will be a join! macro at some point that allows joining multiple disparate types without having to heap allocate them individually).


#4

Thanks a lot, that indeed fixed my problem! At some point I tried to box the future to make it a pointer with Box::new(), but that couldn’t work as I see now. I can’t find the boxed() combinator in the nightly docs, though, is there any place where I could read about it?

For reference, I uploaded the updated version of the code if someone is interested in a full thing.


#5

It’s FutureExt::boxed.


#6

Thanks! For some reason I thought it’s from the std lib.

I’m really excited by all of this :smiley: I’m still very new to Rust, so I have a hard time understanding some of the concept, especially from nightly that are still being changed, but I find programming in Rust very refreshing and I find it extremely efficient with the async/await (minus parts where I totally don’t understand what the compiler is telling me, but I hope it will come with experience).