How wakers get associated with particular future

[jgarvin]

The async book has an example of a Join future that joins 2 subtasks. It uses the simplified Future trait that just takes a wake: fn(). When the real Future definition is introduced, they explain why this becomes &mut Context<'_>:

Secondly, wake: fn() has changed to &mut Context<'_>. In SimpleFuture, we used a call to a function pointer (fn()) to tell the future executor that the future in question should be polled. However, since fn() is just a function pointer, it can't store any data about which Future called wake.

In a real-world scenario, a complex application like a web server may have thousands of different connections whose wakeups should all be managed separately. The Context type solves this by providing access to a value of type Waker, which can be used to wake up a specific task.

This made sense to me and made me expect something: that Context or Waker would require you to pass the Future's self into some method in order to link the two together. This way if you implement Join, you give the first subtask a context/waker that knows to wake up the first subtask and you give the second subtask a context/waker that knows to wake up the second subtask. But this never happens.

In the next section of the book they show how you use waker:

shared_state.waker = Some(cx.waker().clone());

Nowhere do we associate the current future with the waker. My first thought was maybe this somehow happens earlier, e.g. when the Context is created maybe its already tied to the particular Future. But the only way to create a Context is from a Waker. Waker can only be created from RawWaker... which holds executor specific raw data. Does an efficient Join require executor specific code? Not hugely important if Join is only on two things, but if it were a vector of 1000 things it would obviously matter a lot.

[kpreid]

The way I would recommend thinking about this is that the basic use of wakers is to wake up tasks, not futures. A task is what is created by handing off a future to the executor, e.g. via a spawn() function. The key thing to notice here is that there is no need to identify the future value; it's just “whatever future we created this task with”.

When the executor creates a task, it can create a Waker specific to that task, and pass that particular Waker to the task when polling it. Thus, when the Waker is invoked, it knows which task to wake (poll() again).

---

All of that is executor-internal logic, and opaque to the Futures. However, individual futures can implement an efficient join. In particular, futures::stream::FuturesUnordered and FuturesOrdered already do that. The way they do this is creating their own wakers: when a FuturesUnordered is polled, it uses a unique Waker to poll each of its children. That way, if and when that Waker is invoked, the FuturesUnordered knows which child to poll again (after getting itself woken by invoking the Waker the executor provided to it).

But there is some overhead to all that bookkeeping, so whether it is worth doing that will depend on how many futures you're joining and how cheap they are to poll. It's entirely reasonable to not bother for simple cases like polling exactly two child futures.