High level: pin, async, future

1

At a very high level: what is the issue that we run into with async/future where Pin is the answer ? Every time I unwrap an async impl, Pin seems to show up, but it is not clear to me why we need it and what it is for.

2

Basically it's because the future objects that async/await syntax generate are going to contain pointers that point into other parts of the same future object. If the future object is moved, those pointers would still point at the future's old address, and next time you polled the future, it would access the wrong memory locations.

For example, consider this:

async fn my_fn() {
    let my_var = 10;
    let my_ref = &my_var;
    
    tokio::task::yield_now().await;
    
    println!("{my_ref}");
}

Here, when you call poll on that future object, the first call to poll will create the my_var and my_ref values, but then it has to return due to the yield in yield_now. This is handled by storing the local variables inside the future object, so they are ready next time the future is polled. However, storing both my_var and my_ref in the future object results in the situation I described.

Pin is the answer because it prevents moving the future object once the future has been polled, ensuring that internal references remain valid.

4 Likes
3

There’s some IMO pretty nice explanations in this video (starting from the linked timestamp for async/await & pinning in particular, but the other contents are quite interesting, too): The What and How of Futures and async/await in Rust - YouTube. (Note that the thing is recorded live and the Rust code created there is to a certain degree only pseudo code for illustration purposes and can contain mistakes if you look at the details.)

1 Like
4

@alice: Thank you for the informative example. Here is something else I do not understand (and perhaps you address it with the last line).

Why are some futures Pinned and some not ? If it a matter of (1) they use the stack or (2) whether they have been polled or ... something else ?

5

Well, there are two different ways you might interpret "not pinned".

  1. The future implements the Unpin trait.
  2. The future has not yet been polled.

In the first case, this generally means that the type never has internal references, and that it is always safe to move it. It applies mostly to when someone manually implements the Future trait. The async/await syntax never generates futures in this category.

The second category refers to the situation where the future has not yet been polled. In this case, the future can be moved around safely. This generally makes sense because async/await is lazy. E.g. with the my_fn from my previous example, if you haven't polled it yet, then it has not yet created my_ref, and so moving it wont break my_ref.

2 Likes
6

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.