# Desugaring Async Fn

**URL:** <https://users.rust-lang.org/t/desugaring-async-fn/63698>\
**Category:** help\
**Created:** [August 17, 2021, 5:43pm UTC](https://users.rust-lang.org/t/desugaring-async-fn/63698 "2021-08-17T17:43:08Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ambiso](https://sea1.discourse-cdn.com/flex019/user_avatar/users.rust-lang.org/ambiso/32/10843_2.png) [@ambiso](https://users.rust-lang.org/u/ambiso)\
**Post date:** [August 17, 2021, 5:43pm UTC](https://users.rust-lang.org/t/desugaring-async-fn/63698/1 "2021-08-17T17:43:08Z")

</div>

Hi,

I'm trying to desugar an `async fn` and am running into some lifetime problems that I don't know how to solve.

[Here's](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=be53f9075291eea6dafdb449af71671a) my code.

This is the function that I want to desugar:

```rust
async fn read_file(file: &mut File) -> String {
    let mut v = Vec::new();
    file.read_to_end(&mut v).await.unwrap();
    String::from_utf8(v).unwrap()
}

```

I'm desugaring it into a struct that contains the `Vec` and a reference to the `File` passed as an argument.  
Furthermore there's a state enum that is supposed to be updated to contain the `ReadToEnd` Future. However, I can't seem to figure out how to store the Future in the state.

I think intuitively I want to say that the `'b` in `Pin<&'b mut Self>` (`'b` is not explicit in the code) must live as long as `'a` but I can't constrain the lifetime further.

What am I missing?

Best,  
ambiso

---

<div class="post-metadata">

**Author:** ![Yandros](https://sea1.discourse-cdn.com/flex019/user_avatar/users.rust-lang.org/yandros/32/8136_2.png) [@Yandros](https://users.rust-lang.org/u/Yandros)\
**Post date:** [August 17, 2021, 5:51pm UTC](https://users.rust-lang.org/t/desugaring-async-fn/63698/2 "2021-08-17T17:51:38Z")

</div>

> [@ambiso](#):
>
> What am I missing?

> ```rust
> // The `async fn` I'm trying to desugar
> async fn read_file(file: &mut File) -> String {
> let mut v = Vec::new();
> file.read_to_end(&mut v).await.unwrap();
> String::from_utf8(v).unwrap()
> }
> 
> ```

So the returned future starts capturing a `&'file mut File` "upvar", which points / refers to an external / non-self-referential element, hence why it can be named with a proper lifetime parameter.

But the state at the first `.await` point, the whole future must now contain:

- still the initial `&'file mut File`,

- `v: Vec<u8>`,

- `&mut v: &'?? mut Vec<u8>`

- `ReadToEndFuture<'file, '??>`, the future returned by `file.read_to_end(&mut v)`.

As you can see, there is an issue with `'??`: this is actually a self-reference to the `v` field, which is why:

- this future will not be `Unpin`;

- the automagically compiler-generated code will "use `unsafe`" to make the borrow checker look the other way _w.r.t._ `'??`, since by the safety contract of `Pin<&'_ mut Self>` (the receiver of the `Future::poll` method (the _only_ way to make the state machine evolve and to reach the code using `unsafe` to use the self-reference)), we know that in between `.poll()`s, the location of the whole future shall not have changed.

Now, you can try to write this state machine yourself, but the presence of this self-reference means you'll necessarily need to write `unsafe` yourself.

* * *

```rust
async fn read_file (file: &mut File)
  -> String
{ // <- State0
    let mut v = Vec::new();
    file.read_to_end(&mut v)
        .await // <- State1
    .unwrap();
    String::from_utf8(v).unwrap()
} // <- State2

```

The state machine would be something like:

```rust
use ::core::{
    future::Future,
    marker,
    mem::{MaybeUninit as MU},
    ptr,
};

type ReadToEndFuture<'file, 'v> = impl Future<Output = ::std::io::Result<()>>;
fn read_to_end<'file, 'v> (
    file: &'file mut File,
    v: &'v mut Vec<u8>,
) -> ReadToEndFuture<'file, 'v>
{
    file.read_to_end(v.as_mut())
}

enum ReadFileFuture<'file> {
    State0 {
        file: &'file mut File,
    },
    State1 {
        file: &'file mut File,
        file_reborrowed: MU<&'unsafe_self_ref_file mut File>,
        v: Vec<u8>,
        at_v_mut: MU<&'unsafe_self_ref_v mut Vec<u8>>,
        file_read_to_end: MU<ReadToEndFut<'unsafe_self_ref_file, 'unsafe_self_ref_v>>,
        _self_referential: marker::PhantomPinned,
    },
    State2 {
        return_value: String,
    },
    Poisoned,
}

impl<'file> Future for ReadFileFuture<'file> {
    type Output = String;

    fn poll (self: Pin<&'_ mut ReadFileFuture<'file>>, cx: &'_ mut Context<'_>)
      -> Poll<Self::Output>
    { unsafe {
        let this = Pin::get_mut_unchecked(self);
        loop { break match *this {
            | Self::State0 { ref file } => {
                let file = ptr::read(file); // This is `ManuallyDrop::take()`.
                ptr::write(this, Self::Poisoned);
                ptr::write(this, Self::State1 {
                    file,
                    file_reborrowed: MU::uninit(),
                    v: Vec::new(), // <- Function's body
                    at_v_mut: MU::uninit(),
                    file_read_to_end: MU::uninit(),
                    _self_referential: <_>::default(),
                });
                if let Self::State1 { file, file_reborrowed, v, at_v_mut, file_read_to_end, .. } = this {
                    let file_reborrowed = file_reborrowed.write(&mut **file); // <- Function's body
                    let at_v_mut = at_v_mut.write(v); // <- Function's body
                    let _ = file_read_to_end.write(
                        read_to_end(file_reborrowed, at_v_mut) // <- Function's body
                    );
                    continue;
                } else {
                    ::std::hint::unreachable_unchecked()
                }
            },
            | Self::State1 { ref mut file_read_to_end, ref v, .. } => {
                let result = ::futures::ready!(unsafe { Pin::new(file_read_to_end) }.poll(cx));
                let file_read_to_end = ptr::read(file_read_to_end);
                let v = ptr::read(v);
                ptr::write(this, Self::Poisoned);
                result.unwrap(); // <- Function's body
                drop(file_read_to_end); // <- Function's body
                let return_value = String::from_utf8(v).unwrap(); // <- Function's body
                ptr::write(this, Self::State2 { return_value });
                continue;
            }, // I now realize the `State2` could be elided altogether…
            | Self::State2 { ref return_value } => {
                let return_value = ptr::read(return_value);
                ptr::write(this, Self::Poisoned);
                Poll::Ready(return_value)
            },
            | Self::Poisoned => panic!("Violation of `Future::poll`'s contract: future polled _again_ after completion or panic"),
        }}
    }}
}

```

I've tried to write not too awful code, safety-wise (even though I've sacrificed some "pattern consistency" by taking shortcuts abusing knowledge of some stuff (_e.g._, that the only drop glue in `State1` was `v` and the inner future)), although the above code would upset `miri` very much (note: current futures already do upset miri). In order to do this fully correctly, raw pointers ought to be used instead of Rust references _every_ time an `'unsafe_self_ref…` lifetime appears in the type definition.

---

<div class="post-metadata">

**Author:** ![ambiso](https://sea1.discourse-cdn.com/flex019/user_avatar/users.rust-lang.org/ambiso/32/10843_2.png) [@ambiso](https://users.rust-lang.org/u/ambiso)\
**Post date:** [August 17, 2021, 6:12pm UTC](https://users.rust-lang.org/t/desugaring-async-fn/63698/3 "2021-08-17T18:12:20Z")

</div>

Thanks so much for the quick reply!

> this future will not be `Unpin` ;

I'm aware; I actually explicitly marked it

```rust
    _pin: PhantomPinned,

```

> As you can see, there is an issue with `'??` : this is actually a self-reference to the `v` field

Thanks, for some reason I did not see that it was actually a self-reference, I thought the `'a` was sufficient, but I see now that it's an external lifetime independent of the lifetime of the `Vec`.

---

<div class="post-metadata">

**Author:** ![ambiso](https://sea1.discourse-cdn.com/flex019/user_avatar/users.rust-lang.org/ambiso/32/10843_2.png) [@ambiso](https://users.rust-lang.org/u/ambiso)\
**Post date:** [August 17, 2021, 7:41pm UTC](https://users.rust-lang.org/t/desugaring-async-fn/63698/4 "2021-08-17T19:41:08Z")

</div>

[Here's](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=4b43b6cd649e881523c358247d2fdd64) my version using `mem::transmute`.

I can't say whether it's safe usage of transmute (most likely it is not), but it seems to execute fine at least 🙂

---

<div class="post-metadata">

**Author:** ![system](https://sea1.discourse-cdn.com/flex019/user_avatar/users.rust-lang.org/system/32/51372_2.png) [@system](https://users.rust-lang.org/u/system)\
**Post date:** [November 15, 2021, 7:41pm UTC](https://users.rust-lang.org/t/desugaring-async-fn/63698/5 "2021-11-15T19:41:42Z")

</div>

Closed on November 15, 2021, 7:41pm UTC

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.