Non-anonymous async function state

Hi,

I am wondering if anybody have thought of any approaches to writing futures (and/or streams) in async/await syntax yet with their state being observable (hence, non-anonymous)?

In my particular case, I need the underlying state of a stream to be serializable so I have to implement polling function and manage the state machine explicitly. This, however, comes at a rather significant cost of mental overhead and complexity (case in point: bpxe/mod.rs at master · bpxe/bpxe · GitHub). It feels like if I was able to write the entire piece in async/await fashion, it'll be far easier to manage it.

So I've tried to imagine a proc-macro that would convert async functions into explicitly typed state, and hand-coded a rough approximation of what the input and generated code can look like (see the code snippet below).

Are there any other approaches that can be taken to solve this problem? Am I overthinking or overlooking something?

Thanks!

#![allow(non_camel_case_types)]

// The code below is a VERY VERY VERY VERY LOOSE AND HAND-WAVY APPROXIMATION
// of what might be generated for this snippet:
//
// ```
// use serde::*
// use tokio::task;
//
// #[asyncify(methods(test))]
// #[derive(Serialize, Deserialize)]
// struct Test {
//   something: u8,
// }
//
// #[asyncify]
// impl Test {
//   async fn test(&mut self) -> u8 {
//      let counter = self.something + 1;
//      task::yield_now().await;
//      something = counter;
//      counter
//   }
// }
// ```
//

use serde::*;
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
use tokio::task;

// Note how this is serializable
#[derive(Serialize, Deserialize)]
struct Test {
    something: u8,
    asyncify_test: Option<Asyncify_Test_test>,
}

// Note how this is serializable
#[derive(Serialize, Deserialize)]
enum Asyncify_Test_test {
    Start,
    Yield1 { counter: u8 },
    End(u8),
}

enum Asyncify_Test_test_yield {
    Yield1(Pin<Box<dyn Future<Output = ()>>>, Option<()>),
}

impl Test {
    fn test(&mut self) -> impl Future<Output = u8> + '_ {
        self.asyncify_test = Some(Asyncify_Test_test::Start);
        Asyncify_Test_test_future(self, None)
    }
}

pub struct Asyncify_Test_test_future<'a>(&'a mut Test, Option<Asyncify_Test_test_yield>);

impl<'a> Future for Asyncify_Test_test_future<'a> {
    type Output = u8;

    fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        match self.as_mut().1.take() {
            None => {}
            Some(Asyncify_Test_test_yield::Yield1(mut y, _)) => match y.as_mut().poll(cx) {
                Poll::Pending => {
                    self.as_mut().1 = Some(Asyncify_Test_test_yield::Yield1(y, None));
                    return Poll::Pending;
                }
                Poll::Ready(v) => {
                    self.as_mut().1 = Some(Asyncify_Test_test_yield::Yield1(y, Some(v)));
                }
            },
        }
        match self.as_mut().0.asyncify_test.take() {
            None => panic!(),
            Some(Asyncify_Test_test::Start) => {
                self.as_mut().0.asyncify_test = Some(Asyncify_Test_test::Yield1 {
                    counter: self.as_ref().0.something + 1,
                });
                cx.waker().wake_by_ref();
                Poll::Pending
            }
            Some(Asyncify_Test_test::Yield1 { counter }) => {
                match self.as_mut().1.take() {
                    None => {
                        self.as_mut().1 = Some(Asyncify_Test_test_yield::Yield1(
                            Box::pin(task::yield_now()),
                            None,
                        ));
                        self.as_mut().0.asyncify_test =
                            Some(Asyncify_Test_test::Yield1 { counter });
                    }
                    Some(Asyncify_Test_test_yield::Yield1(_, Some(()))) => {
                        self.as_mut().0.asyncify_test = Some(Asyncify_Test_test::End(counter));
                        self.as_mut().0.something = counter;
                    }
                    Some(y) => {
                        self.as_mut().1 = Some(y);
                        self.as_mut().0.asyncify_test =
                            Some(Asyncify_Test_test::Yield1 { counter });
                    }
                }

                cx.waker().wake_by_ref();
                Poll::Pending
            }
            Some(Asyncify_Test_test::End(v)) => Poll::Ready(v),
        }
    }
}

// And this is our test

#[tokio::main]
async fn main() {
    let mut test = Test {
        something: 0,
        asyncify_test: None,
    };

    println!("{}", test.test().await);
    println!("{}", test.something);
}

(Playground)

Output:

1
1

Errors:

   Compiling playground v0.0.1 (/playground)
    Finished dev [unoptimized + debuginfo] target(s) in 2.38s
     Running `target/debug/playground`

I don't immediately see the best way to proceed here, but I strongly recommend taking some time to think if you can come up with a method that avoids this approach.

Thanks for your feedback!

I mean, if I can generate this by hand, then perhaps it's not entirely impossible to get there with a procedural macro.

I am wondering, though, why would you avoid this approach -- any specific concerns?

I've tried to think of other ways to do this but I wouldn't be here asking this question if I found one :slight_smile:. To me, the above seem like a potentially feasible approach and I'd love to learn more about what I might be overlooking.

Well, it's just going to be a complicated way to approach the problem. Approaching other solutions would require knowing more about what you need the serialization to do for you.

No, it's not impossible. The Rust compiler is able to do it.

Well, I am effectively looking for futures that can be resumed after a cold restart. Currently I'm doing this by maintaining a state machine and a polling function manually. I find it error-prone and tedious. That's why I thought just investing some time into a more generalized approach to state machine maintenance can be useful (hence the prototype above).

I know it's not an easy task to develop a procedural macro to do this transformation. But is there anything else worth considering? Genuinely curious.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.