Can I represent the lifetime between two awaits? (For actor coroutines)

I'm trying to add actor coroutines to my Stakker crate, building on top of async/await rather like genawaiter. I'm trying to do this cooperatively with Rust's borrow checker. I think I've boiled the problem down to needing to represent a borrow that only lasts until the next yield (i.e. next .await). Can I represent this lifetime somehow?

Here's a more in-depth explanation of what I'm trying to do. In Stakker, each actor method receives a reference to the actor state structure &mut Self and a reference to a context for various other actor operations &mut Cx<'_, Self>. Using qcell crate, these are based on borrows from the Stakker structure further up the stack. So all access to actor state is validated by the borrow checker.

Now I want to add actor coroutines. The idea being that within the coroutine, the code would also have access to the same two references, &mut Self and &mut Cx<'_, Self>. So behind the scenes, looking at it from the stack point of view, an actor method is called from Stakker core, which then calls into the coroutine to cause it to advance to its next yield. This actor method has access to the two references, and needs to pass them up into the coroutine for it to use, and when the coroutine returns (when it yields, i.e. awaits) those references become invalid. This is because for one thing Cx is a pointer into the stack, and the other thing, they're based on borrows which need to be released for the borrow checker to be happy.

So looking at this from the coroutine's point of view (which would be implemented as an async function or block), it will have two references (this and cx) which get updated every time it yields (i.e. awaits). As far as I can see, there's no way for me to get these lifetimes through the Future poll system, so I can't have things end-to-end borrow-checked, but that is okay so long as I can make it safe for the user. But then comes the problem. Even if I get the references into the coroutine code via unsafe pointers, how do I tell Rust within the coroutine that those references will become invalid at the next await? It seems to me that I need some kind of a lifetime to represent it. Or else there might be ways for references to leak, and Rust's borrow checking to have been circumvented.

Here are some possibilities I've looked at:

  • Passing back the references from the Future that I await on to do the yield. However you just can't attach lifetimes to the Output of a Future.

  • Having a CxThis structure behind an Rc that gives access to the two references so that the coroutine can get hold of them, but then what lifetime should those references have if it returns them? (Being behind an Rc is not ideal, but getting something working and correct is more important than optimisation right now.)

  • Having a CxThis as above which executes closures like|this, cx| {...}), so that the lifetimes can be limited and controlled. This is not at all ergonomic, and it's not going to look like any of the other actor code in Stakker, but it should work. This wouldn't be an acceptable solution unless perhaps I do some proc macro magic to hide it all. If there's a better way than this I'd much rather avoid it.

All through Stakker I've been trying to work with the borrow checker rather than fight it, and let it shape the solution, and it has worked out very well so far. (I will write more on this at some point.) But I can't see how to extend this borrow checking up into async/await code. Any ideas?

I don't think I fully understand your situation, but the one way I can think of to better control yield points is a custom future type.

You then implement the poll method and might make data available in that poll function from the struct.

Other than that, I don't think you can have any control. If you spawn a future, the scheduler is the one that starts it when the waker is ready. It really depends what that waker is. Do you control it? Does it come from IO? You could have types where you can move in and out data, like a shared Option.

Just before waker.wake(), you move the needed data into the Option and the future can take it out, use it and then put it back before yielding.

In any case, it doesn't sound like it will be possible without a lock.

Yes, I control everything, as I'm taking an approach like genawaiter. So I'm not using the Waker mechanism at all. I have my own Future implementation to .await on to do a yield. When there is data to yield from that Future, I'm calling poll directly on the async block's Future to advance it to the next await to implement a coroutine 'resume'. So this is using async/await just for the coroutine-like behaviour. It's all legal though. Effectively it's a mini scheduler just to provide coroutine-like functionality. This part is all exactly the same concept as genawaiter.

Anyway despite controlling everything, I still can't see how to get my borrows and lifetimes through to the coroutine (i.e. the async block) and have them behave safely, even if I use unsafe code for the glue. Maybe there is a way to arrange things that I've not realized, though. That's why I'm putting the question out there to see if anyone has another angle on this that might help.

Maybe I need to ask on internals.

It occurs to me, looking at the "generators" unstable feature in the Unstable book, that if I had access to the translated source like in that example, I could easily modify it to add my extra references to the resume function, and it would all borrow-check like normal, i.e. the lifetime that I'm asking for actually exists right there in that function, no problem.

For future reference, there's already an issue for the generator feature that I need.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.