Implementing lazy error contexts across await points

1

I would like to be able to implement an API surface similar to this:

pub async fn test_api() {
    let mut resp = Response;
    let result = may_fail().with_failure_context(|&mut resp| {}).await;
}

What I'm trying to do is to say, if may_fail fails, we will perform some function that mutates the Response, and then be able to halt control flow with ?. It is meant to be similar to context as provided by the anyhow crate.

My implementation looks something like this:

pub struct Response;

pub trait IntoApplicationResponse {}

impl IntoApplicationResponse for () {}

pub trait WithFailureContext<T, E>: Future<Output = Result<T, E>> {
    fn with_failure_context<'lt, F>(self, f: F) -> WithFailureContextFuture<'lt, Self, F>
    where
        F: FnOnce(&'lt mut Response),
        Self: Sized;
}

impl<T, E, U: Future<Output = Result<T, E>>> WithFailureContext<T, E> for U {
    fn with_failure_context<'lt, F>(self, f: F) -> WithFailureContextFuture<'lt, Self, F>
    where
        F: FnOnce(&'lt mut Response),
        Self: Sized,
    {
        todo!()
    }
}

#[pin_project]
pub struct WithFailureContextFuture<'r, Fut, F> {
    #[pin]
    fut: Fut,
    response: &'r mut Response,
    context: Option<F>, // Option so we can take it out on poll
}

impl<'r, T, E, Fut, F> Future for WithFailureContextFuture<'r, Fut, F>
where
    Fut: Future<Output = Result<T, E>>,
    E: IntoApplicationResponse,
    F: FnOnce(&mut Response),
{
    type Output = Result<T, ()>;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let this = self.project();
        match this.fut.poll(cx) {
            Poll::Pending => Poll::Pending,
            Poll::Ready(Ok(v)) => Poll::Ready(Ok(v)),
            Poll::Ready(Err(_)) => {
                if let Some(f) = this.context.take() {
                    f(this.response);
                }
                Poll::Ready(Err(()))
            }
        }
    }
}

However, Rustc errors out by saying:

error: implementation of `FnOnce` is not general enough
  --> src/lib.rs:14:66
   |
14 |     let result = may_fail().with_failure_context(|&mut resp| {}).await;
   |                                                                  ^^^^^ implementation of `FnOnce` is not general enough
   |
   = note: closure with signature `fn(&'2 mut Response)` must implement `FnOnce<(&'1 mut Response,)>`, for any lifetime `'1`...
   = note: ...but it actually implements `FnOnce<(&'2 mut Response,)>`, for some specific lifetime `'2`

I would like to know whether this is a general limitation of the borrow checker, and maybe this is just something that cannot be implemented without unsafe, or my approach to the problem is wrong.

2

this is just a variant of FutureExt::map(), it's definitely possible.

your problem is you made a mistake, there's discrepancy between the WithFaluireContext::with_failure_context() and the impl Future for WithFailureContextFuture {} - the bound for F is different:

in the trait definition, its where F: FnOnce(&'lt mut Response), where in the impl Future, it's F: FnOnce(&mut Response).

remove the 'lt annotation should make this compile. I think you want a higher ranked lifetime for the FnOnce() bound, instead of a specific lifetime.

btw, I'm not sure I understand what you are trying to do here:

where did this response: &'r mut Response field come from? you put a todo!() in the blanket implementation of with_failure_context(). storing a &mut reference in a struct usually lead to tricky errors if you don't understand the impliciation of invariance.

3

Thank you so much for your reply. I apologize for how tardy my follow up is. I'd given up on people replying to the post.

remove the 'lt annotation should make this compile. I think you want a higher ranked lifetime for the FnOnce() bound, instead of a specific lifetime.

Yes, I removed the lifetime annotation, but it breaks everything. I think what I'm trying to achieve (and not succeeding at it) is that I'm trying to perform a very specific action tailored for a very specific thing. I want the with_failure_context to take a mutable reference to a Response, and mutate it according to the caller. I thought that it was necessary to store the mutable reference to the Response in the future. However, I must admit that I'm not quite sure what to do here to make it work.

4

Who owns the Response that should be mutated?

I think @nerditation's answer assumes that it is the return value of the inner future. Your usage example suggests that it is intended to be a local variable belonging to the caller, but you don't actually pass a reference to that variable to with_failure_context, nor does the signature of with_failure_context accept such a reference.

So: who owns the Response?

5

It is meant to be owned by the caller. The sample usage here is:

pub async fn test_api() {
    let mut resp = Response;
    let result = may_fail().with_failure_context(|&mut resp| {}).await;
}

So, response is meant to be owned by the caller, and it's assumed that it will live longer than the future it is passed to. What I'm trying to express to borrowck is that we would like for it to enforce this, if possible.

6

|&mut resp| {} doesn't mean "pass &mut resp to some closure", it means "create a closure that takes a &mut _Inferred, and then copies out the value of type _Inferred from behind the reference".

Like how this works:

fn example_0(&mut binding_name: &mut i32) {
    let _type_example: i32 = binding_name;
}

And this fails.

fn example_1(&mut binding_name: &mut String) {
}

error[E0507]: cannot move out of a mutable reference
  --> src/lib.rs:62:14
   |
62 | fn example_1(&mut binding_name: &mut String) {
   |              ^^^^^------------
   |                   |
   |                   data moved here because `binding_name` has type `String`, which does not implement the `Copy` trait
   |

Also, let's say you change the call to something like this:

let result = may_fail().with_failure_context(|resp/*: &mut Response */| {}).await;

There's no connection between the local resp variable and the argument to the closure. You're giving the closure away to the body of with_future_context. The closure is not called yet when you give it away. And with_future_contexthas no way to call the closure with a &mut resp of the local variable.

Sort of like this.

fn example2(_arg: &mut String) {}
fn example3<'lt, F: FnOnce(&'lt mut String)>(f: F) {
    f(todo!("???? I don't have a `&'lt mut String` to pass!"));
}
fn example4() {
    let mut arg = String::new();
    example3(example2)
}

You could make the API take a FnOnce(&'lt mut Response) and a &'a lt mut Response in order to create WithFailureContextFuture. But you also could get rid of the lifetimes and the &'lt mut Response in the API and just take a FnOnce() closure. The callers could then capture a &mut Response (or whatever else) in the closure.

#[pin_project]
pub struct WithFailureContextFuture<Fut, F> {
    #[pin]
    fut: Fut,
    context: Option<F>, // Option so we can take it out on poll
}

// ...
        match this.fut.poll(cx) {
            Poll::Pending => Poll::Pending,
            Poll::Ready(Ok(v)) => Poll::Ready(Ok(v)),
            Poll::Ready(Err(_)) => {
                if let Some(f) = this.context.take() {
                    f();
                }
                Poll::Ready(Err(()))
            }
        }
// ...

pub async fn test_api() {
    let mut resp = Response;
    let result = may_fail().with_failure_context(|| { let _ = &mut resp; }).await;
}

Now F may contain the &'lt mut Response as part of the closure type. When it does, most the sort of problems you can encounter from non-'static futures will still apply, even though the type generic "swallowed" the lifetime generic.

7

Thank you so much for your patience. I've gone through your reply, and I appreciate your clarity of explanation. I'd just want clarification on something:

You could make the API take a FnOnce(&'lt mut Response) and a &'a lt mut Response in order to create WithFailureContextFuture.

In a way, this is to say that we cannot enforce taking in a &mut Response; i.e. we cannot enforce that this is a closure that will at least pretend to mutate Response (for the sake of academics at the very least), without passing in a mutable reference to &mut Response first for the Future to store, yes?

8

You can look at it as: if you don't pass &mut Response to the future, then the Response's mutation, or even existence, is none of the future's business.

9

There's no way to enforce that a closure captured a &mut Response. You could probably rig something up using opaques or some trait scaffolding...

// e.g. using something related to this opaque-returning function:
//
// The returned closure is allowed to capture the `&mut Response`
// (and thus will act like it did from the caller's perspective)
fn mk_closure(resp: &mut Response) -> impl FnOnce() { ... }

...but trying to work this into your API or such is going to involve unidiomatic complexity, and would probably create more confusion than clarity. (And you're still passing a &mut Response somewhere.)

10

To be fair, I don't mind the complexity. At this point, it is genuinely an academic question as to how much expressiveness I could theoretically squeeze out of the type system. However, thank you so very much for your explanation. I appreciate it a lot!

11

Well, you usually cannot enforce a callback to do anything ... except to somehow obtain a valid return value (or diverge).

So another possible approach might be something along the line of:

pub struct ProofOfMutation(());

pub trait WithFailureContext<T, E>: Future<Output = Result<T, E>> {
    fn with_failure_context<F>(self, f: F) -> WithFailureContextFuture<Self, F>
    where
        F: FnOnce() -> ProofOfMutation,
        Self: Sized;
}

and have operations that mutate Response return a ProofOfMutation. This way the closure "must" call one of those operations on some Response to be able to return. But this is not airtight as you can do:

pub async fn test_api() {
    let mut resp = Response;
    let fake_proof = resp.mutate();
    let result = may_fail().with_failure_context(move || { fake_proof }).await;
    // the callback did not mutate `resp`, we did.
}

You can avoid the above by using extra generic lifetime on the proof and require the callback to be generic over it (I did not think it through completely, there might be problems or it might not work at all) (I think it is called "branded lifetimes"):

pub struct ProofOfMutation<'a>(PhantomData<fn(&'a ()) -> &'a()>);

pub trait WithFailureContext<T, E>: Future<Output = Result<T, E>> {
    fn with_failure_context<F>(self, f: F) -> WithFailureContextFuture<Self, F>
    where
        F: for<'a> FnOnce() -> ProofOfMutation<'a>,
        Self: Sized;
}

But the future still has no control over which Response gets mutated by the callback. I think it cannot be achieved without it taking both a &mut Response and a callback.

I don't know if this would make a good API (or even work at all), but you wanted to go academic, so this could be one more path to explore.