Is it possible to not use Box in this code?

[cocoliliace]

I'm implementing a rate limiting middleware for tower, whose Service trait requires an associated type Future for the returned future.

To check whether the request should be rate limited, my middleware makes an asynchronous call to a data store like Redis. But since async functions return anonymous futures, and impl Trait in associated types is still unstable, I don't know how to define Future.

I found another project solving the same problem with Box::pin and I'm currently doing the same thing, but I wonder if there's a way to not have to use the heap in this case.

This is my code for the service (the lifetime for K::Key is probably not ideal but I'm tyring to fix the Box problem first):

impl<S, K, ReqBody, ResBody> Service<Request<ReqBody>> for RateLimit<S, K>
where
    S: Service<Request<ReqBody>, Response = Response<ResBody>>,
    K: KeyExtractor<ErrorBody = ResBody>,
    for<'a> K::Key: 'a,
    ResBody: Default,
{
    type Response = S::Response;
    type Error = S::Error;
    type Future = ResponseFuture<S::Future, K::ErrorBody>;

    fn call(&mut self, req: Request<ReqBody>) -> Self::Future {
        let key = match self.key_extractor.extract(&req) {
            Ok(key) => key,
            Err(res) => return ResponseFuture::ready(res),
        };

        let check_future = {
            let mut store = self.store.clone();
            let key = key.clone();
            let limit = self.limit;
            let window = self.window;
            Box::pin(async move { store.check_and_increment(key, limit, window).await })
        };
        ResponseFuture::pending(self.inner.call(req), check_future, self.fail_open)
    }
}

#[pin_project(project = ResponseFutureProj)]
pub enum ResponseFuture<F, ErrorBody> {
    Pending {
        #[pin]
        inner: F,
        #[pin]
        check_future: CheckFuture,
        check_result: Option<RateLimitStatus>,
        fail_open: bool,
    },
    Ready(Option<Response<ErrorBody>>),
}

and the signature for check_and_increment:

async fn check_and_increment<K>(
    &mut self,
    key: K,
    limit: u16,
    window: u32,
) -> RedisResult<RateLimitStatus>
where
    K: ToRedisArgs,
{

RedisResult is just a type alias for Result<T, RedisError>.

The code is MPL-2.0. If you post some code solution but don't want me to integrate it into the rest of the project under MPL-2.0, let me know!

The rest of the code is probably not so relevant, but they are on https://git.sr.ht/~liliace/tower-rate-limit-redis if you want more context of the code.

[nerditation]

the returned Future of async functions are not namable, which means it is impossible to use it unboxed in the ResponseFuture type without impl Trait in associated types or in type aliases.

unless you can (and are willing to) rewrite the async fn check_and_increment() with a named Future type, I'd say just use Pin<Box<dyn Future>> and don't bother.

[cocoliliace]

Hi thanks for the response!

unless you can (and are willing to) rewrite the async fn check_and_increment() with a named Future type

To check my understanding, if I were to make this work by rewriting check_and_increment to return a named Future, I would need to do it recursively for every returned future right? Given my current implementation:

async fn check_and_increment<K>(
    &mut self,
    key: K,
    limit: u16,
    window: u32,
) -> RedisResult<RateLimitStatus>
where
    K: ToRedisArgs,
{
    RATE_LIMIT_SCRIPT
        .key(key)
        .arg(limit)
        .arg(window)
        .invoke_async(&mut self.conn)
        .await
}

I would need to rewrite .invoke_async to return a named Future type. And since .invoke_async is implemented like this:

pub async fn invoke_async<C, T>(&self, con: &mut C) -> RedisResult<T>
where
    C: crate::aio::ConnectionLike,
    T: FromRedisValue,
{
    ScriptInvocation {
        script: self,
        args: vec![],
        keys: vec![],
    }
    .invoke_async(con)
    .await
}

I would need to rewrite this too and so on?

Because if I only rewrote check_and_increment, it would require a generic type arg for the anonymous future returned by .invoke_async. Perhaps something like this:

struct MyFuture<F> {
    inner: F,
}

impl<F> Future for MyFuture<F>
where
    F: Future<Output = RedisResult<RateLimitStatus>>,
{
    // ...
}

fn check_and_increment<K>(
    &mut self,
    key: K,
    limit: u16,
    window: u32,
) -> MyFuture<impl Future<Output = RedisResult<RateLimitStatus>>>
where
    K: ToRedisArgs,
{
    MyFuture {
        inner: RATE_LIMIT_SCRIPT
            .key(key)
            .arg(limit)
            .arg(window)
            .invoke_async(&mut self.conn)
    }
}

Which would lead to the same problem of not having a concrete type for the associated type Future in service.

Is this the rewrite you meant?

[nerditation]

yes, that is what I meant.

to be clear, I was just stating the language/API limitation of the status quo, I'm not suggesting you should do it this way. and as I said, don't bother and just box it, it doesn't worth it.

defining named types then manually implementing Future for them is very painful and error prone, and it defeats the whole purpose of the async/await syntax sugar.

due to the language and API limitation though, sometimes it is the only choice available, especially in low-level libraries which serve as building blocks of the async ecosystem. however IMO, it should be totally unnecessary just for the sake of "not to box a future".

just some thought experiments: if libraries like tower were designed when async fn and return position impl Trait in traits were available, the API probably looks much differently. or, if impl Trait in associated types or type aliases were available, using the API would probably be much smoother experience. but now, we must work with what we have.

[cocoliliace]

Got it. Thank you for the clarification :).
Yeah rewriting all that does seem cumbersome. I was considering it until you've confirmed that I must do it for every nested future... Guess I will stick to Box::pin until a new tower API or stable rust feature makes it simpler to implement. Looking forward to when that happens! Thanks for the answer

[simonbuchan]

FWIW, Every time you spawn a task that's boxed, and every async function call in any other language (including C++!!) is boxed - it's almost never a concern. But the limitation on naming these anonymous types does really suck...

[cocoliliace]

I see. That's good to know. Thanks for the additional information!