Lifetimes issues in async benchmarking code using criterion

[suchapalaver]

Edit:

I've left the post as it was below but I thought I'd share how I eventually solved my issue in a way I was completely satisfied with, which was by creating a blocking async handle in which to run the async function I wanted to benchmark.

async fn do_something(l: &mut NonClonableType) {
    l.method_requiring_mutable_ref().await.unwrap();
}

fn my_benchmarks(c: &mut Criterion) {
    c.bench_function("help_needed", |b| {
        b.iter_batched_ref(
            || {
                let data = "some_data";
                let mut non_clonable_type = NonClonableType::new();
                type.set_some_data(data);
                non_clonable_type
            },
            |input| {
                let rt = Runtime::new().unwrap();
                let handle = rt.handle();
                handle.block_on(async move { do_something(input).await.unwrap() })
            },
            BatchSize::SmallInput,
        );
    });
}

---

I'm trying to use criterion to benchmark an async method on a struct that requires self be &mut, and that is part of a trait using async_trait.

Here's my benchmark code:

use criterion::BatchSize;
use criterion::BenchmarkId;
use criterion::Criterion;
use criterion::{criterion_group, criterion_main};

use criterion::async_executor::FuturesExecutor;

// Here we have an async function to benchmark
async fn do_something(l: &mut NonClonableType) {
    l.method_requiring_mutable_ref().await.unwrap();
}

fn my_benchmarks(c: &mut Criterion) {
    let mut group = c.benchmark_group("Help!");
    group.bench_function(BenchmarkId::new("help_needed: {}", "details"), |b| {
        b.to_async(FuturesExecutor).iter_batched_ref(
            || -> NonClonableType {
                let data = "some_data";
                let mut non_clonable_type = NonClonableType::new();
                type.set_some_data(data);
                non_clonable_type
            },
            |nct: &mut NonClonableType| do_something(nct),
            BatchSize::SmallInput,
        )
    });
}

criterion_group!(benches, my_benchmarks);
criterion_main!(benches);

Part of what I did there was inspired by frequent solver of people's problems Kevin Reid's responses to this StackOverflow question, where Kevin suggests using .iter_batched_ref.

The error I'm getting has come up in other posts, namely:

error: lifetime may not live long enough
  --> project/benches/my_benchmarks.rs:28:39
   |
28 |             |l: &mut NonClonableType| do_something(l),
   |                 -                   - ^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
   |                 |                   |
   |                 |                   return type of closure `impl std::future::Future<Output = ()>` contains a lifetime `'2`
   |                 let's call the lifetime of this reference `'1`

I'm lost to the point of having spent several hours reading to try to find inspiration and being unsure if what I'm trying is possible.

[quinedot]

I don't think your error matches your sample code, so this is an educated guess.

Looking at the definition for iter_batched_ref, it wants the second parameter (R) to be something that takes a &mut I with any lifetime and returns a singular type F no matter what the input lifetime is (FnMut(&mut I) -> F).

But do_something captures the lifetime of the &mut NonClonableType, so the type of the future it returns isn't a singular type -- it varies based on the input lifetime.

To summarize, iter_batched_ref only supports future generating closures that return futures that don't capture any references or other lifetime restrictions.

Playground that illustrates some of these issues.

[suchapalaver]

Thanks so much for your response. The way you broke it down in the Playground made me understand ..., well, how you'd break down the methods I was trying to use.

Somehow it made me realize I can at least make some compiling headway by going back to the Benchmarking async functions examples in the criterion user guide.

Just having the conversation helped me realize I can rewrite my_benchmarks like this:

fn my_benchmarks(c: &mut Criterion) {
    let data = "some_data";

    c.bench_with_input(BenchmarkId::new("input_example", "x"), &data, |b, &data| {
        b.to_async(FuturesExecutor).iter(|| do_something(data);
    });
}