What benchmarking practices exist in the Rust community, and where can I learn about them?

help
1

Based on my initial goals, I have the following requirements:

  1. For the benchmarking itself, I’ll just use Criterion without any fuss and look at its reports.
  2. I want to keep a performance history tied to code changes. Each checkpoint in this history is a commit plus the benchmark reports.
  3. I don’t want to store the reports locally in the repository. For example, the ripgrep approach seems inconvenient to me.

After looking through many popular crates, I’ve concluded that setting up a good practice isn’t so straightforward.

2

I use github-action-benchmark.

3

I agree Criterion is excellent, but its output seems too focused on individual benchmarking. I wrote some more code to provide higher-level overview, how different cases (different crates with comparable functionality in my case) compare with each other. Feel free to pick what suits from you from my benchmarking project, or ideas. Sorry for some Python. I think a good Rust based package that provides overview, and also tracks history, checks for regression, is yet to come.