There are a few open issues for benchmarking commits to rust-lang/rust with regards to performance. https://perf.rust-lang.org is intended for compiler benchmarks (i.e., compile-time performance) and https://github.com/rust-lang/rust/issues/31265 tracks adding runtime benchmarks (i.e.,
cargo bench) results to either perf.rlo or another similar site. While perf.rlo isn’t currently tracking timings correctly due to changes in how
-Ztime-passes works and in the soft-deprecation thereof (i.e, it’s starting to be no longer representative of how the compiler works internally in the move towards a more incremental compiler), this should be fixed as soon as we work out a concrete plan for doing so.
My bisection tool works quite well for finding commits which regressed by introducing a new error or removing an error (i.e., changing stderr output to have / not have some string) but isn’t a great fit for “manual” bisection where the decision whether a commit is good/bad must be made by a human. I hope that can improve with time; there’s certainly potential for user contributions to make it work more closely to how tooling like
git bisect works for the use cases that are assisted by that, and improvements in the ergonomics of using it, since it’s not currently ideal – the setup is somewhat non-trivial.