[tl;dr: This is an invitation to contribute to improving crypto in Rust by improving benchmarks.]
crypto-bench is a framework for measuring the performance of key crypto algorithms across a variety of implementations. The goal of the project is to accelerate the development of crypto libraries in Rust by helping library users and library developers understand crypto performance issues.
As a crypto library developer, I use crypto-bench frequently to make decisions about improvements to the implementation of ring. For example, I use it to measure whether a commit improves or regresses performance, which is particularly helpful when I’m replacing C or assembly language code with Rust code. (The ring project has a policy where, while we favor Rust code over C and assembly language code, we won’t let performance-critical features regress in performance “just” to get the safety features of Rust; we generally require new Rust code to be as fast or faster than the code it replaces.)
As a library developer, I also use it to discover which other libraries have done something better than ring does, so that I can use the same techniques to improve ring. I know the developers of Octavo and other crypto libraries have also used crypto-bench to guide performance improvements.
crypto-bench is also designed to help users of crypto libraries. It can help users decide which library to use because, by design, all the benchmarks are designed to be as close to apples-to-apples comparisons as possible. It can also help users measure the performance of their crypto libraries of choice over time. For example, it makes it easy to verify that an update to the library hasn’t regressed performance.
crypto-bench can also be useful to the
libstd developers, by helping them see areas where the compiler and the standard library need to improve in order for Rust code to be competitive with (or better than) equivalent C and/or assembly-language code. (My ultimate goal is for code in safer languages like Rust to completely replace hand-written assembly and C code.)
The crypto-bench README.md has a table showing which benchmarks have been written and which still need to be written. Note that this table is focused in scope to things I think are the most urgent, but I’m also very open to adding benchmarks of other generally-useful, immediately-useful, and not-obsolete crypto primitives (BLAKE2 and Argon2 come to mind).
It is pretty easy to add new benchmarks to crypto-bench by copying the form of the existing benchmarks. I think it would be great for this project to get more contributors, particularly contributors writing benchmarks for crypto libraries other than ring. (I usually write the ring benchmarks soon after adding the feature to benchmark, although I’ve skipped some like HMAC and random byte generation.)
Also, more generally, it would be great to see some tools for scraping the textual output of
cargo bench and transforming it into a format that can be charted and/or tabulated in useful ways.
I would love to see more contributions to this project. Let me know if you have any questions.