Cargo-benchcmp is now written in Rust (rewrite done by Apanatshka)


#1

It was written in Python, but now that it’s in Rust, you can install it with Cargo!

cargo install cargo-benchcmp

It is a small tool for comparing the output of two micro benchmark runs. For example, run your benchmarks and save it to a file:

$ cargo bench --features cpp uflat | tee snappy.log
test cpp::uflat00_html     ... bench:      43,371 ns/iter (+/- 2,090) = 2361 MB/s
test cpp::uflat01_urls     ... bench:     485,792 ns/iter (+/- 27,269) = 1445 MB/s
test cpp::uflat02_jpg      ... bench:       4,961 ns/iter (+/- 614) = 24812 MB/s
test cpp::uflat03_jpg_200  ... bench:         141 ns/iter (+/- 8) = 1418 MB/s
test cpp::uflat04_pdf      ... bench:       6,781 ns/iter (+/- 287) = 15101 MB/s
test cpp::uflat05_html4    ... bench:     189,075 ns/iter (+/- 863) = 2166 MB/s
test cpp::uflat06_txt1     ... bench:     152,494 ns/iter (+/- 531) = 997 MB/s
test cpp::uflat07_txt2     ... bench:     134,494 ns/iter (+/- 490) = 930 MB/s
test cpp::uflat08_txt3     ... bench:     407,112 ns/iter (+/- 1,214) = 1048 MB/s
test cpp::uflat09_txt4     ... bench:     561,104 ns/iter (+/- 2,461) = 858 MB/s
test cpp::uflat10_pb       ... bench:      42,255 ns/iter (+/- 335) = 2806 MB/s
test cpp::uflat11_gaviota  ... bench:     156,269 ns/iter (+/- 4,935) = 1179 MB/s
test rust::uflat00_html    ... bench:      40,955 ns/iter (+/- 363) = 2500 MB/s
test rust::uflat01_urls    ... bench:     463,733 ns/iter (+/- 7,728) = 1513 MB/s
test rust::uflat02_jpg     ... bench:       4,946 ns/iter (+/- 556) = 24887 MB/s
test rust::uflat03_jpg_200 ... bench:         126 ns/iter (+/- 6) = 1587 MB/s
test rust::uflat04_pdf     ... bench:       6,321 ns/iter (+/- 585) = 16199 MB/s
test rust::uflat05_html4   ... bench:     176,036 ns/iter (+/- 10,895) = 2326 MB/s
test rust::uflat06_txt1    ... bench:     149,843 ns/iter (+/- 6,766) = 1014 MB/s
test rust::uflat07_txt2    ... bench:     133,027 ns/iter (+/- 5,200) = 941 MB/s
test rust::uflat08_txt3    ... bench:     399,391 ns/iter (+/- 14,893) = 1068 MB/s
test rust::uflat09_txt4    ... bench:     558,919 ns/iter (+/- 20,800) = 862 MB/s
test rust::uflat10_pb      ... bench:      37,464 ns/iter (+/- 2,185) = 3165 MB/s
test rust::uflat11_gaviota ... bench:     150,985 ns/iter (+/- 7,598) = 1220 MB/s

then compare!

$ cargo benchcmp cpp:: rust:: snappy.log
name             cpp:: ns/iter        rust:: ns/iter       diff ns/iter   diff % 
uflat00_html     43,371 (2361 MB/s)   40,955 (2500 MB/s)         -2,416   -5.57% 
uflat01_urls     485,792 (1445 MB/s)  463,733 (1513 MB/s)       -22,059   -4.54% 
uflat02_jpg      4,961 (24812 MB/s)   4,946 (24887 MB/s)            -15   -0.30% 
uflat03_jpg_200  141 (1418 MB/s)      126 (1587 MB/s)               -15  -10.64% 
uflat04_pdf      6,781 (15101 MB/s)   6,321 (16199 MB/s)           -460   -6.78% 
uflat05_html4    189,075 (2166 MB/s)  176,036 (2326 MB/s)       -13,039   -6.90% 
uflat06_txt1     152,494 (997 MB/s)   149,843 (1014 MB/s)        -2,651   -1.74% 
uflat07_txt2     134,494 (930 MB/s)   133,027 (941 MB/s)         -1,467   -1.09% 
uflat08_txt3     407,112 (1048 MB/s)  399,391 (1068 MB/s)        -7,721   -1.90% 
uflat09_txt4     561,104 (858 MB/s)   558,919 (862 MB/s)         -2,185   -0.39% 
uflat10_pb       42,255 (2806 MB/s)   37,464 (3165 MB/s)         -4,791  -11.34% 
uflat11_gaviota  156,269 (1179 MB/s)  150,985 (1220 MB/s)        -5,284   -3.38%

Thanks to /u/Apanatshka for the rewrite and the added functionality!