Absolutely absurd bench results with fat LTO

1

I'm seeing impossible benchmark improvement when using fat LTO, improvement from circa 50 ns / iter to 0.25 ns per iteration. My best guess is some time computation error. I don't know how to verify assemblies for actual code. (╥﹏╥)

I prepared repository so anybody captivated can have a look.

Release: Default

cargo bench --test bench --profile release

running 2 tests
test gcd_naive_2_test ... bench:          44.29 ns/iter (+/- 6.54)
test gcd_naive_test   ... bench:          54.92 ns/iter (+/- 3.56)

[profile.bench]
opt-level = 3            
lto = "fat"
debug = false
split-debuginfo = 'off'
strip = "none"
debug-assertions = false
overflow-checks = false
panic = 'abort'
incremental = false
codegen-units = 16
rpath = false

cargo bench --test bench --profile bench

test gcd_naive_2_test ... bench:           0.25 ns/iter (+/- 0.01)
test gcd_naive_test   ... bench:           0.25 ns/iter (+/- 0.01)

Benchmarked code in question is some recursive binary GCD implementation.

2

My experience of benchmarks which give results in the picosecond range usually means that the whole thing has been optimised out by the compiler - maybe enabling fat LTO gave the compiler enough information to do that. In particular, try creating an actual benchmark case e.g. with criterion, taking care to use things like black_box where relevant.

3

In this case, the place you must use black_box is around the inputs to the function being benchmarked:

+use std::hint::black_box;

 #[bench]
 fn gcd_naive_test(b: &mut Bencher) {
     let num_1 = 2_559_031_471u64; // 150531263ᵖ ⋅17ᵖ
     let num_2 = 1_956_912_061; // 150531697ᵖ ⋅13ᵖ    

-    b.iter(|| gcd_naive(num_1, num_2));
+    b.iter(|| gcd_naive(black_box(num_1), black_box(num_2)));
 }

This hints against optimizations which make use of the specific values being passed to the function. (You don’t need a black_box() on the output of the function because iter() takes care of that for the value it receives.)

4 
b.iter(|| gcd_naive(black_box(num_1), black_box(num_2)));

Very true.