That is wrong. Slow RAM access does not make speeding the computation proccess has no effect. Total latency isn't just loading data, but the sum of all of loading data + processing data, so if you make any of them faster, you make the whole thing faster. And CPU has intellegent prefect in an already contigous data, they will be fetched to the cache chunk by chunk with multi tiered cache (L1, L2, L3). They are faster than RAM
And your code has SIMD under the hood, so you don't compare non SIMD with SIMD. To show the actual number, you have to tell LLVM to not auto SIMD with LLVM's no vectorize flag
RUSTFLAGS="-C llvm-args=-vectorize-loops=false" cargo run --release
Your code, I just complete it to make it compiles
use std::time::Instant;
fn main() {
let data_storage: Vec<usize> = (0..1000).collect();
let data: &[usize] = &data_storage;
let start = Instant::now();
let mut index = 0;
for _ in 0..1_000_000 {
let d = &data[index..][..256];
let total = d.iter().copied().sum::<usize>();
index = total % (data.len() - 256);
}
let elapsed = start.elapsed().as_millis();
println!("{index}");
println!("Elapsed time: {} ms", elapsed);
}
The result :
With SIMD. I didn't use flag target-cpu=native because Neon is already the best SIMD in my phone, and Rust already pick Neon by default. In x86 CPU that supports AVX512, you have to put the target native flag to not make Rust use general available x86 SIMD (SSE2) but to use the best available one that the CPU has (for example AVX512)
[root@localhost tes2]# cargo run --release Finished `release` profile [optimized] target(s) in 0.07s
Running `target/release/tes2` 0
Elapsed time: 113 ms
[root@localhost tes2]# cargo run --release
Finished `release` profile [optimized] target(s) in 0.08s
Running `target/release/tes2`
0
Elapsed time: 114 ms
[root@localhost tes2]# cargo run --release
Finished `release` profile [optimized] target(s) in 0.09s
Running `target/release/tes2`
0
Elapsed time: 115 ms
Without SIMD, almost 3x slower, for a simple operation not even complex yet. And only use Neon instruction, not even the big x86 AVX512
[root@localhost tes2]# RUSTFLAGS="-C llvm-args=-vectorize-loops=false" cargo run --release
Finished `release` profile [optimized] target(s) in 0.09s
Running `target/release/tes2`
0
Elapsed time: 268 ms [root@localhost tes2]# RUSTFLAGS="-C llvm-args=-vectorize-loops=false" cargo run --release
Finished `release` profile [optimized] target(s) in 0.07s
Running `target/release/tes2`
0
Elapsed time: 270 ms
[root@localhost tes2]# RUSTFLAGS="-C llvm-args=-vectorize-loops=false" cargo run --release
Finished `release` profile [optimized] target(s) in 0.08s
Running `target/release/tes2`
0
Elapsed time: 271 ms
[root@localhost tes2]#
Even better if you change the data type of your code from usize to smaller one that still fit the size. I changed it to u32. The SIMD becomes 2x faster thsn before, while the non SIMD stays in its previous speed. So in total, the SIMD is almost 5x faster
use std::time::Instant;
fn main() {
let data_storage: Vec<u32> = (0..1000).collect();
let data: &[u32] = &data_storage;
let start = Instant::now();
let mut index = 0u32;
for _ in 0..1_000_000 {
let d = &data[index as usize..][..256];
let total = d.iter().copied().sum::<u32>();
index = total % (data.len() as u32 - 256);
}
let elapsed = start.elapsed().as_millis();
println!("{index}");
println!("Elapsed time: {} ms", elapsed);
}
With SIMD :
Without SIMD :
