CPU time (and not elapsed)

help
1

Hello,

Maybe a silly question...

I'm trying to bench my app. I noticed that Criteron or Divan measure Elapsed time.

First question :
This surprises me, as it seems the representative metric to measure an algorithm should be CPU time.
Am I right ?

Second one :
So to measure the CPU time I'm using sysinfo (code below). But now I notice that in fact the returned time depends of CPU usage of other processes. Is it really CPU time ? Where is my misunderstanding ?

use sysinfo::System;

let mut proc_info = Cpu::new();
let cpu1_before = proc_info.cpu_time();
count_throw_buf_read_splitter(buf_size, CONTENT_LEN);
let cpu1_after = proc_info.cpu_time();

pub struct Cpu {
    system: System,
}

impl Cpu {
    pub fn new() -> Self {
        let system = System::new_all();
        Self { system }
    }
    pub fn cpu_time(&mut self) -> u64 {
        self.system.refresh_all();
        let pid = sysinfo::get_current_pid().unwrap();
        let process = self.system.process(pid).unwrap();
        process.accumulated_cpu_time()
    }
}
2

If your goal is minimising how quick to finish, then elapsed is the right one. Nothing else should be running that you think could invalidate it.

What leads you to believe it is counting other processes? (Also what OS?)

3

My goal is to measure CPU usage independently of other processes running in parallel.

It's because , for the same program, the CPU time returned double if a CPU-intensive-process is running in parallel (I'm on Linux 6.1.0-47-amd64)

4

Your measurement is probably right, and the doubling is real. Two separate things are going on.

First, sysinfo's Cpu is the system-wide CPU, not your process. For per-process CPU time use Process::accumulated_cpu_time() from sysinfo, or skip the crate and call getrusage(RUSAGE_SELF) / clock_gettime(CLOCK_PROCESS_CPUTIME_ID) via libc, or the cpu-time crate (ProcessTime::now() then .elapsed()). Those only count cycles your own process was scheduled for, so another process running in parallel does not add to them.

Second — and this is the part that surprised me when I tried it — per-process CPU time still goes up under contention on modern hardware, because CPU time is measured in seconds, not in work. I ran a fixed 400M-iteration integer loop and measured it with getrusage and cpu-time:

alone, pinned to a P-core (cpu0):       39 ms CPU time
alone, pinned to an E-core (cpu12):     74 ms CPU time
8 busy loops on other cores, unpinned:  85 ms CPU time
8 busy loops on other cores, pinned P:  42 ms CPU time

Same instructions, same process, ~2x the CPU time — because under load the scheduler moved it to a slower core. Even on a non-hybrid chip you get a smaller version of this from all-core turbo being lower than single-core turbo, and from shared L2/L3 and memory bandwidth.

So for your question "what is the representative metric for an algorithm": if you want something that does not depend on what else is running, CPU time is not it either. Pin the benchmark to one core (taskset -c 0) and run with the machine quiet, which is what Criterion assumes you are doing — or count instructions/cycles with perf stat -e instructions,cycles, which is the only number that really is independent of the neighbours.

5

Really interesting ! Thanks !