Hash digest performance (Rust vs. Python)

Hi, I recently began to learn Rust (after using mostly Python for the last couple of years). As part of my training, I translate some system scripts to Rust. Among these is a script which traverses a directory tree, creates hash digests for each file, and compares the digest to a baseline. I noticed that the Rust version of this script takes 3x longer to execute, and tracked the difference down to the hashlib / sha256 performance. The Python (test) script looks like this:

import hashlib

def get_hash(file_path):
	myhash = hashlib.sha256()
	with open(file_path,'rb') as file:
		content = file.read()
		return myhash.hexdigest()

if __name__ == '__main__':
	filepath = "/path/to/file"

The Rust code looks almost identical:

use sha256::try_digest;
use std::path::Path;

fn get_hash(filepath: &str) -> String {
    let input = Path::new(filepath);
    let hash_digest = try_digest(input).unwrap();

fn main() {
    let filepath = "/path/to/file";

The test file at /path/to/file is 229 MB (to make the difference more pronounced), and a release build of the above Rust code takes 0.99 second to complete, while the Python code finished in 0.19 seconds. I am aware that Python relies on a C implementation for hashlib, but I still wonder why Rust takes more than 5x longer (in this isolated example) and 3x longer (in the real world scenario with many small files described above).

I am grateful for any hints.


If you ran the Rust version first and Python version afterward, what are the results when running the Python version first and then the Rust version?

The difference in execution speed may be as simple as the OS caching data between runs.

1 Like

Unfortunately, this is not the case. Rust does profit a bit from caching (0.83 on a second run vs. 0.99 on the first run), but Python (with C) stays much faster at 0.19.

Looks like the sha2 crate which the sha256 crate is a wrapper around only uses SIMD when both the sha and sse4.1 target features are supported by your cpu. If you are using linux what does cat /proc/cpuinfo show? You might also want to try ring::digest::digest(&ring::digest::SHA256, data) from the ring crate. It has several optimized implementations written in assembly for different cpu feature levels.

1 Like

This thread looks like a dup? The issue was solved in

It was potential-spam hidden for awhile and they reposted. I'll close this one.