Why tokio::fs is slower than std::fs

1

Hello everyone

i've being playing with tokio lately and i've noticed that enumerating files in a directory is really slow, so i wanted to make sure that its tokio and not my hardware. so i used ~60000 files that are stored in the second inner directory (Level 2 depth).

The example of tokio goes like this:

use std::io;
use tokio::fs;

pub async fn grep_files() -> io::Result<()> {
    let mut read_dir = fs::read_dir("/home/arya_helen/Downloads/cache/epub").await?;
    let mut count = 0;

    while let Ok(Some(entry)) = read_dir.next_entry().await {
        let mut read_dir = fs::read_dir(entry.path()).await?;

        while let Ok(Some(entry)) = read_dir.next_entry().await {
            count += 1;
        }
    }

    println!("Number of entries: {}", count);

    Ok(())
}

The output

[ START ] Game on..
Number of entries: 61581

--
4.535264998  secs   4535264998 nanos

and i did the exact same for rust std

use std::fs;
use std::io;

pub fn grep_files() -> io::Result<()> {
    let mut read_dir = fs::read_dir("/home/arya_helen/Downloads/cache/epub")?;
    let mut count = 0;

    while let Some(Ok(entry)) = read_dir.next() {
        let mut read_dir = fs::read_dir(entry.path())?;

        while let Some(Ok(entry)) = read_dir.next() {
            count += 1;
        }
    }

    println!("Number of entries: {}", count);

    Ok(())
}

The output

[ START ] Game on..
Number of entries: 61581

--
0.374388248  secs    374388248 nanos

and as you can see, i get the exact same number in both versions except with std::fs its a lot faster. is this normal, or am i doing something wrong?

2 Likes
2

Do you use release mode (--release)?

3

for --release tokio::fs took ~3 seconds where std::fs took ~0.3 seconds

1 Like
4

See the benchmarks in:

You are encountering the same overhead with tokio::fs use of spawn_blocking.

2 Likes
5

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.