[lib] jwalk: Fast recursive directory walk


#1

I’m happy to announce the release of my first crate jwalk:

  • Walk is performed in parallel using rayon
  • Results are streamed in sorted order
  • Custom sort/filter/skip

Thanks to everyone here who’s helped answer my many rust questions! I hope this crate proves useful. I’m leaving it at 0.1.x for a while. I’m hoping for API/code/testing feedback before moving to 1.0 when everything is deemed solid.

Inspiration

This crate is inspired by both walkdir and ignore. It attempts to combine the parallelism of ignore with walkdirs streaming iterator API.

Why use this crate?

This crate is particularly fast when you want streamed sorted results. In my tests its about 4x walkdir speed for sorted results with metadata. Also this crate’s process_entries callback allows you to arbitrarily sort/filter/skip entries before they are yielded.

Why not use this crate?

Directory traversal is already pretty fast. If you don’t need this crate’s speed then walkdir provides a smaller and more tested single threaded implementation.

Benchmarks

Benchmarks comparing this crate with jwalk and ignore.