The rust_xlsxwriter crate that I wrote/maintain has a reasonably large number (~700) of doc examples that take a long time to run (~4 mins).
I saw some significant speedups with the Rust 2024 Edition compilation of compatible doctests here and here but I can't replicate that speedup with my repo.
Could someone point out what I am doing wrong.
See the steps below to reproduce. I am on Rust 1.85
$ git clone https://github.com/jmcnamara/rust_xlsxwriter.git
Cloning into 'rust_xlsxwriter'...
$ cd rust_xlsxwriter/
# Run the integration tests to pull in the dependencies.
$ cargo test --test integration
$ time cargo test --doc
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.06s
Doc-tests rust_xlsxwriter
running 685 tests
...
real 4m22.778s
user 6m46.697s
sys 1m29.343s
Then with the 2024 edition:
# Not sure if this step is required.
$ cargo fix --edition
Migrating Cargo.toml from 2021 edition to 2024
...
There are two extra costs to doctests compared to #[test] tests.
Each one is compiled separately (before edition 2024).
Each one is run as a separate process.
Therefore, doctests may still run slowly if process spawning is slow. This is usually on systems which have antivirus software that is checking each executable when it is started — typical for Windows and macOS. What operating system are you using?
If you are using Windows antivirus software, there may be an option to exclude a directory from scanning — use this on your target/ directory. This may also speed up builds.
On macOS, you need to exempt the application that is acting as a parent process for your cargo test command (such as your IDE or terminal program):
Not sure if this is relevant to your issue, but someone working on Bevy made a similar observation, however they also noticed it seems to work properly on nightly.
The results above were for macOS. Adding iterm2 to that whitelist gave a 4x improvement in the test run time. Thanks for that. That is a good overall win.
However, the run time is still the same for the 2021 and 2024 Edition cases.
I also tried on a couple of Linux systems and got the same result in both cases.
That is interesting. Running the test with +nightly does give a slightly different result where I see the ~670 tests split into a group of 20 and 650. So that looks like there is some merging of the doc tests happening. Unfortunately the overall run time is the same since the 650 block is still the same.
Which makes we wonder if the issue here is that the majority of the doc tests in the rust_xlsxwriter source have a main function and maybe those can't be merged. The ~20 that I see merged separately are simpler tests/examples that don't have an implicit main.
Looking at the code of the PR (look for has_main_fn), it seems that the only thing the presence of main affects is whether the test gets one automatically generated instead. Doctest code is concatenated into the structure
mod test_1 {
fn main() {...}
}
mod test_2 {
fn main() {...}
}
so main is always expected and is not an obstacle to merging. (Unless something else changed since the original PR.)
