Doc tests never run under release mode

rickwebiii · April 8, 2025, 8:02pm

I have a library that really needs optimizations turned on to run efficiently. Hence, in CI I always run cargo test --release. I noticed that a doc test takes over a minute to run when the same test completes in under a second when run outside of a doc example.

However, doc tests always seem to run in debug mode as confirmed by this playground snippet. If you run the tests in debug mode, the non-doc test will panic. However, if you run in release mode, the same test put in a doc comment will panic while the non-doc test passes. Is this expected?

bjorn3 · April 8, 2025, 8:24pm

-Copt-level is ignored by rustdoc. The full set of arguments that rustdoc passes to rustc for compiling doc tests can be found at

github.com/rust-lang/rust

src/librustdoc/doctest.rs

d4f880f8c


      
          /// Execute a `RunnableDoctest`.
          ///
          /// This is the function that calculates the compiler command line, invokes the compiler, then
          /// invokes the test or tests in a separate executable (if applicable).
          fn run_test(
              doctest: RunnableDocTest,
              rustdoc_options: &RustdocOptions,
              supports_color: bool,
              report_unused_externs: impl Fn(UnusedExterns),
          ) -> Result<(), TestFailure> {
              let langstr = &doctest.langstr;
              // Make sure we emit well-formed executable names for our target.
              let rust_out = add_exe_suffix("rust_out".to_owned(), &rustdoc_options.target);
              let output_file = doctest.test_opts.outdir.path().join(rust_out);
          
              // Common arguments used for compiling the doctest runner.
              // On merged doctests, the compiler is invoked twice: once for the test code itself,
              // and once for the runner wrapper (which needs to use `#![feature]` on stable).
              let mut compiler_args = vec![];

Dit bestand is afgekapt. origineel weergeven

rickwebiii · April 8, 2025, 8:51pm

Is this an oversight or intentional design? Naively, I'd expect to run doc tests with the same profile settings as my application tests.

kornel · April 9, 2025, 10:50am

This is a reasonable expectation. File a bug.

rickwebiii · April 9, 2025, 10:20pm

Filed. doctest doesn't run under the same profile as the rest of the tests · Issue #139603 · rust-lang/rust · GitHub

Topic		Replies	Views
Is the doctest compiler flag broken or is my understanding? help	9	635	March 26, 2023
How run doctest and tests? help	3	791	June 30, 2020
Rust's Documentation Test doesn't works	6	658	June 29, 2021
Testing rust doc help	3	371	August 30, 2021
Why is doctests so slow compared to unit tests and integration tests? help	3	1355	July 22, 2021

Doc tests never run under release mode

Related topics