Tests for documentation output?

help
1

Hello,

I'm trying to help a library use @alice's helpful post here to get some feature flags requirements to show up on docs.rs.

In working on this, I got wondering how to write some kind of regression test to ensure that the HTML output for the docs contained the feature flag notification. How might one go about this? Was just thinking of something that would run the RUSTFLAGS=... cargo doc ... and then search the output HTML.

I've seen approaches like assert_cmd - Rust for CLI apps -- is there a similar approach for the html output for docs?

Likely easier to just do this in GitHub Actions with some bash -- is there a pure Rust approach that people might recommend?

2

I'm not quite srue I understood your question. But I guess you want to ensure all #[cfg] are documented with a banner[1]
image

image
image705×274 17.4 KB

Then with doc_auto_cfg feature from rustdoc, you don't have to define docsrs cfg flag only for #[cfg_attr] and you don't need to write the second line for every #[cfg]:

#[cfg(feature = "rt")]
#[cfg_attr(docsrs, doc(cfg(feature = "rt")))] // You don't have to write this line any more
fn f() {}

An example:

// src/lib.rs
#![feature(doc_auto_cfg)]

#[cfg(custom_flag)]
pub fn cfg_banner() {}

#[cfg(feature = "custom_feature")]
pub fn feature_banner() {}
  • run cargo +nightly doc --all-features
Toggle to see pictures. 
// equivalent in Cargo.toml
[package.metadata.docs.rs]
all-features = true

image
image

  • run RUSTDOCFLAGS="--cfg custom_flag" cargo +nightly doc --all-features
Toggle to see pictures. 
// equivalent in Cargo.toml
[package.metadata.docs.rs]
all-features = true
rustdoc-args = ["--cfg", "custom_flag"]

image
image
image

  1. if the banner is the notification you memtioned ↩ī¸Ž

3

Thanks for your response!

Yes, I submitted a PR with this exact change, but it didn't work for some reason.

My question for this thread is how I could implement some kind of test in CI that would ensure it was present in the generated HTML.

4

Each banner seems to have a class="stab xxx" attribute in its HTML tag, so you could write a shell script like:

# untested; replace PROJECT with the package name
doc_auto_cfg=$(grep -P 'class="stab \w+"' target/doc/PROJECT/* | wc -c)
if [ $doc_auto_cfg -ne 0 ]; then
    echo "doc_auto_cfg works"
else
    echo "Error: doc_auto_cfg is broken" 1>&2
fi
5

Definitely -- easily done in bash. But is this what people would recommend? Again, using CLI apps as an example, many people have found a good way to do integration tests in rust for command line apps. Is anyone doing anything analogous for the documentation output?

Or should I just stick with a non-rust solution?

6

I don't have a suggestion for that. Just in case one wants to try testing in the Rust way:

#[test]
fn test_doc_auto_cfg() {
    use std::process::Command;
    const DOC_DIR: &str = concat!("target/doc/", env!("CARGO_PKG_NAME"));

    // Run `cargo doc --all-features --no-deps`
    assert!(
        Command::new("cargo")
            .args(["doc", "--all-features", "--no-deps"])
            .output()
            .expect("Can't run cargo doc")
            .status
            .success(),
        "cargo doc doesn't exit successfully"
    );

    // Run grep to check the existence of tag property in HTMLs
    let output = Command::new("grep")
        .args(["-P", r#"class="stab \w+""#, DOC_DIR, "-d", "recurse"])
        .output()
        .expect("Failed to execute grep");
    if output.status.success() {
        let stdout = String::from_utf8_lossy(&output.stdout);
        if !stdout.starts_with(DOC_DIR) {
            panic!("The result of grep is unusual:\n{}", &stdout[..100]);
        }
    } else {
        panic!("{}", String::from_utf8_lossy(&output.stderr));
    }
}
1 Like
7

Thanks for the edit -- I was going to mention how CLI integration tests with assert_cmd mean that the cargo build step is assumed, and how cargo doc might not be run yet. But your edit cleans that up!

I was hoping there would be something cleaner, but that certainly seems like it would do the trick.

I wonder if there is space for a new crate here; perhaps I'll make a POC using this as a template? (With @vague's permission of course -- ?)

8

Feel free to use it. I'm glad it's helpful.

1 Like
9

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.