Maintaining example code in rustdoc comments

help
1

I've been using Visual Studio Code with the Rust Analyzer for years now and it's nearly flawless except for maintaining example code within rustdoc snippets. Here's an example:

/// StaticFunction implodes the universe into a conveniently sized travel lozenge
///
/// ```
/// use crate::ImportantThing;
/// ImportantThing::StaticFunction();
/// ```
fn StaticFunction() {
    todo!()
}

What I find problematic is during refactoring, I may have renamed the function or imports, or both and so I need to go back and edit without the convenience of code completion, and the rest.

I was wondering how others handle this. In an ideal world I could reference some test code that would get imported during the doc building phase, maybe with a macro named include_function!. Something like this:

/// StaticFunction implodes the universe into a conveniently sized travel lozenge
///
/// ```
/// include_function!(test::test_static_function)
/// ```
fn StaticFunction() {
    todo!()
}

#[cfg(test)
mod test {
    #[test]
    fn test_static_function() {
        use super::StaticFunction;
        StaticFunction();
    }
}

Thoughts? Any ideas on how to maintain rustdoc example code is welcome.

2 Likes
2

I don't have a good way for you, I just run cargo test --doc and fix things manually, sorry :neutral_face:

1 Like
3

could you do something like

/// ```
#[doc = include_str!(test_foo.rs)]
/// ```
pub fn foo() {}

remember that you can use macros either to generate attributes or within attributes, and that doc comments are just syntactic sugar for the doc attribute

1 Like
4

To be fair, breaking changes aren't usually very common once you've gotten far enough to add doc examples like that.

1 Like
5

in many cases, I would suggest to use the "scraped examples" feature:

https://doc.rust-lang.org/stable/rustdoc/scraped-examples.html

it is supported by docs.rs too, and many crates are already using this feature, here's an example how it looks (you are not limited to a single example, rustdoc will automatically find all references, notice the "+More examples" button below?):

6

Thanks for the advice everyone! I think maybe this is worth an RFC... I like the idea of spending a little extra time on my unit tests so they're readable and well documented for newbies, then referencing them from the rustdoc block. Having those tests defined in individual code.rs files isn't the canonical way of writing unit tests so that would feel weird for both creators and consumers of public APIs.

Scraped examples is neat but looking at winit examples (I actually use this crate in a project) they're not friendly to read... Getting work done and making things understandable don't always go hand-in-hand and several examples in those docs aren't terribly useful.

Doing some digging on the doc macro... It might be that the best way to implement what I'm thinking would be to extend that macro. Something like:

/// StaticFunction implodes the universe into a conveniently sized travel lozenge
///
#[doc(example(test::test_static_function))]
#[doc(example(test::test_static_function_failure))]
fn StaticFunction() {
    todo!()
}

I'm struggling a bit to find where the macro is defined. Looking at rust's source I'm only finding unit tests. Any ideas on where it's defined so I can hack on the idea?

7

To be honest, this sound like an editor problem mor than a language problem.

8

Yeah, I can see that perspective. For me, it feels a bit unreasonable that rust-analyzer needs to know how to refactor something hidden inside a comment. Also, as a thought exercise, if this were the canonical way to provide examples there wouldn't be a need to run unit tests against documentation.

9

@GuillaumeGomez it looks like you have a few commits around testing this macro. Would you be willing to show me where it's defined so I can experiment? I think it's likely best to make a proof-of-concept crate that just automagically generates doc macros but it'd still be nice to see how it's implemented.

10

One weakness with this approach, which you may want to spend some time on before putting together an RFC, is that an item's attributes do not participate in the narrative structure of documentation, while examples do. For an example of this in practice, consider the examples in:

and numerous others.

Documentation, taken as a whole, is written to be read; doctests are there to ensure that the documentation is correct and that the examples will actually work if followed, as well as to act as tests for the code being documented.

As Rust stands today, a #[doc(example…)] attribute does not obviously allow these authorial choices, as there is no clear way to position the example within the larger flow of the documentation.

For what it's worth, I love the bones of the idea, and I do think it'd be useful to have some way to take larger examples out of code blocks inside /// comments, for ergonomics reasons if no other, so I hope you do keep working on this.

3 Likes
11

FWIW, Rust documentation isn’t really “comments”, or rather, it doesn’t have the traditional problems with comments being outside of the normal AST; the source code

///foo

is completely equivalent to

#[doc = "foo"]

and lives in the attributes section of the AST; there is no need to track them specially to preserve them like regular comments. It’s still true that code in documentation (doc-tests) is more troublesome to manpulate, but that’s because:

  • the code sections are notated in Markdown, and that Markdown isn’t part of the normal Rust AST (but rust-analyzer already knows how to parse these for syntax highlighting at least), and
  • doc-tests are compiled as separate crates from the library they are for, and Cargo doesn’t do that (instead, rustdoc does while running tests) so there isn’t an easy way to compile them and extract metadata from them like there is for normal Cargo-managed code.
12

first of all, doc is an attribute, not a macro.

each line of comment is equivalent to a #[doc = "blahblah"] attribute in the parsed AST. however, this attribute is preserved but not used by rustc (the compiler), but is processed by rustdoc.

and yes, I agree, to extend the current doc attribute is probably the most viable path to go. currently, the valid attributes that I know of include #[doc(hidden)], #[doc(inline)], #[doc(alias = "foobar")], and some unstable ones like #[doc(cfg(xxx))]. it feels natural to me to add more features here.

in fact, the compile diagnostics you see when you compile source code with the "extended" attribute is not generated by the compiler itself, but by a builtin lint.

so, if you disable the lint, code like below can actually compile, on the current compiler. it's just the "extension" isn't understood by rustdoc so it is completely ignored:

// src/lib
#![allow(invalid_doc_attributes)]
#[doc(example(tests::bar))]
pub fn foo() {}

$ cargo build
   Compiling hello v0.1.0 (/tmp/hello)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.07s
$ cargo doc
 Documenting hello v0.1.0 (/tmp/hello)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.18s
   Generated /tmp/hello/target/doc/hello/index.html

if you want to play with the idea of extending the doc attribute, here's what I think you need to do:

  • first modify the invalid_doc_attributes lint to accept the new syntax;
  • then add a rustdoc pass to consume the attribute;
    • e.g. to get started, just insert the source code of the specified tests as code blocks into the docs;
    • presumably also add the no_run or ignore attributes, since it's already tested as unit tests.

unfortunately I'm not a compiler hacker, I cannot give more detailed instructions, but the tracking issue for invalid_doc_attributes and the exising passes of rustdoc can be a good starting point.

13

Since doc attributes can use other macros to get their string content it is possible to mostly implement this feature in a library:

// Make "playground" available as an "external" crate name to easier write doctests
extern crate self as playground;

/// StaticFunction implodes the universe into a conveniently sized travel lozenge
///
/// ```
#[doc = test_static_function!()]
/// ```
pub fn StaticFunction() {
    todo!()
}

// #[my_custom_attribute_macro] // <- user would add this
fn test_static_function() {
    // Don't use super::StaticFunction or crate::StaticFunction here since that would fail in the doc test.
    use playground::StaticFunction;
    StaticFunction();
}

// #[my_custom_attribute_macro] would expand to:
macro_rules! __test_static_function_get_code {
    () => {
        ::core::stringify!(
            use playground::StaticFunction;
            StaticFunction();
        )
    }
}
pub(crate) use __test_static_function_get_code as test_static_function;

Which would cause cargo test to fail because of the todo!() call in StaticFunction during the doctest.

See longer example with declarative helper macro at the playground.

Rendered by cargo doc as:

pub fn StaticFunction()

StaticFunction implodes the universe into a conveniently sized travel lozenge

use playground :: StaticFunction; StaticFunction();

Fix cargo doc formatting

Here we can see that the the doctest code loses its formatting so it would be better to run some kind of formatter (maybe prettyplease or just call out to rustfmt) inside a proc-macro when generating the __test_static_function_get_code macro.

Keep comments

We can also see that comments were lost in rustdoc's output. I don't think comments are visible to macros so it isn't possible to keep them using this approach.

An alternative approach would be to implement this as a procedural macro that manually parsed the current project's code using syn and then found the location of the wanted function:

  1. #[doc = doctest_proc_macro!("path/to/module.rs", test_static_function)]
    • The first path could be optional if we manually parsed all modules using for example the syn-file-expand crate.
  2. Proc macro parses file using syn.
  3. Proc macro finds the wanted function test_static_function and gets the code in its block.
  4. Proc macro removes leading spaces.
  5. Proc macro expands to a string containing the trimmed function code.
1 Like
14

You were already proposing a proc macro be involved anyway though, so the string-conversion just happening earlier would fix it, wouldn’t it?

I.e.

// #[my_custom_attribute_macro] would expand to:
macro_rules! __test_static_function_get_code {
    () => {
"use playground::StaticFunction;
StaticFunction();"
    }
}
1 Like
15

I think proc macros only get tokens so even if they are converted to a string immediately it would still lose the formatting. Perhaps it is possible with proc_macro::Span::source_text.

16

Yes, perhaps it is indeed. The function’s body should be a Group that can give you a single Span already on stable rust.

17

Tested Span::source_text on the playground and it actually works!

Don't know if we can rely on that function always working though. But since it is only for documentation tests then perhaps the macro could emit an empty string if it fails (and hopefully a warning). Actually it could probably just emit a string like panic!("failed to get doctest") (that would cause doctests to fail if it was included somewhere).

Edit: fully implemented attribute macro

1 Like
18

Just double-checked, for reference, and RustRover does correctly refactor in doctests (and highlight and give suggestions) - though notably only in actual doctests, arbitrary code fences in eg bins don't do anything.

1 Like