Assign a doc comment to a string literal?

The examples I've seen that get and use doc comments in the program are usually done with a macro, which uses the syn and/or quote crates, and then they parse it themselves.

Since Rust itself also uses doc comments to generate documentation, I wonder.. Is there a more built-in way to get this information at compile time, and end up with a const string literal at run time?

The following non-working code is a visual of what I'm trying to accomplish.

/// Add two numbers
fn sum(a: usize, b: usize) -> usize {}

const fn get_docs(func: impl Fn) -> &'static str {
  let documentation = func.attributes.doc.to_string();

  println!("The docs for the passed in function reads: {}", &documentation);
  // "The docs for the passed in function reads: Add two numbers"
}

What would be "more built-in" than a macro, which is a core language feature?

I think you have a fundamental misunderstanding about Rust: doc comments (or regular comments, or any other source attribute/metadata) do not exist at runtime. You can't, therefore, recover the source attributes from a value using a regular, value-level function. You have to do it while the compiler is still dealing in the source at the syntactic level – hence, proc-macros.

I would be curious as to why you need this construct, and what you are trying to achieve with it that a proc-macro can't.

(Incidentally, you can create a #[derive]-like proc-macro that automatically implements a trait for the item you pass it to, and that can be used for generating a const DOC: &'static str associated const with the contents of the docstring.)

3 Likes

I meant the closest examples I've found do this by building their own macro, parsing source code with the syn and/or quote crates. Since Rust does this with documentation generation, I'd like to reuse the existing code instead of building my own, be it a macro or function or whatever.

doc comments (or regular comments, or any other source attribute/metadata) do not exist at runtime.

To be clear, I want everything to happen at compile time and end up with a string in the binary. The example I gave was to convey my thoughts and hope of a simple solution, but perhaps using valid-ish Rust code was a bad choice.

Rust turns doc comments into static strings to populate https://docs.rs with. My use case is similar; I want to turn comments into static strings in the binary so at run time, I can print them.

you can create a #[derive] -like proc-macro

I don't think I can use a #[derive] macro because in my example, I'd like the compile-time const fn get_docs to build the strings from within this function instead of requiring every function I want to pass in be prefixed with a #[derive] attribute. Because, I'm trying to encapsulate this functionality behind the API of const fn get_docs.

Rust turns doc comments into static strings to populate https://docs.rs with.

The rustdoc tool (which you can run yourself on your work-in-progress project, by the way; try cargo doc --open) is essentially a special version of rustc that generates documentation instead of machine code. There's no existing compiler mechanism by which the crate being compiled can inspect the documentation (other than macros reading token trees).

That's why you have to add a macro. You don't necessarily need to write a macro — one might already exist on crates.io — but you definitely have to use a macro at the location of every documentation string you want to preserve.

1 Like

You can include_str! the same source file into a doc attribute and a &static str.

But there's no runtime system for querying attributes of values like in your example. Moreover, you can't implement some custom trait for individual unnameable types like function items... much less other things like enum variants, constants, statics, type aliases, modules, etc.

So I think your proposed API needs a rethink.

1 Like

I haven't tried anything of the sort but I do wonder if it isn't possible to end up with a single proc macro invocation per-file, by using the include macro, and the The #[doc] attribute - The rustdoc book

anyhow it seems at least plausible you might be able to run the proc_macro over the include!() and have it generate constants for all the doc attributes in the file, presumably after rustdoc generates doc attributes from doc comments..

The strategy I would use is to write a procedural macro which, given input like this...

/// Here is really interesting documentation.
#[derive(Documented)]
struct Foo { ... }

... would generate something like this:

/// Here is really interesting documentation.
struct Foo { ... }

impl Documented for Foo {
  const DOCS: &'static str = "Here is really interesting documentation.";
}

Where the Documented trait is defined elsewhere.

Alternatively, you could turn it into an attribute macro which generates a FOO_DOCS constant when annotating a foo() function. Or something like that.

@kpreid
I did not know rustdoc was a sort of fork of rustc like this. Your whole reply helps me understand the boundary being hit here.

And

There's no existing compiler mechanism by which the crate being compiled can inspect the documentation (other than macros reading token trees).

Maybe this would be possible at a lower level?

i.e.
When you pass something to a function, the compiler knows all about what that something is. Where it came from, what its type is, its traits, etc. So perhaps at this lower level, compiler code could be written to recognize a special macro that can read this information.

For example, in my limited experience writing macros I wouldn't be able to write a clone of the compiler built-in env! macro because (I assume) it has insider knowledge. From what I understand, in macros we can only rearrange expressions we can see from within the macro call's parentheses, such as text passed directly to the macro, or code that follows the macro (#[derive]).

Although I'm not good at explaining this, I think the use case is there; rustdoc and crates like clap wouldn't need to each write parsing logic for some of the same things. It could even enable people to write custom doc generation tools, or generate OpenAPI specs for APIs written in Rust, all kinds of dreams could come true. (yes, I dream about perfectly automated documentation for everything, not just seeing them on https://docs.rs)

@quinedot

You can include_str! the same source file into a doc attribute and a &static str .

Wouldn't this just put a string into a doc comment? I'm trying to put a doc comment into a string (At compile time, of course).

But, you're right - I'm not married to the proposed API, and must of course work with what's possible.


@Michael-F-Bryan

This is indeed the most common way I've seen others do it.. it just would make me a little sad having to don #[derive] on things all over the place. Also, would this even work for functions? I've only ever seen it on structs.

Maybe this would be possible at a lower level?

My point isn't about how to implement this; the compiler could certainly have this feature. I'm saying only that it currently does not.

… a special macro that can read this information.

If I were designing it, I wouldn't make it a macro at all; I'd say that there was a special trait implemented by all structs, enums, and function item types like:

trait Documentation {
    fn doc() -> &'static str;
    fn doc_by_val(&self) -> &'static str {
        Sel::doc()
    }
}

/// Example function
fn example() {}

/// --- The compiler effectively generates this impl for us ---
impl Documentation for <typeof(example)> {
    fn doc() -> &'static str { "Example function" }
}

This has the advantage that it's a completely ordinary trait function and can be understood just like any other auto-implemented trait.

(That said, if you want to engage in further discussion of the design of a hypothetical or proposed documentation-reflection feature, https://internals.rust-lang.org/ is the best place to do that.)

rustdoc and crates like clap wouldn't need to each write parsing logic for some of the same things.

It was an intentional design decision that macros receive TokenStreams and not anything more structured. This means that the compiler maintainers do not have to preserve compatibility — across all language versions — with a more complex syntax tree or reflection interface.

1 Like

Ah yes, I like your hypothetical idea much better!

What I meant by

rustdoc and crates like clap wouldn't need to each write parsing logic for some of the same things.

was if this feature were to exist, nobody else would have to think about macros at all, if something like your idea was the solution. And if a built-in macro was the solution, they wouldn't need to reimplement it.

Many thanks for the continued discussion on this @kpreid, I'll see if I can collect my thoughts for further discussion on that forum and link to this thread for context.

Would it work this way, however? Doc comments are essentially just attributes, and attributes are processed in order, i.e. doc comments will probably not be visible to macro unless it it positioned before them. Can't check right now, though.

Yeah, it works.

I once made a system for implementing Project Euler problems where you'd create a struct and use a custom derive to register the implementation with the inventory crate instead of needing to write a central list. I was writing the problem's text in the implementing type's doc-comments and they would be stored in the derived trait as an associated constant.

I don't think custom derives can see the presence/attributes from other derives, though. Not sure how it works for attribute macros.

1 Like

You can put the contents of a file into both a doc comment and a string, so they stay in sync. (It's not a full solution for whatever you're trying to do, but maybe it could be part of it.)

1 Like

If you want to replicate rustdoc and process the whole crate in one go, and you still don't want to use macros (despite the repeated explanation of several people as to why you should), then write a build script and call it a day. A build script can basically do literally whatever, including parsing all the source files in the crate and extracting the doc-comments from them.

1 Like