Tool to extract data from comments?

1

Hi!

In a project I implement a (proprietary) protocol. I want to be able to add some metadata as comments in my code, to specify which part of the protocol is implemented where. Think of something like:

/// Implements: spec-3.1.0.1, spec-3.1.0.2
struct Something;

for example. I want then to be able to extract this meta-information and cross-check whether all specification items are linked (that part is on my side, using some pdf-to-text and sed magic or something).

Does someone know a tool to extract metadata from rustdoc comments, possibly with source code location? The format how to specify the metadata doesn't matter (does not have to look like the above), as long as it is human readable :laughing:
Or does someone know an alternative way to do this?

2

If the format of the meta-information is so simple a grep or awk script is probably enough.

3

I really want to experiment with rustdoc's --output-format=json (unstable) feature one day, especially for diff-ing doc builds in CI. But I believe the generated JSON could also be useful for extracting data from comments.

1 Like
4

yeah but that does not give you other interesting information like what type/struct/foobar is annotated...

5

You said "source code location", i.e., file name and line number. If you want more obviously you'll need a tool that can understand, at least a bit, Rust syntax. Sorry, I can't help you there.

1 Like
6

Write it yourself(in python is easier, in rust for convenience), or give me specifications of what you wanted it to do and i will write it for you in rust

7

You can parse Rust source code files with syn:

and /// comments are parsed as doc attributes.

4 Likes
8

Maybe you also want to consider implementing this requirement in Rust's type system instead.
I don't know the details of your program / library, but you could implement traits for each spec and require the full protocol to implement all traits.
I did something like this here: ezsp/src/commands.rs at ef9d37a4aed1401f92a16ea195c0c3c509ea1cae · PaulmannLighting/ezsp · GitHub

9

I know nothing much about it but this sounds like a job for tree-sitter: tree-sitter/lib/binding_rust at master · tree-sitter/tree-sitter · GitHub. Which as far as I understand can extract comments as well.