Should rust force good documentation usages?

1

Hi :slight_smile:
around a month ago I opened this issue on rust internals, describing an idea to try to fix bad documentation: rust internals

I only got two replies which made me question if my idea was actually worthwhile and whether I should actually invest more time in it.

In summary, my main points are:

  • rust was built in a smart way with smart choices which makes people write better code, yet it does not do the same with documentations
  • there are ton of crates who do not have high-level documentations, thus leaving you with no choice other then reading the source code or scraping the limited amount of examples
    .. notable examples: egui, bevy (only the non-popular stuff, in particular bevy-render)
  • there are a lot of resources regarding "note-taking" and knowledge representation in general which can be translated into good documentation usages. We also got decades worth of programming experiences regarding documentation writing. None of this is actually implemented, in some way or another.
  • everybody loves to code fun and very technical stuff but noone cares about writing good documentation, which is why I thought that forcing people to write good documentation would have been a good idea

What do you think?
P.S. I obviously don't know a perfect solution to solve these issues: I'm still questioning the fundamental principle of forcing people to write good docs.

2

For addressing lack of examples, developers can choose to enable a lint (at least for nightly rustdoc) to avoid forgetting examples for any API. For higher-level documentation, similarly, at least there’s this lint reminding you to have a top-level documentation at all, which is probably supposed to become available on stable, and a warning by default eventually.

3 Likes
3

Nothing works better than good examples. IMHO what should exist is:

  1. Some working group perhaps that would create documentation for selected high-profile crates to show how it should be done and create guidelines.
  2. Maybe some sort of clippy rules for documentation if it makes sense

Without people feeling a need for documentation and peer pressure of "can you do what X did? their docs are great" enforcing anything will only annoy people.

Btw this is golden standard for me:
https://requests.readthedocs.io/en/latest/

What Rustdoc generates is for more people (especially newcomers) just not very usefull.

4 Likes
4

You can't force people to write good documentation. Lints and such may help a little. I would instead suggest figuring out how to promote a culture that values the sort of documentation your want to see. This is hard to do.

26 Likes
5

Can you expand on this?

6

I would expect to see a lot of

  /// [Description]
  /// Returns the length of the list.
  /// [How to use]
  /// `list.len()`
  /// [When to use]
  /// When you want to know the length of the list.
  fn len(&self) -> usize {..}
5 Likes
7

Might have phrased it wrong; what I wanted to say is that every time someone has to write a new documentation he has to start from scratch without any guideline. In addition to that, I think it's difficult to have feedback regarding documentation, because the people who are reading it are reading it in order to understand something, thus they obviously don't have not enough knowledge to comment on bad documentation. I think this leads to people who start documenting without knowing how to do it, with the addition that they keep doing so because they rarely get feedback regarding how to do it.
People with a lot of experience in programming (not me) must certainly have a lot of insights regarding documentation writing and bad / good practices.
The "None of this is actually implemented" refers not only to the hypothetical "experienced programmers" but also to the before-mentioned researches (mostly psychological ones), referring to findings and criteria that characterize how we learn and understand stuff

8

Oh yeah, it was actually mentioned as a problem in the original post but you can obviously find some workarounds and solutions to these problems; I'm currently more focused at understanding the nature of this problem, which would potentially lead to "trying to find a solution for it"

9

Nope. S/he starts with what rustdoc produces by default. And then we immediately hit the wall. Namely: why the heck what Rust produces by default is bad? When I wrote my first proc_macro I opened the documentation for proc_macro and I immediately liked it: short, concise, tells me everything that I need to know. What's not to like? Yes, sure, I have read the little book, thus I knew what to expect… and this auto-generated doc is perfect “content list” for the actual documentation (which is, of course, source of the proc_macro).

For me the most important knob to have in the documentation is that blue source link near structs, functions and so on — and it's always there in any rustdoc-produced documentation. Except it doesn't work when macros are involved, heh. I would love to see that fixed somehow, but even from your proposals it's obvious that your “dream documentation” is something radically different.

Nope. I have decades of programming behind my belt. I still have no idea how to write documentation that other people would perceive as good. And what I perceive as good and important (high-level overview of all the parts and components… enough for me to dig in the actual code and find out the places where things are actually happening) many novices would consider “not enough”, or, even worse, “not important”.

The short answer. We are different and have different expectations. As I have said: to me all the manually written documentation is gloassary or table of content for them actual documentation: source code. But I'm spoiled: I worked either on open source or, at least, internally-open source (where code is open for the developers even if not for users) all my life. People who don't have that luxury expect different kind of documentation. Because they couldn't see the source and have to understand how to use libraries which are black boxes for them.

The problem is that people are different. As already noted before.

For me the source code is a documentation. Very concise and detailed but without intro. Add that intro (in a form of some examples) and I'm happy.

But for many documentation does not include the source code. Either because they worked with closed-source libraries or, sometimes, because of how they are taught. The them the need to actually look in the source code is a failure in documentation.

Both camps exist for half-century, both camps are perceiving that what they are doing is the right thing to do… how do you propose to reconcile these two approaches?

And that's just very first division where people are split on topic of what good documentation even is! There are many others.

10

I might have misused the word "force". You can always guide programmers and bother them if they don't write good documentation, which would lead them to "choose" not to write good documentation, which is completely different from the current status of not doing anything at all (the example-scraping lints are optional)

I think this topic is more complicated then it sounds; you might get to know how to use something by looking at an example but you won't know what the different things actually do (unless they are intuitive), i.e. you won't get a deeper knowledge about the stuff you are using.
This would lead to people copy-pasting stuff and getting stuck when they want to do something different (me included); I'l call on the "copy-paste from stackoverflow" memes that are born from these situations

11

99% of the time all I care about is the example - either because I understand what it does and just need the syntax, or I know nothing and want some starting point I can work with, and I'look for more detailed guide later (that would be the case with async runtime for example for someone new to the idea). If some explanation is required, you can always write under the example.

To be more specific what I usually want:

  1. Hello world example in the front
  2. Menu of topics/other examples so I can get to specific one quickly
    .those may have minimal amount of comment to explain what's happening
2 Likes
12

Additionally, there are a lot of flavors of documentation to consider. API docs are good for fine grained information about public methods and types, but it's very hard to write API docs that can point readers in the right direction [1].

Examples and tests both serve as documentation (which has been mentioned before). If nothing else, they provide precise information about call sites for the APIs. The tooling around these is pretty good, but nothing prevents maintainers from just not having any tests or examples.

Prose, book, or tutorial form documentation currently lives completely outside of the compiler, and often does not even live in the same repository as the project it documents. This flavor of documentation can provide some of the high-level context that API docs inherently lack. This is where technical writers come in.

Finally, I would say blogging is another form of documentation. Perhaps to describe historical decision making, or to help clear the air on relevant challenges, describing "the right way" to use some piece of code, and so on. Blogs are much less organized than any of the others, but equally important. [2]

There are a lot of things that can't be solved by adding a new feature to the language or compiler.

  1. I cannot count how many times I have searched docs.rs for some crate and found a type that seems relevant to my question, but the docs for that type don't tell me anything about what methods on other types return it. ↩︎

  2. Aggregators like This Week in Rust (this-week-in-rust.org), Hacker News (ycombinator.com), and Lobsters can help a bit with organization. Q&A sites like stack-overflow also fall under this category of disparate pieces of useful information. ↩︎

13

It can't.

I've seen what happens in languages that try, and it's worse than not trying. See, for example, how often C# code ends up with documentation from SubMain / GhostDoc - Painless Help Documentation because "it has to have documentation".

Having to copy

/// <param name="cancellationToken">The token to monitor for cancellation requests. The default value is None.</param>

onto every cancellable method to meet the "good documentation" rule doesn't actually improve anything.

Adding a "you have to document it" rule is how you get garbage like BlobAccessPolicy Class (Azure.Storage.Blobs.Models) - Azure for .NET Developers | Microsoft Learn being described as "An Access policy." with its DateTimeOffset? PolicyExpiresOn property getting a description of "the date-time the policy expires."

The only way to get good documentation is to not use libraries that don't have it. Forcing it can't work.

11 Likes
14

To add to the dog pile ( :stuck_out_tongue: ): forcing good documentation is equivalent to forcing people to become good technical writers. Which is like a teacher trying to force every student in their language class to become good fiction writers. You just can't. It's not a thing, as a species, we know how to do, if it's even possible in the first place.

This is why people like Steve Klabnik and Carol Nichols (among others) are so incredibly valuable: because they can write good documentation.

The best you can do is find ways to lower the skill/training/time barriers to writing documentation. For example, it's likely that part of the reason for lacking good top-level documentation is that there's basically no support in rustdoc for prose that is not structured alongside code. If I want to provide a "book" on how to use a library, I can't reasonably have it as part of the rustdoc, I have to write it completely separately, and find somewhere separate to host it, and make sure it's linked and kept up to date, and... oh wait, I need to make sure old versions are also available if someone needs to understand a previous version and oh great I just realised I can't easily use code tests to make sure what I'm writing is actually correct and the examples work, so I need to sort out infrastructure for that and make sure tests get run and...

...you know what? Stuff it. I don't have time for this, even if I was any good at it in the first place! Which I'm not!

And so, high-level docs don't happen.

There's possibly also something that could be done in terms of training, to help programmers improve as writers, but I have no idea what that would look like.

So, yeah. If there was an easy solution to this, we wouldn't be having this discussion. :slight_smile:

2 Likes
15

well said.

don't try to manage or moderate the community, users will choose high quality libraries naturally.

(although not a good metaphor, it's like regulated market vs free market. Economic principles apply to many fields beyond economics)

3 Likes
16

totally agree! in many cases, an example gives you more context than a long paragraph of text description.

there's a unstable feature scraped-examples, but unfortunately it's not widely used. I rarely encounter them when reading https://docs.rs I think partially because not many people know about it, but sometimes it's due to misconfiguration causing doc.rs build failure.

some examples about what it looks like:

in my opinion it's much more pleasant to write actual example programs than to write small code snippet nested in markdown doc comments.

1 Like
17

In the past I've found GhostDoc to be very useful, but not if you use it blindly. I'm conscientious enough to take it as a starting-point and modify as appropriate. But if it were mandated then no doubt you will just end up with the situation you describe!

18

And many others, I guess, would want to know not only "what's happening", but "what's expected, what we want to guarantee". That is, somehow draw the line between actual API and implementation details. That's what the good documentation is about (partially), I think.

19

In my experience, the way to good documentation goes roughly like this:

  • Someone writes something useful
  • Many people use it
  • Many people reach out to the author(s) when unsure how to use it
  • The author(s) realize(s) that adding some point to the documentation would reduce the number of questions users have
  • Repeat many times
9 Likes
20

If raise the bad that hight then I'm afraid “good documentation” doesn't even exist.

Foe example Rust's std documentation tries to express that line explicitly and dozens (hundreds?) of people work on that (at least sometimes). Still Hyrum's law delivers: people find a way to rely on some obscure undocumented properties.

In practice if you want for people not to rely on something you just have to break that something from time to time. Like, e.g., if Rust wants to ensure that people wouldn't write code that relies on struct layout then plastering the documentation with warnings would work much less reliably than option that would actually randomize struct layout or, even better, for that option to be enabled by default, at least in debug builds.