Documentation of null pointer optimization?

Where can I find authoritative documentation on Rust's null pointer optimization? I've found repr(Rust) - The Rustonomicon, which says:

The classic case of this is Rust's "null pointer optimization": an enum consisting of a single outer unit variant (e.g. None ) and a (potentially nested) non- nullable pointer variant (e.g. Some(&T) ) makes the tag unnecessary. A null pointer can safely be interpreted as the unit ( None ) variant. The net result is that, for example, size_of::<Option<&T>>() == size_of::<&T>() .

But that seems like an aside rather than the official documentation of this feature. Is this officially documented somewhere?

I was in particular trying to figure out whether the optimization applies also to Box<Option<T>> (or &Option<T>).

No, the optimization does not apply to Box<Option<T>> or &Option<T>. In fact, I'm not even sure what it would mean for the optimization to apply for them.

Well, I guess strictly speaking it does apply in the sense that Option<Box<Option<T>>> has the same size as Box<Option<T>>, but I don't think that was what you meant?

1 Like

I guess the documentation of Option is the official point where this is noted

https://doc.rust-lang.org/std/option/index.html#representation

1 Like

UCG reference and the Option documentation.

4 Likes

Excellent, thanks all! I was surprised I'd never see that Option documentation before, until I realized it's on the module-level docs, not the docs for the enum itself. Would it be sensible for me to file an issue or a PR to copy that info to the enum docs?

My thinking was that Box<Option<T>> containing None would be represented as a null pointer, same as Option<Box<T>>.

Thinking about it some more, this would cause problems for Box::as_mut().

1 Like

I don't think that that's necessary. The type itself doesn't really have any documentation at all besides the link to the module level documentation.

This paragraph is literally the whole thing; in my opinion this is clear enough:

The Option type. See the module level documentation for more.

1 Like

On a second thought, maybe it could be a useful thing to create put a (wikipedia style) table of contents in the beginning of the module level documentation and then just reproduce this TOC in the type documentation. This would help discoverability of those contents. This would also apply to a lot of different types. E. g. the docs for Result do the same thing; maybe someone who's already feeling familiar with enums in general would be more inclined to click through to the module level documentation of Result if they could already see that it talks about the ? operator whereas otherwise that person would think that the enum doesn't need any additional explanation.

I don't think we should actually duplicate too many contents, but more links should not hurt. After all, for some reason you either never clicked the link to the module level docs, or at least didn't read through those. (I'm actually curious which of these you think was the case and why. Feel free to try to reflect on that, it could also be useful information for evaluating the user experience.)

If you think that what described above (or some other approach that you can come up with) could make the docs better, feel free to open an issue or start a thread on internals.rust-lang.org.

2 Likes

Funny, I was just reflecting on that. :slight_smile: I am often tripped up by the split between module-level and impl-level docs, in the specific case where the module is mostly geared around a single type. Examples:

My usual workflow when looking for docs is to Google, e.g. "rust str". If I wind up on the module-level page and it says "See also the str primitive type," I click that right away.

I learned that habit mainly from str and slice. For those, the module-level page includes a lot of structs that I've never used, and it doesn't include the methods, which are usually what I'm looking for. The impl page does include the methods. So I've gotten in the habit of thinking "impl docs good, module docs usually not useful," and ignoring the link that says "See the module level documentation for more."

2 Likes

Interesting, just noticed that for slice there's actually some significant duplication.


now that you say that (including the preceding explanations) I feel like that my experience was/is very similar. Which is probably unfortunate/suboptimal if the docs turn out to commonly teach people such a habit.

1 Like

I do the same thing. There's actually a step in the middle where I land on the doesn't-have-most-documentation page, scan the column on the left for the method or whatever I'm thinking of, and then go "oh yeah" and look at the main content area to find the link to the "real documentation".

(I guess I've never really been too bothered because generally Rust documentation is so far ahead of anything else I've had the pleasure to rely on.)

1 Like