What are smart pointers?

[jbe]

Disclaimer: I have virtually no experience in C++ myself, so I'd like to apologize if I might not grasp some things right here.

@Sykout

I think that "Smart Pointer" is quite an archaic description that is mostly used in the context of C++, […]

Yeah, as I learned in the previous thread, it is originating from C++. I'm not exactly sure if it's clear what it is in C++ though .

I'm not sure what you mean with:

@Sykout

A struct/class that is generic to a caller defined type

Do you mean a type constructor like struct T<U> { /* … */ }?

I got the following rough description regarding the origins of "smart pointers" from @trentj in the previous thread:

@trentj

The term "smart pointer" comes from C++ where it basically just means any pointer-like type that overloads operator* and operator-> (and usually has a destructor, although I guess that's not strictly necessary as you could do all the "smart" things in the overloads...)

You describe it as…

@Sykout

[…] a type that is:

• A struct/class that is generic to a caller defined type
• Contain a destructor

We had similar differing p.o.v.'s regarding Rust, where you could describe a smart pointer

• formally as "implements Deref" (in C++ that'd be overloading * and ->, I guess?),
• or by talking about what the type contains, which other properties/behavior it shows (e.g. uses "indirection" (whatever that actually is) or has some drop glue / destructor, etc).

I don't want to compare this 1:1, but I observe that both in C++ and in Rust, there seems to be a "formal" view to say what is a smart pointer:

• In C++: overloads * and ->
• In Rust: implements Deref (or DerefMut in case of mutability)

And we have other views that reason by semantics or other properties whether we have a smart pointer or not.

Even if the term "smart pointer" originates in the C++ world, we can't deny that it exists in the Rust world:

1. Smart pointers are mentioned in the reference as being contained in the standard library.

2. The documentation of the standard library also refers to the term "smart pointer" by warning that Deref should only be implemented for "smart pointers".

Regarding 1:

Smart pointers

The standard library contains additional 'smart pointer' types beyond references and raw pointers.

Regarding 2:

On the other hand, the rules regarding Deref and DerefMut were designed specifically to accommodate smart pointers. Because of this, Deref should only be implemented for smart pointers to avoid confusion.

Now if we would define smart pointers in a formal way, then we would have a problem with the documentation of Deref as the warning notice is not really helpful: We should only implement Deref for smart pointers, but anything that implements Deref is a smart pointer, so everything is fine always .

Thus for me as a "user" of Rust, I need to understand when to implement Deref and when not. Currently, the documentation ties this to the phrase "smart pointer", thus I see a need to discuss the definition of what is a "smart pointer" in a Rust context, ideally not by defining it through "implements Deref" as that leads to a cycle.

I think it's reasonable to look at

• which types implement Deref in order to get a clue what's really special about them, and
• the consequences of implementing Deref (mostly automatic coercion, I think).

I would also like to note the following important note in the Deref documentation:

[…], [the] trait [Deref] should never fail. Failure during dereferencing can be extremely confusing when Deref is invoked implicitly.

@Sykout mentioned Option<&T> and MutexGuard:

@Sykout

With those two classification, we can split up most caller defined generic types in this matrix

[…]

I find these distinction more useful in analyzing code, particularly in function definition. […]

Both Option<&T> and MutexGuard are in the "Singular" and "View" categories.

While I think it's an interesting classification, I'd like to point out that Option<&T> is different from MutexGuard (which implements Deref) in that way that &Option<&T> cannot be converted to &T without failure, while &MutexGuard<T> can. That is an important difference and the reason why Option<&T> must not implement Deref<Target=T>.

This leads me to the following attempt to define what a "smart pointer" is, in the Rust world:

A type T is a smart pointer to U if it fulfills all of the following properties:

1. T is neither a raw pointer to U nor a reference to U.
2. There exists a conversion from &T to &U which should never fail.
3. Due to semantics, automatic coercions from &T to &U are always wanted at coercion sites.

Admittingly, rule 3 is a bit fuzzy yet. But what does these rules say?

Rule 1 basically distinguishes smart pointers from other "pointer types" in Rust. In this context, it is also interesting what @Sykout said here:

@Sykout

[…] &T/&mut T dropping out of scope can actually change what is valid code, which raise the question if &T/&mut T are "Smart" pointers

I would say there is really a strong similarity between smart pointers and references. I would say the only reason why (Rust) references aren't smart pointers is a matter of definition (hence rule 1).

Rule 2 basically says that if we get/have any T by reference (i.e. we actually have &T), we can always get a (normal) reference to U (i.e. we can convert to &U). In other words: T points to U, because we can follow (i.e. dereference) a T (we only need a reference to T for that) to get to an U (with immutable access only, i.e. we get a &U).

Rule 3 is most tricky, because it's quite blurry yet. When do semantics really want us to enable automatic coercion? This rule again is a bit cyclic as it basically says that it's a smart pointer if we want to implement Deref on it. If I remember some other threads correctly, most people said the newtype pattern should not use Deref and thus do not enable automatic coercion. The same holds for something like inheritance, where Deref and/or automatic coercions are a bad idea. (I think it's more obvious on the latter case, and maybe there is no 100% consensus on all newtypes? Or is there?)

Let's look at some examples:

• References fulfill rule 2 (we can convert &&T to &T) and rule 3 (we can pass a &&T where a &T is expected), but they fail rule 1.
• Option<&T> fulfills rule 1 (it is neither a raw pointer nor a reference), but fails rule 2 (we cannot convert from &Option<&T> to &T without panicking in the None case). Thus also rule 3 is failed.
• Mutex<T> fails rule 2.
• MutexGuard<T> certainly fulfills rule 1 and 2. I guess whether rule 3 is fulfilled is a matter of preference (due to how blurry that rule is yet).
• Box<T> fulfills all three rules.
• Finalized<T> (as defined above) definitely fulfills all three rules.
• Cow<T> also fulfills all three rules.
• Pin<T> with <T as Deref>::Target being the type "pointed to" also fulfills all three rules.
• CaseInsensitive (as also defined above) is the newtype case, where it's usually said that we do not want automatic coercion (thus failing rule 3).

It would be nice to phrase rule 3 in a better way than this:

Due to semantics, automatic coercions from &T to &U are always wanted at coercion sites.

Edit: Added Mutex<T> in the list of examples.