(This is a spinoff from this thread that felt like it should be discussed independently of the proposal there)
I’m trying to understand the purpose of this warning in the Deref docs:
... the rules regarding Deref and DerefMut were designed specifically to accommodate smart pointers. Because of this, Deref should only be implemented for smart pointers to avoid confusion.
I have always interpreted this to mean that Deref should only be implemented for an object that serves as some kind of proxy for another. By analogy with Box, I often write code like this when I want a static guarantee of some runtime property:
pub struct MyNewtype<T>(T);
impl<T> std::ops::Deref for MyNewtype<T> {
type Target=T;
fn deref(&self)->&T { &self.0 }
}
impl<T> MyNewtype<T> {
pub fn new(val: T)->Result<Self, ()> {
// Verify some property of val here
Ok(MyNewtype(val))
}
pub fn into_inner(self)->T { self.0 }
}
According to the arguments that @Kestrer and @jschievink put forth in the original thread, however, this is an incorrect implementation of Deref because MyNewtype contains T inline instead of storing it on the heap. I dislike this line of reasoning because it uses a private implementation detail to argue about what the public interface should be. Is there a way to describe users’ expectations in terms of only the type’s public interface?
Stepping away from the terminology of “smart pointer”, what behaviors should types that implement Deref exhibit in order to avoid the user confusion that the docs warn about?
If this particular newtype pattern doesn’t exhibit these behaviors, is there an example of a similar pattern causing confusion? What is a preferred formulation of this pattern?
I always assumed this warning was meant to discourage the use of Deref for type casting (or similar unintended goals) outside of simple "reference taking". I don't see the problem in using Deref for a non-heap allocated value.
References can be dereferenced, but your wrapper type doesn't forward a dereferential action, you just borrow the inner value as-is, thus it is appropriate to implement Borrow, but not Deref.
C++ showed what excessive operator overloading does to a language, just for a bit of syntactical sugar, ignoring semantics completely…
Doesn’t this exact same argument apply to Box<T>? In my conceptual model of Rust, it’s not referring to some external T, it’s providing access to its owned T; where it chooses to allocate memory for that T is none of my business.
Box is a magical language item. It does not store the item in itself, but is an interface to operate with an owned value stored on the heap, i.e. Boxis a reference. Box<T> was represented as ~T, back in the early days of Rust, which captured its special purpose better, IMO. Box is kinda like &mut, but with the additional property of owning the pointed-at value.
EDIT: To showcase how special Box is, this is its implementation for Deref and DerefMut:
impl<T: ?Sized> Deref for Box<T> {
type Target = T;
fn deref(&self) -> &T {
&**self
}
}
impl<T: ?Sized> DerefMut for Box<T> {
fn deref_mut(&mut self) -> &mut T {
&mut **self
}
}
It simply dereferences itself, which would be impossible in any other circumstance.
For a wrapper type, I want to implement Deref to make things like method call syntax work transparently on the inner type, no? I don't want to call borrow first. You "deref the wrapper away".
Write an inherent method on MyNewtype<T>. This can always be done.
pub fn get(&self) -> &T
Implement AsRef<T> for MyNewtype<T>. This can always be done.
Implement Borrow<T> for MyNewtype<T>. This has implications for Eq, Hash and Ord implementations (if present), so it cannot always be done.
Implement Deref<Target=T> for MyNewtype<T>. This means that MyNewtype<T> is essentially a T, but with special ownership rules defined by MyNewtype. You can mostly use it as a T, but you have to allow MyNewtype to do its thing (so you can't take a &mut unless it also implements DerefMut, and you can't move the T out unless MyNewtype has a means to do that).
The list is roughly ordered simple to complex. You should use the simplest (first) thing that works. If a bespoke get method works for you because MyNewtype is always used concretely, you don't need to go any farther. If you need to write code that's generic over different types that contain or reference a T, go for Borrow and/or AsRef. The only real reason to go for Deref over these is for the convenience of not having to occasionally write .get() or .as_ref(). Is that important? Does it give you any additional flexibility, in terms of the API? Or is it just to spare a few keystrokes?
Semantically, if you use any of the first three options, you're telling the user: "Wrapper<T> has a T, but it is or could be more than that." Whereas if you use Deref, you're telling the user "Wrapper<T>is a T (with some form of ownership shenanigans)."
Cow<T> is an example of something that implements Deref and also (maybe) contains a T inline. The reason is because Cow expresses a kind of ownership: either an owned value or a reference. Box<T> also expresses a kind of ownership: unique ownership (of a heap allocated value). Arc<T> expresses shared ownership, MutexGuard could be said to express temporary ownership of a shared value, etc.
Does MyNewtype<T> express a kind of ownership of T? Or does it just contain a T? And if it's the second thing, why do you need Deref?
I'm not dismissing out of hand the ergonomic aspect of using Deref. It can be used to make it easier to use a thing as its underlying type. But I think we'd have to talk about a concrete use case and why it's useful to not have to write as_ref() to get at the inner value. If the reason is basically "it makes my code shorter," well, I'm not convinced. (But I'm also not completely opposed to this use of Deref; I just think it's usually unnecessary.)
(Note also that AsRef is generally better if the API is made to be flexible over different types: if you have a fn foo<T: AsRef<Bar>>(_: T), you can use that with lots of things that contain a Bar, but if you have a fn foo(_: &Bar) you can only use it with things that specifically implement Deref<Target = Bar>.)
Ok, so would this be acceptable? I’m no longer storing the T myself, but instead keeping a reference to it (specifically, a Box). It certainly feels like a change that users shouldn’t care about.
pub struct MyNewtype<T>(Box<T>);
impl<T> std::ops::Deref for MyNewtype<T> {
type Target=T;
fn deref(&self)->&T { &*(self.0) }
}
impl<T> MyNewtype<T> {
pub fn new(val: T)->Result<Self, ()> {
// Verify some property of val here
Ok(MyNewtype(Box::new(val)))
}
pub fn into_inner(self)->T { *self.0 }
}
To be clear, I'm not saying that implementing Deref on a newtype is bad, necessarily. I'm just saying that because you have a newtype is not sufficient justification to implement Deref for it. std::cmp::Reverse would be a good example of a newtype that doesn't have or really need Deref, since you can always access the wrapped value directly.
I disagree with this distinction. Your type shouldn't care where the data being guarded/proxied is stored, and implementing Deref doesn't necessarily imply your type needs to add a level of indirection to reach the Target.
I believe this is the main intent behind Deref. You add a level of logical indirection because accessing the Target requires going through your type and satisfying the necessary ownership rules and invariants, but it doesn't require the physical indirection that comes from storing a reference or Box.
That seems fair, and I may start another thread about my specific case at some point. Right now, I’m mostly interested to see how various people in the community think about this— I suspect that the particular wording in the docs may be misleading for some people, but I can’t propose new language until I understand how the community thinks about Deref.
I'd say impl Deref on newtype pattern is bad. The whole purpose of the newtype pattern is to make the new type not interchangeable to the original type so accidentally mixing them will results type error. But if you impl Deref on it the deref coercion implicitly convert them in many cases.
I had the same thought initially. But I actually am not sure the more I think about it. Let’s say you have a new type NoSpace(String) that verifies that a string doesn’t have a space in it on creation. If you have a function that requires the no space invariant, it can specify the NoSpace type in its signature. But that doesn’t necessarily mean it’s bad to use it where &str is allowed since it’s still a valid string. So if the only reason for the new type is adding requirements on the underlying type, I don’t see the harm in letting it behave like &str in this case.
EDIT Obviously DerefMut should not be implemented in this case
Does that mean, it's always OK to implement Deref as long as the target is either not directly accessible (stored on the heap) or and the target wrapped value cannot be made publically accessible?
DerefMut would have the additional restriction, that it must not be implemented, if mutation of the target can lead to UB, e.g. CheckedF64(f64) implements Eq¹, because it guarantees, that the inner f64 value is always a number and implementing DerefMut would cause UB, if the user set the value to NaN.
This sounds a lot like Liskov substitutability. It’s probably too technical for general documentation, but is it reasonable to say that Deref<Target=T> should only be implemented for types that are substitutable for &T? And similarly for DerefMut/&mut T?
Like most patterns there are no hard and fast rules on when you should do something and when you shouldn't.
Evaluate it on a case-by-case basis. If you believe it makes sense and helps express what you are trying to do then implement Deref. If it would cause confusion or others feel it's not appropriate to implement Deref in that scenario, don't.
The human element is a large part of writing code. AsRef<T> and Deref<Target=T> generate identical machine code, but the intent you are trying to express and how it is interpreted by other developers will be different.
As I put in the other thread, the core thing is that -- unlike every other trait -- implementing Deref changes name resolution for method calls. This is a huge deal.
So the core property that smart pointers have is that when you're using a Box<T> you almost always are actually using the T inside it, not the box-ness of it. As such, Box is careful not to add methods that could conflict with yours. When it wants to offer more functionality, it does so with associated functions like Box in std::boxed - Rust that aren't picked up in method name resolution.
And places where this doesn't happen -- trait methods being the important one -- it can be a trap. For example, because of ambiguity between foo.clone() and foo.deref().clone() the docs for Arc used to say
The Arc::clone(&from) syntax is the most idiomatic because it conveys more explicitly the meaning of the code. In the example above, this syntax makes it easier to see that this code is creating a new reference rather than copying the whole content of foo.
~ std::sync::Arc - Rust
I admit, I don't feel nearly as bad about NoSpace(String) as I do about MyNewtype<T>(T), for two reasons:
NoSpace probably implements Deref<Target = str>, not Deref<Target = String>, so rather than inventing a new smart pointer, you're just delegating to the inner impl. I find it hard to argue with the idea that a newtyped smart pointer is still a smart pointer.
The Target type is not generic, so methods on the wrapper are unlikely to interfere with method resolution for the underlying type (unless you specifically give them identical names).