It's your choice whether to implement Deref. The main concern here is trying to justify to your users why the added ambiguity and implicitness that results is worth it. "Smart pointer" in this context means that the type's pointer semantics are secondary to the fact that it acts as a storage for some T. In other words, the T being pointed at is far more important to the user than the fact that it's wrapped by something else.
This usually doesn't hold for newtypes, because the newtype pattern is most frequently used to modify the observable behavior of the T inside, rather than just add additional qualifications on a pointer to it. (ByAddress appears to be doing exactly that -- it's a newtype intending to modify the behavior of the pointer, not the pointee.)
But this is just one way to look at it, and it is no less valid than other, conflicting interpretations. SmartPointer<T> isn't a provided abstraction in Rust, so basically the 'canonical' definition of smart pointer in Rust is just 'anything that implements Deref/DerefMut'. But that would be circular reasoning if you're trying to decide whether to implement those traits in accordance with some definition of smart pointer. You'd ultimately have to make that determination based on your project requirements.