Object-safe traits consuming self?

Hello,

While experimenting in order to better understand dynamic dispatch in Rust, I came up with the following working toy example:

trait Trait {
    fn method(&self);
}

impl<T> Trait for T
// where
//     T: std::fmt::Display,
{
    fn method(&self) {
        dbg!(std::any::type_name::<Self>());
    }
}

fn main() {
    let obj: &dyn Trait = &123;
    obj.method();    // Dynamic dispatch.
}

I became extremely confused when I realized that the above example continues to compile even if one replaces both occurrences of &self by self.

I kind of understand why this is the case: When the argument to method is self, the method that gets called in main is the one for Self = &dyn Trait, so self is Sized and can be moved. In contrast, with the above program unmodified, method is called for Self = i32.

I also understand that my example is artificial. If any real work is to be done in the method, T must be bound to some traits (let's say Display if we want to print the value), and these traits won't be implemented for a reference to a trait object, so compilation will fail.

Still, I'm a bit confused. Object-safe traits must not require Self: Sized, but having a method that takes self does effectively require that Self is sized. Shouldn't the above example with &self replaced by self not lead to a compilation error? Or are there any valid use cases of object-safe traits taking consuming self?

1 Like

This is a specific rule in the definition of object safety:

  • All associated functions must either be dispatchable from a trait object or be explicitly non-dispatchable:
    • ...
    • Explicitly non-dispatchable functions require:
      • Have a where Self: Sized bound (receiver type of Self (i.e. self) implies this).

That is, a fn method(self) is explicitly non-dispatchable, which means that it doesn't disqualify the trait from being used in a trait object, but you can't call it.

In your case, it happens that a version for the reference is found instead, because that's a valid receiver autoref where the method is callable. But if the trait weren't blanket implemented, you'd instead get an error:

trait Trait {
    fn method(self);
}

impl Trait for i32 {
    fn method(self) {
        println!("hi");
    }
}

fn main() {
    let obj: &dyn Trait = &123;
    obj.method();
}
error[E0161]: cannot move a value of type `dyn Trait`
  --> src/main.rs:13:5
   |
13 |     obj.method();
   |     ^^^ the size of `dyn Trait` cannot be statically determined

Also that if you do want to be able to call by-value methods on a trait object, you can do that with the help of Box:

trait Trait {
    fn method(self: Box<Self>);
}

impl Trait for i32 {
    fn method(self: Box<Self>) {
        println!("hi");
    }
}

fn main() {
    let obj: Box<dyn Trait> = Box::new(123);
    obj.method();
}
7 Likes

&dyn Trait can implement Trait in other cases too, either due to an explicit definition (but that's more common for Box<dyn Trait> or other owned variants), or due to blanket implementations for example.

fn witness<D: Display>() {}

fn main() {
    witness::<&dyn Display>();
}
2 Likes

Many thanks for the explaination! Now everything makes perfect sense.

I must have overlooked that specific rule. It would have helped if one of the examples demonstrated it - as far as I can see this is not the case. (Is it useful to open documentation issues/PRs about such minor points?)

It's easy to infer it logically from the fact that trait objects are unsized, therefore they can't be passed by value, so when Self = dyn Trait, then a method fn foo(self, ...) -> _ can't possibly work.

That's why I was so surprised that replacing &self by self seemed to work!

...so I personally was sure that these methods would make the trait not object-safe, just as methods with Self arguments do. TIL that that's not correct.

To be clear, there are (at least) two different kinds of errors you can get when dealing with trait objects:

  1. When creating a trait object from a static type implementing the trait, the coercion from T: Trait to dyn Trait itself will fail if the trait has methods that take self by value. No method has been called at this point.
  2. When calling a trait method on a type (any type) implementing the trait, the compiler can emit an error if the signature of the method doesn't permit the particular call. This is when you'll see things like "cannot move out of reference" or "cannot borrow _ as mutable, as it is behind a & reference". This hasn't got much to do with trait objects, it's just that trait objects implement their own trait, so you can still run into this kind of error when you call a method on a trait object.

The "method taking Self by value works on a trait object" phenomenon is observed e.g. when the trait is blanket-implemented for &T where T: Trait. In this case, dyn Trait implements Trait, so &dyn Trait also implements Trait, resulting in successful method resolution given Self = &dyn Trait. In this case, taking ownership of self is possible because it itself is a reference and therefore Sized.

3 Likes

Perhaps it's just me, but coming from C++ with a lot of experience in both template metaprogramming and class inheritance, Rust's traits do require quite a lot of mental gymnastics before they become familiar. (I consider this a good sign, since this means that truly new concepts are involved. But sometimes I wonder whether all that complexity is really necessary.)

For example, when I learned that traits may must not require Self: Sized my first (and second) reaction was: "But surely I can require Sized for a trait and still build vtables for the types that implement that trait".

Or (another example): why isn't it possible to build trait objects from traits that require Default?

I might be able to answer these questions now, but I still have to think twice about them.

I can assure you that Rust's trait system is a lot easier to use correctly than C++ TMP.

You might be confusing "easy to use" with "fewer compiler errors". The good thing about Rust's generics and traits is that they are type checked upfront. Once your code compiles, you can be sure that it will continue to compile for anyone using it in the way you intended. This is not true for C++ templates, which have to be instantiated in every possible foreseeable way to check whether they still compile for a potential use case/downstream user.

100%. Rust doesn't include complexity just for the sake of complexity. We are not trying to look smart; we are trying to guarantee correctness.

I'm not sure what specific constraints you propose could be relaxed around trait objects, but if you think just a little bit about the mechanisms for implementing them, it should become pretty clear that there's not much one can hope to simplify.

1 Like

I wonder if trying to compare to C++ templates may be misleading. This discussion has been about Rust trait objects of course, not Rust traits used as generic bounds. And these are very different things.

When using Rust traits as a generic bound (not a trait object), comparisons to C++ templates are useful since there are a lot of similarities; for one, the generic code is expanded based on the types used by the caller. Of course they are different in some ways, the big one being that Rust type checking is performed without instantiating the generic types, like it must be with C++ templates. And this constrains Rust generic code to only use the methods that are specified via the generic bounds.

But with trait objects, it may be more useful to compare their methods to virtual methods in C++, where the common factor is dynamic dispatch. You wouldn't expect C++ code that accesses a virtual base class to depend on the size of the concrete class, right? Would it help to think of trait objects as virtual base classes?

I agree that it can be confusing in Rust that these two uses of traits are so different. For me it helps to think of them as very distinct things.

The only exception is, amusing, the trait system. You said so youself:

Note how you haven't said that you have to actually run your TMP program, only just instantiate it.

Yes, there are corner cases where TMP program would be miscompiled and wouldn't work, but Rust have similar corner cases, too (e.g. if you add method for a type it would be called instead of similarly named method for a trait) so that's a wash.

With C++:

  1. You can easily and non-contrivingly write your TMP program.
  2. It may fail, badly, at the instantiation time.
  3. It would almost never fail at runtime.

With Rust:

  1. You have to do extremely complicated mental gymnastic to make your traits compile.
  2. But it would rarely, almost never, fail at the instantiation time.
  3. It would almost never fail at runtime.

Both methods are nice to have just in different cases: Rust approach is better when you are writing libraries, C++ approach is better if you are writing code that wouldn't part of the API.

Yet C++ address the shortcomings with concepts while Rust doesn't even admit there are problems with it's approach.

Just add one keyword (or annotation) like #[no_verify] and do all the typechecking analysis after instantiaon, not before, and you would make life of many users infinitely easier.

That's already done like that with constants, why couldn't that be done with functions?

They are intervened.

Well… the topicstarter's example shows that they are not so distinct. And the question of why object-safe traits may include functions that may not be called, but couldn't include various other functions is very interesting one.

I understand that it's possible to make language like that… but why? If we allow functions that may not ever called dynamically in trait then why do we even need that “object safety” notion? Why not make it possible to turn any other trait, even non-object-safe one into dyn Trait with the restriction that all functions which couldn't be called via vtable would only be accesible via impl Trait and not via dyn Trait?

Where that design decision comes from? Someone's mistake which may not be rectified because of backward compatibility or is it something deeper?

Are you still confused? You seemed to figure it out on your own that &dyn Trait was being passed, so I didn't comment further on this aspect before.

Possible sources of confusion:

  • References in Rust are distinct, full-fledged types and always Sized. Just because dyn Trait doesn't meet a bound doesn't mean &dyn Trait doesn't meet a bound... and vice-versa.
  • You provided an implementation of Trait for &dyn Trait
    • (it does not get a compiler-provided implementation like dyn Trait does)
  • Method resolution details

A tangent:

This issue can be avoided in practice, by not adding Default as a supertrait. Going beyond that, whenever you have a trait that is going to be used as a trait object, the trait usually should not have any constructor functions, whether via Default supertrait or not. This is because obligating every implementor to be constructible using no inputs (or some specific set of inputs) is often too constraining on the actual implementations — an implementing type might want to be constructed using information specific to that type.

If you do specifically have a use for Default construction, just keep the Default bound separate and explicit:

pub trait Foo {}

pub fn default_trait_object<'a, T: Foo + Default + 'a>() -> Box<dyn Foo + 'a> {
    Box::new(T::default())
}
4 Likes

It's different question. The question is not “how to sidestep rules for object-safe traits”, but “why the heck these rules even exist and what's the point of having them”?

I always [naïvely] assumed that object-safe traits are traits where all functions can be called via vtable. And I suspect topicstarter assumed that, too.

This made sense. Now you have pointed out that object-safe traits may also include bunch of functions with seemingly arbitrary restrictions which couldn't be called via vtable.

That makes no sense to me: if we are supporting some random functions which are only usable when you use impl Trait and couldn't be used with dyn Trait then why do we even have object-safe trait notion? Why don't allow all traits to be usable as dyn Trait by removing the ability to use “problematic” parts via dyn Trait?

No, I think I get it now. I was just trying to explain to @H2CO3 that I didn't find the issue quite as trivial. But your additional bits of wisdom are most welcome.

Good point!

Thanks for your explanations. I happen to know old C++ very well (C++-11 and newer less so, but this does not matter here). I'm aware that Rust traits are similar both to C++ templates (when traits are used with generics), and to C++ virtual base classes (when trait objects are used).

What has been unfamiliar to me (but it's getting better, thanks), is when both roles are mixed. Like in the above example where obj.method(); corresponds to dynamic dispatch for a trait object, but after substituting self for &self it becomes static dispatch of a generic function with the reference to the trait object being the generic parameter...

I find it fascinating that both of these things got "unified" in Rust (and there's even more: traits are also like C++ concepts).

addition:
Perhaps it doesn't help that the dot operator (method call) can mean so many things in Rust. Not sure whether this is "ergonomic".

Me too! In spite of the complexity and restrictions.

I can't find it now, but someone said that at first it seemed fantastic to be able to use traits as generics, and then reuse the same traits for dynamic dispatch using trait objects. Then they discovered the object safety rules and they were very disappointed because of the complexity. Finally they realized that it's best to use trait objects in only limited cases where dynamic dispatch is really needed, and use regular generics in all other cases, and with that they were pretty satisfied.

I've found that perspective to be very practical.

The point of needing to mark non-object-safe methods is explicitness and consistency, two cornerstones of the language. If what you proposed were true, then trait objects couldn't just implement their own trait; they'd have to be treated specially. When a trait method is called, the compiler (and human readers of the code) would have to check separately whether the receiver is a trait object, and if so, prohibit calling the non-object-safe method. This would not be apparent from the source, whereas the where Self: Sized requirement is visible to the consumer of the code.

3 Likes

Why not invent some kind of special mark, then? Because right now this:

pub trait Default: Sized {
    // Required method
    fn default() -> Self;
}

is not object safe while this:

pub trait DefaultDynOk {
    // Required method
    fn default() -> Self where Self: Sized;
}

is, suddenly, object safe. For a layman they look equivalent.

Heck, RFC that introduced that change had not such exception.

Also this, of course, immediately raises the next question: just why Default is not object-safe? It could have been made object-safe easily, after all. Was it conscious decision to make it non-object-safe or have that just happened by accident?

P.S. At least now I understand why it's so hard to fix that annoying bug #20671: if you try to fix it naïvely… you would make Default object-safe, which is, apparently, undesirable for same unknown reason. Or maybe it wasn't made object safe by accident?