Why is `Sized` trait not allowed in dyn MyTrait?

1

I have read about dyn compatibility, "object safety", "trait objects" in the Rust reference, Rust book, etc. However, I'm confused about why we can't use a trait, bound with Sized constrain like in the following example.

trait MyTrait : Sized {
} 
struct MyStruct;
impl MyTrait for MyStruct{}

fn my_method(obj: &dyn MyTrait){
}

fn main() {
    my_method(&MyStruct);
}

This code does not compile because only sized types allow to implement MyTrait. This part is clear to me.

Next, let me break &dyn MyTrait into pieces to be sure I interpret it correctly. As far as I know &dyn MyTrait means "a reference to a value that implements the trait MyTrait" which is widely know as a trait object. I also understand that a trait object is roughly represented like follows:

pub struct TraitObject {
    pub data: *mut (),
    pub vtable: *mut (),
}

A trait object is a mechanism to implement dynamic dispatch (or polymorphism). So, when we write a method like

fn my_method(obj: &dyn MyTrait){
}

we basically say "Hey Rust! we are going to pass a reference to some value that implements MyTrait but whose concrete type is unknown at compile time". The type might be u16, String, &str, or even custom MyCoolType.

According to the trait object representation the size of the value is not relevant when we are tossing a trait object around.

So, this is what I don't understand: how does "being Sized" prevent implementation of dynamic dispatch?

Yes, we cannot implement MyTrait for str, but why should it prevent passing Sized types as trait objects? Maybe I want to pass only Sized types? After all, Rust does not care about sizes which may be know or unknown at compile time.
Why is NOT possible to create vtables for only sized types? After all, if we might pass any value of any size, this also could be classified as "unknown size" at compile time: 2 bytes, 10 bytes, maybe 1Kb? The type will be known at run time.

could anyone clarify it?

2

(n.b. There were some edits to the OP while I wrote this up which hopefully don't invalidate it :slight_smile:)

You lose me at this point -- it's the other way around. Not being Sized is the barrier to being coerced to a dyn Trait (and thus performing your typical Rust dynamic dispatch).[1]

It doesn't. You added a requirement for Sized, not prevention of being Sized.

Even without the Sized bound -- i.e. when you can implement MyTrait for str -- you cannot coerce a &str to a &dyn MyTrait, because str is not Sized.

The rest of this comment is about why you can't coerce values of non-Sized types to dyn Trait. (That's my best guess as to what you find confusing, but I could be wrong.)

The types of non-Sized types possible in Rust so far are DSTs. Their sizes are known at runtime, but not compile time.[2] Those include slice types like str and [T], and also dyn Trait types.

For dyn Trait, the size of the erased base type is stored in the vtable. How do you find the vtable? A pointer to the vtable is stored in a wide pointer. So this:

pub struct TraitObject {
    pub data: *mut (),
    pub vtable: *mut (),
}

is an approximation of a &mut dyn Trait or Box<dyn Trait>, etc (and not an approximation of a dyn Trait itself). You're pretty much always working with a wide pointer to a trait object.

Slice types also have wide pointers, but instead of a vtable, they carry the length (which can be used to recover the size). So this is an approximation of a Box<str>:

pub struct Str {
    pub data: *mut (),
    pub length: usize,
}

Note that these types (e.g. &str and &dyn Trait) are Sized: They have statically known sizes (the same size as two usizes).

In order to do dynamic dispatch, the compiler needs to be able to downcast from &dyn Trait to &ErasedBaseType. So if the base type was str, for example, it would have to be possible to recover the original length, as there's no other way to create the wide reference (&str).

And the problem is that there is no good way to do that.

You can't store the length in the vtable unless you have a vtable for every distinct length. Which is a runtime property. Possible by allocating vtables at runtime in some language perhaps, but dyn Trait doesn't require a heap[3] and Rust doesn't like implicit costs like "my unsizing coercion allocates" either.

You would need a "super wide pointer" to hold both the length and the vtable in the reference itself. But we can't have &dyn Trait of two different sizes -- it's Sized -- so we'd need a new type of dyn Trait, or... try to make every &dyn Trait take a size hit perhaps. But even then there are challenges around slices of zero-sized types.

If you try to generalize this approach so that any DST can be coerced to a dyn Trait, you end up with arbitrarily wide pointers: each coercion increases the size of the pointer.

I've written this same explanation with different words over here.

Finally: Truth be told, the ": Sized means not-dyn-compatible"[4] mechanism is a hack, as Sized is being used as an approximation of "not a dyn Trait". I rant about that over here.

  1. The exception is coercing between different forms of dyn Trait, like from dyn Trait + Send to just dyn Trait. ↩︎

  2. We want a third category of types with completely unknown sizes for the sake of extern and the like, but we don't have those yet. ↩︎

  3. e.g. no_std embedded programs use it ↩︎

  4. or when on a method, not dyn-dispatchable ↩︎

7 Likes
3

Thank you for a long answer, but I don't understand your answer.
I am still confused about why adding Sized results in compile error. If dynamic dispatch is ok with

trait MyTrait {
}

why doesn't it compile the following code?

trait MyTrait : Sized {
}

The second trait is kind of "subtrait" of the first.

4

Fundamentally, the reason is that dyn Trait is its own type which is not Sized, just like str or [T].

There’s not really any technical reason why the coercion has to fail, but then you’d run into the problem of dyn Trait not implementing Trait because it doesn’t satisfy the prerequisites, which means there’s nothing useful that you can do with it once created.

4 Likes
5

As far as i understand a Trait defines behavour not size, so the trait itself can't be sized but if you want any implementor of the trait to be sized you can use generic bounds.

Another point dyn Trait is unsized because it represent a type that implements Trait and we can't ensure that all of those types are sized at runtime.
In addition in rust if a type if not sized we cannot store it directly so we take a reference to it or box it :boxing_glove:
please correct me if i'm wrong this is how i understand it

6

No,I am not trying to implement anything, my goal is just to understand while it doesn't compile when add Sized

And who cares whether it is sized or unsized at runtime? What I care about is that the type implements a certain trait (has a particular method, and that's all). Why should I bother of type's size?

I don't see any connection to my question. I guess you misunderstand my question.

7

The last link shared by quinedot explains it:

This makes some sense, as dyn Trait is not Sized. So a dyn Trait cannot implement a trait with Sized as a supertrait, and a dyn Trait can't call methods (or associated functions) that require Sized either.

You should click on the links from his reply to get a deeper undestanding.

8

Yes, let me read "A tour of dyn Trait" part. I guess it'll help me understand this dyn thing :grinning:. Thanks @quinedot for sharing.

9

Ah, perhaps I understand now.

If a trait is not dyn-compatible, it's not possible for dyn MyTrait to implement MyTrait. In such cases, instead of allowing you to coerce to an approximately useless dyn MyTrait that doesn't implement MyTrait and can't dispatch to any methods, the language just says that dyn MyTrait is an invalid or non-existent type altogether.[1]

The OP errors because Sized is being used in a hacky way to mean "not dyn-compatible", so having a Sized bound on trait MyTrait makes dyn MyTrait be considered an invalid or non-existent type altogether.

Implicit in this design is the idea that "can't implement MyTrait for dyn MyTrait" and "dyn-compatible" are the same thing. Or to phrase it differently, the idea that dyn MyTrait exists if and only if dyn MyTrait implements MyTrait. That's not actually true though! However, exceptions would be much easier to hit if you could always coerce to dyn Trait.

As far as I know, the language could allow dyn MyTrait with the Sized bound even though it wouldn't be able to implement MyTrait, similar to the existing exceptions. Then your OP would compile (but probably not be useful). Or alternatively, it could let the dyn MyTrait type exist, but never let you coerce to it.

  1. So even this fails to compile. ↩︎

3 Likes
10

What I understand from Rust reference about the dyn Trait and how it is used inside Rust code is when I pass a dyn Trait from one method into another one, the compiler can't know the exact type (at compile time), but it ensures that the trait methods can still be called on that object. This is what I know about it, maybe not enough or wrong.
So two key points are:

  1. compiler does not know exact type (at compile time)
  2. trait methods can still be called on that object

So, as for the first point: compiler does not know exact type. If compiler does not know about the exact type, then why is the compiler bothered with Sizedness of the type? :exclamation:

As for the second point:

trait MyTrait {
  fn my_method(&self);
}

without digging too deep, I assumed as long as a type (struct Foobar) implements my_method,
there shouldn't be any problem with dynamic dispatch, right? Who cares about the size of Foobar if my only goal is to invoke my_method? :exclamation:

In other words, the dyn Trait's goal is to achieves polymorphism (duck typing) in Rust, so why the Sizedness is a problem is still uncler to me. I guess the dyn Trait is not about it only. :thinking:

11

An important general property is that if you have generic code:

fn foo<T: ?Sized + MyTrait>(value: &T) {...}

then T = dyn MyTrait is a valid instantiation of that generic parameter T. That means that dyn MyTrait must meet the trait bounds on T, which include MyTrait. But when you write trait MyTrait: Sized {}, you are saying that part of what it means for a type to implement MyTrait is that the value is of statically known size. Therefore, in this case, dyn MyTrait doesn't implement MyTrait, and doesn't exist.

This principle applies to every supertrait (or other bound) MyTrait has — the special thing about Sized is that dyn can't implement it, whereas other traits just become part of the vtable. This principle also explains why associated types and constants disqualify a trait from dyn compatibility: part of what it means to implement the trait is to have those types or constants, and dyn can't, so dyn doesn't implement the trait.

There isn't actually any strong reason that dyn MyTrait couldn’t exist without implementing MyTrait, but you wouldn't be able to call any MyTrait methods on it (without having additional method dispatch rules with a special case for dyn), since the dyn MyTrait wouldn't be implementing MyTrait.

4 Likes
12

I can think of multiple answers. When you have a : Sized bound on the trait:

  • At the language-design level, : Sized is a hack that means "dyn MyTrait isn't a valid type"
  • You told it to care more generally by adding a : Sized requirement
    • Even if the hack didn't exist, dyn MyTrait couldn't implement MyTrait, because dyn MyTrait doesn't implement Sized

And when you don't:

  • The size of the erased type still needs to be known [1]
    • dyn Trait are dynamically sized (size known at run time), not "unknown sized"
    • My first reply talks about the difficulties of retaining this information when erasing DSTs
    • And, as per the &str example, how you can't dispatch without this information either

If @kpreid is right that the main confusion is

aka

Even if the hack didn't exist, dyn MyTrait couldn't implement MyTrait, because dyn MyTrait doesn't implement Sized

then perhaps the confusion is: dyn MyTrait isn't a magic alias for any concrete type that implements MyTrait. dyn MyTrait is its own concrete type, distinct from any implementing types. In order for dyn MyTrait to implement MyTrait, it has to implement the supertraits. But it cannot implement Sized.[2]

  1. Example ↩︎

  2. It can't implement any other non-dyn-compatible supertraits either. ↩︎

2 Likes
13

@buy-rum I wonder if the confusion is because you're not thinking about dyn Trait (which cannot be statically sized), because you're thinking only about &dyn Trait (where the size of the type really doesn't matter). Is that possible?

In other words, while what you're saying might be roughly true if there were no such type as dyn Trait, there is such a type. Are you aware of the other similar types like str that cannot be statically sized?

Another way to help resolve your question is to turn it around and ask: Why do you need a dyn Trait that is Sized? By leaving off the Sized bound, the type that implements the trait can be statically sized or not statically sized, so why include the bound?

One common reason you need a Sized bound in a trait is when a trait method returns the type by value:

trait Trait {
   fn new() -> Self;
}

In that case it would be impossible to return a type that is not statically sized, since the return value is allocated on the stack and may be stored in array, a struct field, etc, by the caller. So that's an example of a method that cannot be part of a dyn Trait and also makes no sense to be callable via &dyn Trait. The concrete type and its size must be known by callers of the new method.

14

A-ha!
so, can we say that when we define

trait Trait {
  fn foobar(&self);
}

we automatically obtain a new type dyn Trait which is (as @quinedot and others I guess already mentioned) is unsized by definition (probably by the Rust compiler). Thus when I define

trait Trait : Sized {
  fn foobar(&self);
}

we get a contradiction that dyn Trait is unsized by Rust compiler and we try to get dyn Trait as sized?

15

Yes, exactly.

It's not exactly a contradiction. If you managed to bypass the compiler’s check for this case[1], you’d end up with a dyn Trait type that doesn’t implement Trait, and is thus nearly useless.

  1. which is possible to do with some twisty trait bounds, but that's essentially a compiler bug ↩︎

16

How do you deduce that dyn Trait does not implement Trait when I bound it with Sized?
I've just tried to build and got the following error message:

the trait `Trait` cannot be made into an object
only type `MyStruct` implements the trait, consider using it directly instead

Does the message "only type MyStruct implements the trait" mean dyn Trait does not implement Trait?

17

Sort of, but the compiler is trying to implement dyn Trait (dyn Trait is implemented only if it is used by the program) and this fails with the message:

the trait `Trait` cannot be made into an object
18

That note is just a help message heuristic to give a suggestion how to arrive with working code. It doesn’t further describe the problem. Instead it’s a message of the kind “hey, I looked at your whole program, you’re using this Trait trait for only a single type, and then you want to use the trait like a type (via dyn Trait) – so maybe you don’t need to involve any trait at all?”

I think the deduction is rather that it can’t implement Trait, as implementors of Trait must also fulfill the supertrait bound (Sized), which un-sized (aka dynamically-sized) types like dyn Trait don’t do.

Ultimately, this is all largely a language design question though.

There’s no strict reason why the interface of dyn Trait objects absolutely must be through the mechanism of dyn Trait being an implementor of Trait itself. E.g. the trait methods could instead also be inherent methods, too, where a : Sized supertrait bound would not be an issue.

But there’s also design decisions around stability. Currently, : Sized supertrait bounds is explicitly documented as a mechanism that will make Trait not dyn-compatible; and as such it’s a mechanism to ensure a trait can support e.g. future addition of other non-dyn-compatible items in the future without breakage (e.g. a new method with a default implementation that doesn’t have a self argument, or that has one of non-dispatchable type).

19

Implementing Sized means that the the implementor has a single size known at compile time. But implementors of MyTrait can coerce to dyn MyTrait (behind a pointer), no matter what their size is. dyn Trait not being Sized is a necessity in order to support coercing implementors of different sizes.

Consider:

    // A 4 byte value
    let i: i32 = 0;
    // A 24 byte value
    let v: Vec<i32> = vec![];

    let dt: &dyn MyTrait = if some_condition() {
        &i
    } else {
        &v
    };

At run time you will have one of

     +----------------+----------------+
     |                |                |
dt = | data pointer   | vtable ptr. ---+---> ... (static memory somewhere)
     |      |         |                |
     +------+---------+----------------+
            |
            V
     +--------+
i =  | i32    | as dyn MyTrait (4 bytes on the stack)
     +--------+

or

     +----------------+----------------+
     |                |                |
dt = | data pointer   | vtable ptr. ---+---> ... (static memory somewhere)
     |      |         |                |
     +------+---------+----------------+
            |
            V
     +----------------+----------------+----------------+
v =  |  data ptr      | length         | capacity       | as dyn MyTrait
     +----------------+----------------+----------------+ (24 bytes on the stack)

Both erased types have been coerced to dyn Trait, but they have different sizes. So dyn Trait does not have a staticly known size (a single size known at compile time). It has a dynamic size (a size only known at runtime).

If I have

trait Trait: FnOnce()

then I can deduce that String does not implement Trait, because I know that String does not implement FnOnce().

If I have

trait Trait: Sized

then I can deduce that dyn Trait does not implement Trait, because I know that dyn Trait does not implement Sized.

Again, the language goes even further than that and says "due to Sized being a shorthand for not-dyn-compatible, dyn Trait isn't even a valid type in this case", but I don't think there's any reason it has to. That is, dyn Trait could still be a valid type, but it still would not be able to implement Trait, because it cannot implement the supertrait (Sized).
2 Likes
20

Yeah, I understand your point.
Because coerced types might be of different sizes, it does not make sense to define Trait as Sized and require implementors be of a fixed size, say 4 bytes.
Well, Rust could allow somehow to make the Trait Sized but then dyn Trait for only 4 byte types would sound weird. Do I get you right? :grinning: