Why is the discriminant of a `#[repr(u8)] enum` not `u8`?

I was playing with transmute and wanted to manually serialize (as an exercise) an enum, witch is using #[repr(u8)] for its discriminant. The layout of the enum is effectively a single u8 following by a number of bytes corresponding to size the biggest variant of the enum, witch is exactly what I expected. However when I try to extract it with std::mem::discriminant, I got a 64bits interger, as you can see in the sources. Is there any reason why it isn't a u8?

  1. Were you compiling in release mode, or in debug mode? I suspect the latter.
  2. Is the first field in one of the enum variants a field that requires 64-bit or greater alignment? If so, particularly in debug mode, the discriminant field will be padded out to provide that alignment.

I haven't seen an explicit reason for this stated anywhere, but it was probably just simpler that way. u64 fits all the possible values from the largest supported enums, and there's no benefit to making it any smaller in only some cases. The Discriminant type is intentionally an opaque wrapper for other reasons, including forbidding comparisons of Discriminant values from different enums, so it doesn't really matter if its underlying type "matches" the enum it's from.

The layout of the enum is effectively a single u8 following by a number of bytes corresponding to size the biggest variant of the enum, witch is exactly what I expected.

Maybe you already knew this, but it's probably worth stating explicitly that this "tag and value" layout is not true in general. Most enum layout optimizations (often called "niche optimizations" since most of them exploit niches) will violate this property, including the classic "null pointer optimization". #[repr(X)] does opt-out of this, but even the tag in a tag-and-value layout will often only be a few bits, smaller than any integer type Rust has. So it's difficult to argue that there's any deep reason why any enum's discriminant value should have any particular type after being separated from the enum.

4 Likes

There is also the runtime efficiency issue. If you're on a modern 64-bit platform the most-efficient load is probably a single 64-bit-word load. There's no point in space-optimizing, particularly if this was a debug build.

1 Like

Don't really want to hijack the thread, but since it is related to almost all the things discussed here, is it not possible to #[repr(NonZeroU8)] on an enum to give it well defined niche of zero.

Having tried it doesn't seem to know of the NonZero* types as being numeric representations.

It's not possible. Only these specific #[repr(...)]s are in the language. It's not a generic attribute that can take any type as an argument.

Thanks, now I can stop wondering! I just wasn't sure if there as any escape hatch/trickery possible due to NonZero* being in std::num.

So I guess the same rules apply to the std library.

Well, "the same rules" is a bit fuzzy, since the standard library is very much allowed to rely on perma-unstable compiler features, like #[rustc_layout_scalar_valid_range_start(1)] and #[rustc_nonnull_optimization_guaranteed], which are part of how the NonZero* types are implemented (you can see them in the std::num source).

But yeah there's nothing like that in the stable language, and I haven't seen any IRLO threads or RFCs about adding anything like that.

2 Likes

I doesn't change anything (I tested it, and you can see it in the source I linked)

Is the first field in one of the enum variants a field that requires 64-bit or greater alignment? If so, particularly in debug mode, the discriminant field will be padded out to provide that alignment.

My enum is #[repr(u8)], so it would be a hard compile error much before calling std::mem::discriminant.

The layout of the enum is effectively a single u8 following by a number of bytes corresponding to size of the biggest variant of the enum, witch is exactly what I expected.

Maybe you already knew this, but it's probably worth stating explicitly that this "tag and value" layout is not true in general.·

What you say is right for #[repr(Rust)], but not for #[repr(u*)]. The later have a guaranted layout to interface with other languages, like C or C++. The size of the discriminant will always be u* (in my case u8).

I haven't seen an explicit reason for this stated anywhere, but it was probably just simpler that way.

It is also what I thought, but I wanted to be sure.


To give a bit more context, I was trying to pack a Vec<SomeEnum> in a Vec<u8>, where SomeEnum is #[repr(u8)]. The idea is to store the discriminant (one byte), then exactly as few bytes as needed to store the current variant for each elements of Vec<SomeEnum> into Vec<u8> (using mostly transmute). Then to read them back, you "just" have to read the Vec<u8>, read one byte to get the discriminant, and then (based on the discriminant) read the next bytes and, re-interpret them as the original value.

Implementation in C, in Rust using Union, and in Rust using Enum.

The code above are just POC, I didn't cleaned it at all, and it isn't idiomatic either (nor for C nor for Rust).

Even if your union implementation is sound (it looks sound by just skimming the code), the enum one definitely isn't (try running MIRI on the top-right tools of the Playground): you are forgetting something quite important w.r.t to an enum: the padding (bytes) between the discriminant and the payload:

#[repr(u8)]
#[derive(Debug)]
pub enum Enum {
    Pair (u8, u8),
    F(f32),
    Array([u8; 6]),
}

This Enum has an alignment of 4 = max(align_of::<Disc = u8>(), align_of::<(u8, u8)>(), align_of::<f32>(), ...), so the Enum and thus the u8 discriminant both sit at an address A that is a multiple of 4.

In the case of the F variant, the payload itself must be aligned to 4 too, so even if it could just be located at A + sizeof(disc) = A + 1, that wouldn't be a multiple of 4, so there is some padding in there so that the payload is located at a (next) multiple of 4: A + 4:

|  A   | A+1  | A+2  | A+3  | A+4  | A+5  | A+6  | A+7  |
| disc | P  A  D  D  I  N  G| P   A   Y   L   O   A   D |

Now, whether that padding is variant-specific or common to all variants (e.g., does the 6-byte-long-1-aligned payload of the Array variant start at A + 4 or at A + 1?) depends on whether the enum is #[repr(C, u8)] or just #[repr(u8)] (source):

  • if #[repr(u8)], the padding is variant-specific,

    • e.g., the (1-aligned) payload of the Array variant would start at A + 1,
  • whereas for a #[repr(C, u8)] enum, the padding would be common, so that all the payloads start at the same offset (determined by the alignment of the payload = maximum alignment of all payloads)

    • e.g., the (less-than-4-aligned) payload of the Array variant would start at A + 4, and the enum would be 2 bytes bigger than its #[repr(u8)] version.

    • This second case has the advantage of mapping perfectly to your enum implementation

2 Likes

That's right, I totally forgot about padding between the discriminant and the data. I also didn't know you could have #[repr(C, u8)].

Do you know if it's possible to access to the size of the padding? ie. I would like to be able to do more or less let &data = transmute((&my_enum).as_ptr() + padding) (with the type of data being the type of the currently active variant).

The size of the payload's padding, in the #[repr(Integer)] (no C) case, can be computed as follows:

  1. start at the first "free byte", i.e., let mut addr = <*const _>::cast::<u8>(&my_enum).add(mem::size_of::<Integer>());.

    • .add() is sound (w.r.t the always sound wrapping_add()) because at worst (when all the payloads are zero-sized and 1-aligned) we are at most 1 byte past the Enum allocation.
  2. if addr is not aligned to let align = align_of::<VariantPayload>();, i.e., if let trail = addr % align; trail != 0, then add the padding equal to align - trail (given that the alignments are powers of 2 = 0b10, you can do the padding with bitmasks (non-power-of-2 % is an "expensive" CPU computation), but I'd expect the compiler to be able to optimise these ops with that knowledge (which you can hint using:

    • if !align.is_power_of_two() {
          unsafe { ::core::hint::unreachable_unchecked() }
      }
      
      • using unsafe here may seem overkill, but since you are doing pointer arithmetic this is not less safe than doing bitmasks.

    ))


In the #[repr(C, Integer)] case, the computation is the same, except for align being defined as max(align_of::<FirstVariantPayload>(), ..., align_of::<LastVariantPayload>())

1 Like

Thanks for the details. I think it is enough to be able to progress in my POC.

To sum-up this thread if anyone read-it in the future.

  • There is no real reason except that it was easier to implement
  • You can find workaround explained in details by @Yandros
  • There is more interesting stuff to read about memory layout and the different version of #[repr(...)].

You'll note that you had to look at the source to know that. The return type is intentionally an opaque struct so that this can be improved in the future -- to allow it to become smaller for smaller things, but also to potentially allow things like [repr(u128)] that would need it to be bigger. (Or potentially even so that it could be "well, I know you said repr(u64), but the two variants are 0 and 1, so I'm just going to have mem::Discriminant<> store a bool for your type".)

2 Likes

It's because it didn't match the mental model I had! Anyway, I totally agree with the reasoning. This just means that I need to find a way (if I ever convert my POC to a real macro) to access to the attributes of the enum.

I discovered that this was discussed in the last triage meeting, and may change in the future to support 128 bits discriminants https://github.com/rust-lang/rust/issues/70509