Is there a good reason repr(Rust) doesn't guarantee it will "pass through" identical layouts?

1

As far as I'm aware, the following is guaranteed to be okay, due to layout/ABI guarantees for MaybeUninit:

let mu: &MaybeUninit<bool> = todo!();
let b: &bool = unsafe { std::mem::transmute(mu) };

However a strict reading of the reference says that the following very similar thing is not okay:

struct S1(MaybeUninit<bool>);
struct S2(bool);

let s1: &S1 = todo!();
let s2: &S2 = unsafe { std::mem::transmute(s1) };

Is there any good reason that the Rust repr isn't guaranteed to be unchanged by substituting one field type with another of equivalent layout? (Perhaps this also requires saying something about niches.)

In case the above example is over-simplified and works for some special reason, here is something closer to the actual case I care about:

struct S1<T> {
    field1: AtomicU8,
    field2: MaybeUninit<T>,
}

struct S2<T> {
    field1: NonZero<u8>,
    field2: T,
}

I can of course do the type punning I want if I use repr(C), but that forces me to opt out of the layout optimizations that can be made with repr(Rust). My understanding is that these optimizations are more or less the reason for reserving the right to change things under that repr, but it doesn't seem like those optimizations should care about this difference.

2

First of all, MaybeUninit disables niches, which can be a problem inside enums. Similarly NonZero<u8> has a niche that AtomicU8 does not.

Second, there aren't many cases where this is needed (and the existing cases can be solved with #[repr(C)]), and there's always the risk that by guaranting that we close the door for some future layout optimization (e.g. trying to pack multiple fields in one). But if you have a convincing use-case, you can try submitting an RFC or discussing that with the lang team.

3

Exactly this.

They do a lot. Check this out:

fn main() {
    use std::mem::{size_of, MaybeUninit};

    dbg!(size_of::<bool>());    
    dbg!(size_of::<Option<bool>>());
    dbg!(size_of::<MaybeUninit<bool>>());
    dbg!(size_of::<Option<MaybeUninit<bool>>>());
}
  • A bool is one byte, where two values 0 and 1 are used (2 states out of 256, 254 unused and illegal states).
  • An Option<bool> needs a third state for the None, maybe it will use value 2 for None
  • A MaybeUninit<bool> is one byte where are 256 states possible, but there are 254 states which are not a valid bool.
  • An Option<MaybeUninit<bool>> needs some additional state. But all 256 states of the underlying byte from the MaybeUninit<bool> are already used. The Option can not know, if the inner MaybeUninit contains a valid bool which value is either 0 or 1, or some other, invalid and uninitialized byte. So a second byte is needed. The layout cannot be the same as Option<bool>.
4

As I tried to hint in the original post, I’m familiar with the concept of niches. But I’m pretty sure you could state a sufficient set of conditions to make the issue of changes to niches irrelevant for the two fields in question. Is there any reason, at least in principle, that this couldn’t be guaranteed safe under those conditions? Or is it somehow not possible to state those conditions?

(I’m not proposing changing the language, just trying to understand the design space.)

5

The reason is obvious: any change that makes language harder to understand is rejected by default. IOW: you don't need reason to reject such set of rules (any set is more complicated that zero set), you need a reason to allow it.

True. And then millions of developers would have to study these rules and hundreds of compiler developers would need to think about what to do when (not if!) compiler development would be impeded by these… that ay be an acceptable price if that would actually buys us something… what exactly would that enabled that repr(transparent) and rept(C) don't enable?

You absolutely do propose to change the language and any change to the language needs a justification.

The design case is simple: any change to the language needs a use-case and justification.

6

One obvious example: rnadomize-layout exists and it can be used as security hardening feature (AFAIK Linux kernel currently doesn't use it, but given the fact tat they even brought it into C it's obvious that sooner or later they would want it int Rust, too). You want to break it… why? What's your justification?

7

@jacobsa made no reference whatsoever to proposing a change. They are asking a question about the motivation for existing design. Questions from newcomers should be met with facts, thoughtful reading, and kindness, not aggressive opposition.

8

The problem is that you're restricting unknown future things by doing so.

If we'd done that 10 years ago, the rule would have been "well if the fields have the same size+align values, then it'll match" because that's the only thing that repr(Rust) looked at at the time.

But it would have been a mistake to do that, since we've since learned that it's useful to also do niche-aware field rearrangement.

Specifically for your example, see https://play.rust-lang.org/?version=nightly&mode=debug&edition=2024&gist=f431815c41b2cd6cfb756c2b820b06f3

[src/main.rs:14:5] offset_of!(S1<u8>, field2) = 1
[src/main.rs:15:5] offset_of!(S2<u8>, field2) = 1
[src/main.rs:17:5] offset_of!(S1<ascii::Char>, field2) = 1
[src/main.rs:18:5] offset_of!(S2<ascii::Char>, field2) = 0

Even though Layout::new::<u8>() == Layout::new::<ascii::Char>(), it's already the case today that S2<u8> and S2<ascii::Char> have different field orders.

The other reason -- see also why repr(transparent) exists -- is that it's good that if you're relying on something that you say so in the code. That makes it clearer to the code reader that something is going on, and helps the compiler give you errors or at least lints if you try to so something weird.

So it's possible that there could one day be a repr(size_align_and_declaration_order_only) that would do what you wanted, but it would be a specific choice the programmer would make.

9

the status quo is simple : giving no guarantees about layout gives maximal freedom to the compiler to modify how they are defined at any point in the future, which is very desirable.
for a basic example the layout of Option is currently in the process of being modified. Prefer `-1` for `None` by scottmcm · Pull Request #155473 · rust-lang/rust · GitHub

the rust compiler has no clue what new features may be added in the future, what new quirky platforms it may support, and what new optimizatins may be thought of;
giving minimal guarantees leaves it maximally prepared to face any of these changes.

that being said, there have been and continue to be efforts to specify things like layout compatibility and exact layout. these are efforts are slow because they are not necessarily priorities for the rust compiles, and because mistakes are not acceptable ;
if a mistake is stabilized, it will stay there forever.

for every guarantee, the gain of more valid code must be weighted against the cost of less valid optimizations.

here is an example of such a proposal : https://github.com/rust-lang/rfcs/pull/3775#discussion_r1966660879

10

Thanks all (except @khimru), it sounds like it's just a matter of future flexibility. That's perfectly appropriate, I had just wondered if there was some other optimization that exists today or is likely on the horizon besides niches. It sounds like mostly no, but again I understand the motivation and that's fine.

11

(No idea how feasible, but) I can imagine some PGO where different types of similar contents get different layouts due to different usage patterns.

12

scottmcm's example shows that, currently, we rearrange fields to prefer having a field with a niche to be at the front of the struct. This is so that, if we later put this struct into an enum, then we have an easier time optimizing that enum. Relevant code: layout.rs - source