Memory layout introspection

[Michael-F-Bryan]

@Fredrik

For my use case, the memory layout doesn't even need to be stable across compiler versions. It would be enough to have a way to query the memory layout of the current compiler version and feed this data into the program that generates the non-Rust code.

@Fredrik

• Parse the 'rlib' files of the standard library to figure out the memory layout of each type for the current compiler version.

I don't think you can/should rely on this. The layout of #[repr(Rust)] types (i.e. everything without a repr annotation) is implementation defined, can't be relied upon, and may change between compiler versions or runs. Parsing rlib files means you'll probably end up making invalid assumptions about layouts, which will lead you to writing/generating broken code.

Mario Ortiz Manero (I don't remember his u.rl.o username, but his website is NullDeref and he has lots of good content) brought this to my attention after reading an article I wrote several years back regarding plugins, and you can't just say "I'm okay with a little UB". To quote a previous thread on unsafe:

Save me from going `unsafe` here?

To reiterate the point: Undefined behaviour means that although the compiler is allowed to do what you wanted, it also has permission to do something else. Sometimes you just get lucky.

@Fredrik

If the layout of this struct is identical to the layout of String , this conversion would hopefully be optimized away, but if for some reason the layout of String changes, there would be a costly conversion.

Note that your #[repr(C)] version of String will still only contain a pointer and two usizes. So unless you are passing these things around in a really tight loop I doubt shuffling three numbers between registers will have a meaningful impact on performance.

@Fredrik

I may need to query more aspects of the memory layout of a type, for example the address and size of a private field or the address, size and value of a niche.

One approach that comes to mind is writing your own custom derive which lets you query an object's memory representation at runtime.

The full implementation is probably larger than you can comfortably read in a Discourse comment, but this is my toy implementation on the playground. While it looks complex, code like this is really easy for a custom derive to generate.

Once you have a representation of the memory layout, you can copy @Yandros's trick from safer_ffi and use a special cfg-gated function to generate code to consume it.

Again, all the same comments around #[repr(Rust)] being implementation defined/unreliable apply. You should probably look into the abi_stable crate if you want to pass Rust standard library types across the FFI boundary.

@Fredrik

Options I could also convert into a type with repr(C) , but this would be inefficient, as repr(C) doesn't optimize for niches. Option<std::io::Error> can fit in two registers, but a repr(C) alternative to Option cannot, and would instead be placed on the stack

I don't think you will be able to reliably understand niches in C without either using some sort of shim function that does x.is_none().

You could parse std's source code and look out for the #[rustc_layout_scalar_valid_range_start(...)] or #[rustc_nonnull_optimization_guaranteed] attributes, and reimplement the rest of rustc's niche optimisation logic in your own code, but at that point I'd rather pull in rustc as a library and ask the type system directly.