The following playground link works as expected:
But I am uncertain if the correctness of this is always 100% safe as I think it is.
My rationalisation is that since the value of the pointer is never taken, no actual "dereference" occurs, and thus I did not dereference an invalid pointer; I'm not sure if it's valid because of an optimisation (bad) or language semantics (good!).
The semantics here are pretty subtle. This part of the FAQ for the LLVM GetElementPtr instruction indirectly covers this kind of thing: The Often Misunderstood GEP Instruction — LLVM 16.0.0git documentation.
I believe the more clearly reliable approach is to do the computation with an instance of the struct on the stack. It'll still optimize down to a constant, but opens the door to less UB: https://github.com/rust-lang-nursery/unix-socket/blob/master/src/lib.rs#L21-L29
Reminds me - does anyone know if there’re plans to add a compiler builtin to get the offset of a field?
I don't know about a compiler built-in, but here's what I've been working on:
https://crates.io/crates/memoffset
It's great that you've published that in a crate. But, this really ought to be in the compiler 
. For instance, it should be a const fn.
What would the signature of the function be?
It can be a macro, doesn’t have to be a function (yes, Rust doesn’t have a way to express this as a generic fn without some stringly API). My main point is this ought to be a builtin, with whatever syntax/expression chosen (separate bikeshed). I mostly wanted to see if any thought had been given to this by core lang team, and if so, what the thinking is.
As it stands, one can rely on LLVM optimizing out the “stack dummy” approach (but maybe it’ll fail to do so under some circumstances), a bit of unsafe code (granted, contained but there should be nothing unsafe about getting this value), but we don’t get const context goodness (as an example).
I completely agree, but that's still needed for the meantime, until someone implements it in the compiler.