I recently found this blog post, published about two months before the 1.0 release: A Type Safety Hole in Unsafe Rust, by Florian Weimer.
The claim is, essentially, that unsafe blocks can be abused to permit arbitrary conversions between types (a la C-style casts). I don't know whether this is true or not, or whether it would be considered a "soundness" hole or not (my impression was that undefined behavior, and hence type-safety holes, are possible in poorly written unsafe code).
However, the code in the blog post does not actually behave in the way the blog post claims. Instead, the "magic" conversion function triggers the assert; without the assert, it simply returns a null pointer.
What I'd like to know is:
When did the behavior change? Did the 1.0 release have this type safety hole? (I was unable to install the 1.0 toolchain using rustup toolchain install 1.0.0.)
Is there some way to annotate the lifetimes and runtime behavior to clarify what was going on when this did work, versus what happens with the current compiler?
Is there a variant of this code that does expose an arbitrary cast-like conversion with the current compiler?
I’m not entirely sure what type safety hole that post is intending to show, but I believe what the code is relying on, and what has likely changed to make this no longer work, is having the same data layout for Magic::A and Magic::B (along with the first and second values using the magic binding occupying the same stack location, which does still occur).
Specifically uncopied is a pointer into the inner structure of the original magic stack value. When the Magic::A is written to magic the compiler reuses the same stack location to store this new value (this is not required or guaranteed by the compiler), now uncopied is a pointer into the inner structure of this new value. Previously it seems that both variants used the same internal layout, so the pointer into the original value and the new value both happen to point at valid Uncopyable structs that had the same layout since they contained thin pointers (although of different types). So access via uncopied would (very unsoundly) pull a value that happened to be a pointer of one type out as a pointer of a different type.
Here’s a playground that may illuminate some (try it in both debug and release mode). At a guess in debug mode Rust is keeping distinct storage locations for every enum variant, even if they could overlap. In release mode as of right now (this is highly dependent on optimisations) something very interesting happens, rather than return a pointer to the expected string, a pointer to one of the static strings used in the extra debug printlns is returned, no idea how this happens, but it goes to show that this code is very unsound and relying on non-guaranteed optimiser behaviour.
EDIT: Re-reading the blog post I think this is where Florian trips up:
The key ingredient there was a mutable variant type which has aliasable components (similar to a union in C). We mirror that in Rust with the following type definitions
A Rust enum is not the same as a C union, a C union has a guaranteed layout that makes this sort of thing possible. A Rust enum is much more abstract and you can’t infer anything about its memory layout from the definition. Rust actually now has unions, so I believe it would be possible to now show this type system hole; but that’s why unions are unsafe, by using them you are promising you won’t abuse them in this way (and as @kornel mentions below the standard library already includes this exact function for you, so you don’t really need to write it out to show this hole exists).
You don't need anything as elaborate as this blog post. Rust's standard library has a function specifically for violating all type-system guarantees: std::mem::transmute().
If you write unsafe code and do something really unsafe, then the warranty is void. Code in unsafe blocks can break things outside that block, e.g. it can change Rust's safe reference to be null, and crash may happen out of that block in "safe" code, but it still because of rule violation only allowed in the unsafe block.
unsafe blocks in Rust are for doing things that Rust doesn't understand or can't prove they're safe, but you still have to write safe code in the unsafe blocks.
Oh right, I had completely forgotten about that. (I have not used it.)
Yes, I totally understand this, which is why I mentioned undefined behavior. It seems the author understood this as well; as I read it, the blog post seems to be a demonstration, not a complaint.
I just wouldn't call it "type safety hole", because Rust has its way of defining what its type safety is. As far as Rust is concerned, shenanigans in unsafe are not a hole, but a feature I'd expect a "hole" in Rust's type safety to refer to something bad happening without use of unsafe.
unsafe does not permit anything and everything. And, in particular, it still has a type system, since unsafe code still has types. So I'm not sure that unsafe counts as opting out of "the type system", even though it opts out of compiler-guaranteed soundness.
Well, given the first paragraph in bold, it seems the author was responding to what they considered a common misconception that the guarantees provided even in an unsafe context prevented arbitrary type transformations.
Hopefully this misconception is not so common any more!
It allows you to essentially reinterpret bits. Sure, it has some checks still and it allows you to use types, but the soundness is opted out - you can create unsound types and trigger UB. I’m not sure why that’s surprising? My disagreement is with calling it a type system hole.
As @kornel mentioned, a true hole would be triggering memory safety violations or other UB in safe Rust. Leakpocalypse is a decent example of that.
unsafe does permit anything and everything. It permits cast of anything to anything, and that's an escape hatch to everything. You can rewrite every byte of your program's memory.
let game_over = unsafe {
std::slice::from_raw_parts_mut(0 as *mut u8, -1isize as usize)
};
