Memory safety accessing POD (plain old data)

1

I have a question about safety, but it is not about the theoretical specification of UB, which I understand to mean "anything can happen because the behavior is undefined". Rather I'm asking about the actual behavior in the case of access to POD, as defined by bytemuck's Pod trait. I have been hesitant to ask this question because I'm not interested in the usual "UB is UB" answer. I know this is not a question that is normally entertained, and if you feel "UB is UB" is the answer, then please just skip this post.

My motivation is that I have the following requirements for my database cache, which is a large array of bytes divided into fixed size blocks.

  1. Each block has an id and is owned by a single Table or Index. Block ownership can be transferred, with proper synchronization.
  2. The runtime cost for accessing a block is limited to a simple bounds check on the block id, which is just an index into the array.
  3. Data races when accessing the blocks should be avoided, but are tolerated in the case of access bugs, because the runtime cost of guaranteeing they will not occur is more than a simple bounds check.
  4. Using raw pointers and the large amount of necessary associated unsafe code is not acceptable. Therefore, regular references to blocks are used.
  5. As a result of the above requirements, the "mutable xor shared" constraint cannot be guaranteed for references to the cache blocks, across threads or within a single thread.

My understanding of this scenario is that it could cause both data races (which I can tolerate) when accessing a single block in multiple threads, as well as possible miscompilation when violating the "shared xor mutable" rule by accessing a single block via multiple conflicting references within a single thread.

My specific question is whether the miscompilation risk is real. I assume it is, but I'd like to double check with those who more than I do. If so, this rules out Rust for me, which ok -- I'm already resigned to using another language without the "mutable xor shared" restriction. But I'd like to double check my understanding just in case I might possibly be able to use Rust.

Thanks in advance.

2

Obligatory UB is UB. But I get where you come from.

Pratically speaking, as long as you are not holding &mut Block for very long, there should not be "miscompilations". You can hold a raw pointer in a newtype wrapper but provide a dereference method without unsafe marker. Something like this:

struct BlockRef<'a>(*mut Block, PhantomData<&'a mut Block>);

impl DerefMut for BlockRef<'_> { /* ... */ }

And you create BlockRef with something like &raw mut to avoid creating mutable reference directly.

3

I have never done this but I think you can get meet your requirements by wrapping each block in your cache with UnsafeCell.

struct CacheBlock(UnsafeCell<[u8; BLOCK_SIZE]>);

Then to access a block by id there is only one bounds check and you only need a single unsafe dereference.

        // Bounds check (handled by Rust's indexing)
        let block = &self.blocks[id];

        // Obtain a raw pointer and dereference it.
        // UnsafeCell ensures this won't be miscompiled.
        let block = unsafe { &mut *block.0.get() }

I don't think we should be afraid of "unsafe". It is essential, we just need to keep it confined and under control.

I'm curios to know what alternate language you have in mind to use?

2 Likes
4

violating the alias xor mutability rule is ultimately potentially UB in any language and can do litterally anything. if you are lucky you end up reading corrupted data and storing it somewhere in memroy where it can cause UB at a later time, if you are unlucky you can end up taking the wrong path in a branch, invoking code at a random location in memory or have nonsensical values in your cpu registries which can cause all kinds of UB

i recommend you just try to find a way to organize your operations so you do not violate alias xor mutability in most cases it only a matter of rethinking who owns the data and what parts of it you give out with every borrow

1 Like
5

As a concrete example of a "miscompilation" due to violating aliasing, here's a program that prints 1 then 2 in debug mode, but prints 1 then 1 in release mode.

fn main() {
    let mut x: i32 = 1;
    let ptr: *mut i32 = &raw mut x;
    foo(unsafe { &mut *ptr }, &x);
}

fn foo(a: &mut i32, b: &i32) {
    println!("{b}");
    *a = 2;
    println!("{b}");
}
9 Likes
6

Also, data races, even benign-looking ones, can cause "miscompilations", even in C. https://www.usenix.org/legacy/event/hotpar11/tech/final_files/Boehm.pdf

4 Likes
7

Here's a "miscompilation" in C involving data races. Compiler Explorer

The assembly says: "If the flag is nonzero, return. Otherwise, enter an infinite loop of doing nothing."

The compiler here is assuming that data races do not occur, so it assumes that no other thread can possibly modify the flag value, so it assumes that the flag value won't ever change after the first check.

4 Likes
8

Miscompilation risk is absolutely real. The most common miscompilation is as follows. Imagine that you have code that looks like this:

let value = my_shared_data.value;
if value != 0 {
    println!("{value}");
}

And imagine that my_shared_data is written to in parallel on another thread. Then you might say, sure, we can get some garbage value due to data race, but it's definitely the case that this will never print zero, right?

Wrong!

A very common and standard optimization will change above code to this:

if my_shared_data.value != 0 {
    println!("{}", my_shared_data.value);
}

That is, it reads the shared data twice! Under the assumption that there is no data race, the above optimization is perfectly legal. Both reads of shared memory must return the same value, since if they didn't that means the data was modified in parallel, but no data races is assumed by optimizer.

So reading POD with data races is definitely dangerous. The correct answer here is to use relaxed atomic load/stores to access this shared memory. Relaxed atomic load/stores compile down to normal mov instructions, so they are exactly as fast as normal non-atomic operations. However, the above optimization is not legal for atomics, so the same miscompilation will not happen.

17 Likes
9

Thanks everyone!

I had a misunderstanding, which is that, when using the cache I described, miscompilations could not occur without conflicting borrows in a single thread. As pointed out, they can occur due to data races. And this is true for Zig (the language I have been using) as well.

My revised approach, which is a compromise, is to use debug-mode runtime checks for cache block ownership and borrowing conflicts. An additional array is created, parallel to the cache array, containing the ownership and borrowing state. In release mode there are no such runtime checks, so this compromise relies on unit tests that are fairly complete, as well as stress/reliability tests that can be run with debug assertions enabled.

The ownership checks are necessary in my Zig implementation as well (I misunderstood that), although the borrowing conflict checks are not necessary (Zig does not require "shared xor mutable" references.) The larger point is that this approach frees me to choose to use Rust instead, without compromising release-mode performance.

3 Likes