Is it UB to read uninitialized memory from another process via shared memory?

1

Imagine this setup:

  • Two processes share a memory region created via shm_open + mmap (or similar).
  • In process A, I have some uninitialized memory.
  • I copy its raw bytes into the shared memory.
  • In process B, I read those bytes and treat them as opaque data (e.g. write them to a file), without interpreting them as a Rust value.

Example Process A:

// process A (writer)
use std::mem::MaybeUninit;
use std::ptr;

let src: [MaybeUninit<u8>; 1024] = [MaybeUninit::uninit(); 1024];

unsafe {
    // shared_mem_ptr: *mut MaybeUninit<u8> points into the mmap'd shared region
    ptr::copy_nonoverlapping(src.as_ptr(), shared_mem_ptr, 1024);
}

Example Process B:

// process B (reader)
unsafe {
    // shared_mem_ptr: *const u8 pointing into the same shared region
    let bytes = std::slice::from_raw_parts(shared_mem_ptr, 1024);
    std::fs::write("dump.bin", bytes).unwrap();
}

Now, here is my line of thinking:

  1. Whatever the process A is doing is perfectly legal, you are allowed to use ptr::copy_nonoverlapping with uninitialized memory per the answer to Is it UB to byte-copy memory containing repr(C) structs with padding?
  2. There is no way for reader to know whether the part of RAM it is reading is or isn't uninitialized memory, hence it seems impossible for compiler to create UB in this case.

Am I missing anything? Can this really go wrong in any way?

2

In practice it should work fine and semantically it works like the proposed freeze operation. But if we are talking about theory, then AFAIK inter-process memory-based communications are largely a grey area in current Rust (especially without asm!-based barriers), so you have a bigger problem.

1 Like
3

So long as "opaque data" stays as MaybeUninit<u8>, you can do that.

It's very unclear to me what writing uninit into a file means, though. You'd probably have to ask the OS what that means to them.

4

The compiler does not create UB. That is a property of the operational semantics of your code. For instance from_raw_parts requires that the pointee is valid hence the execution of the call to that function as in Example Process B is undefined behavior (uninitialized memory is not a valid u8 array).

I don't see a problem with this in principle, but very few interfaces for it are defined so you're going to have troubles creating something provably sound in practice. For instance you just can't use many of Rusts' io traits since they require slices of initialized bytes to be passed to most functions. The raw OS interfaces generally don't have a concept of uninitialized memory so you could call them just fine. (Do consult your OS documentation though, if the pages behind the memory are not yet mapped it may well behave very unexpectedly; in a way your code needs to handle).

Using the OS to externalize memory and read it back is a funky way of interpreting an arbitrary potentially-uninitialized region of memory.

1 Like
5

The OP question is specifically about copying [MaybeUninit<u8>] data to shared memory in one process and re-interpreting the memory as [u8] in another process.

1 Like
6

The compiler is free to assume that &[u8] can't change at all, and make memory safety depend on that, but when shared memory is involved, the memory exposed as this slice could be changed by another process.

It won't be UB if you can ensure that another process can't write to this memory while it's exposed as a slice.

5 Likes
7

That really doesn't exist at the OS level as machine level bytes only have 256 possible values. So writing to a file must have an implied freeze on the uninit value.

The same must be true with shared memory between processes, otherwise you could never treat it as a security boundary (which is necessary when user space calls into the kernel for example).

You could rightly claim that this is all underspecified, and it is. But I can't see any other way for multi-user/multi-process computing on typical hardware to work at all if it doesn't work like that. Maybe it would be different on something like CHERI, but that isn't something you can actually buy, so it seems quite irrelevant unless you are an academic (and as I understand it, CHERI doesn't really look at uninit memory, just provenance of pointers).

1 Like
8

There are signs as to how it might be specified eventually: a concept that compiler must bring all memory visible to a foreign call into accordance with its AM values, and otherwise follow the calling conventions. The former might include that every visible byte, including padding and otherwise uninit, is set to one of [0; 255] values.

Given ReadProcessMemory and /proc/*/mem, we effectively already have the freeze operation on major systems. If the above ends up as specification, we'll probably have even std::hint::black_box(&raw mut maybe_uninit) freezing the specific bytes.

10

But it can still be UB in the C code in the kernel before it gets to the disk, at which point it could be anything.

11

The moment the write syscall was made there must be an implied freeze as the buffer crosses from user space to kernel space. So no? Unless you mean that the kernel itself initiated the write and contents to write, which is unusual but fair enough. In which case there is again an implied freeze when the DMA controller takes over and copies it to disk.

There are never uninit bytes on disk, but whatever gets written could be random garbage, which is consistent with a freeze having happened.

12

The same must be true with shared memory between processes, otherwise you could never treat it as a security boundary (which is necessary when user space calls into the kernel for example).

Calling the kernel is a separate issue. The kernel is carefully coded specifically to avoid corruption by userspace. Most programs aren't. Sharing memory is a deliberate action from all participating processes, and it is in itself a rejection of the inter-process security of memory. While one can use it as a safe IPC channel, it's not uncommon to directly interpret shared memory as in-process data structures, which can easily cause UB if the other process misbehaves.

The moment the write syscall was made there must be an implied freeze as the buffer crosses from user space to kernel space.

That's likely, but you won't find any guarantees of that kind in OS manuals, and I think bug reports of the kind "OS behaves weirdly when I cause it to read uninitialized memory" would not be well received, unless it affects the system's security or stability.

There are never uninit bytes on disk

There certainly can be. Memory cells can be in a metastable state, where reads would return whatever. Magnetic storage doesn't have to be in any well-defined "up" or "down" state if you never write to it. SSD controllers could do weird things when you ask for sectors which were never written. And I wonder what happens if one reads uninitialized data on a DVD disk, i.e. the sectors that were never burnt and are factory-shiny.

Are those things likely? Not really, at least not in quality equipment. It's not that hard for storage devices to implement edge-case semantics for those accesses (either forbid them or return something relatively well-defined), and it increases system stability. But it is something that is at least possible, and one can encounter all kinds of weird behaviour in out-of-spec operations on cheaper or more simplistic devices.

1 Like
13

The Rust's memory model basically assumes that the whole world behaves as a collection of Rust Abstract Machines, obeying Rust's rules. Deviations from that model are possible, but they must not be observable as far as a Rust process is concerned. Your example directly violates Rust's assumptions, so I'd say it is UB. Your question is rather whether you can get away with it, and that crucially depend on the specifics of the system which don't allow any general answer. Probably you can, but I wouldn't put any bets on it.

For example, on the writer side, if the compiler knows that it's reading and writing uninitialized bytes, then it's free to skip those operations entirely. That is specifically what it tries to do when e.g. dealing with padding bytes. Those are defined to be always-uninit, so that the compiler can skip any operations on them if it helps.

So your write can be skipped entirely. Then, on the reading side you're not just reading "written MaybeUninit<u8>". You're truly dealing with uninitialized memory, so all bets are off again. The memory could be write-only, so any reads will fault. It could be achieved via madvice, or it could physically be a piece of read-only memory, which is ok since the writer never wrote anything.

The memory pages could be unmapped, not backed by any physical memory. Again, the OS is free to do whatever if you try to read unmapped pages. Most likely it will conjure a fresh page, either a freshly-zeroed one or one of your stale process pages with old garbage. But it could return you a page previously owned by some other process (not on a modern desktop OS, but older or more simple ones certainly can do that). It could fault, since it knows that the page was never mapped, thus never written, thus you're reading uninitialized memory, which is a bug. A solid choice for a security-focused OS. It could even skip operations on that page entirely, since you're not reading it yourself but pass it to a syscall. It's not likely, since if that syscall checks page tables, it could and should just fault on invalid accesses. But it's certainly possible, and I doubt any OS would guarantee you otherwise.

And of course if you can't guarantee that the filesystem write ever happened, you can't assume what happens when you try to read it later.

Overall, it truly is in the Undefined Behaviour territory. But in practice, I think you'll get away with it on modern platforms, at least if the relevant pages were prefaulted and read-write accessible.

4 Likes
14

Generally for SSDs you get all zeros or all one back in my experience. Pretty sure CDs and DVDs would just return read errors / not let you read the empty bits at all. Optical media have heavy error detection and correction that is performed in the hardware, as software you never get to see the "fuzzy" state.

A dying HDD or SSD can absolutely give you different results each time (but more often than not on modern SSDs you get read errors since checksums will be bad). However all of these are becuase of hardware failures / cosmic rays / bit rot, not because you are able to write uninit bytes. That is what I claimed you couldn't do, and I stand by that statement.

How a badly implemented reader can misuse it isn't really relevant for my claims. If the reader does a memcpy from the shared memory and starts processing it, it must logically have frozen. If working on place on the data you would need volatile/atomic operations just as you would if dealing with MMIO. A slice is not enough since it would violate the safety invariants if the other side writes concurrently.

Uninit memory however cannot transfer between AM instances, so logically there must be a freeze. Uninit really only exist for the optimiser/compiler. Not in hardware (for standard RAM at least, if you were to read some address that isn't electrically connected you can get floating voltages on some older designs, such as old 15 bit home computers, but with modern DDR you can't even get to such a situation, the MMU knows what memory addresses exist). So my claim is still correct.

3 Likes
15

I don't think that's true. I never saw any official statement of that kind. If anything, I think that the memory exists as a global shared state between all Rust AM's (although not all of that is accessible to every AM). Uninit memory specifically is modelled as an extra 257th value of the byte, so it would be the same for every AM which accesses it.

I agree that it's practically unlikely, on quality modern hardware. But we're splitting hairs at that point. From a general "is this defined" question we have turned it into a question about various specific pieces of hardware, which at the very least can all provide a different answer. So the question "what does fs::write of uninitialized bytes do" can only be answered with "who knows, try and see".

Maybe you're running on Itanium or CHERI, so it never even gets for the periphery to decide. The CPU will just fault unconditionally on invalid reads.

16

If that was true FFI with C would be impossible, aliasing rules in the two AMs are different. And shared memory isn't explainable in the AM so we have to drop down to the CM or treat it as FFI/assembly.

How would this be possible on the concrete machine (CM)? The kernel initialises the memory before giving it to userspace and even if it didn't you would have garbage in it. As there is no 257 value for a byte in the CM, it cannot transfer between AM instances. As such the AM must treat the data being read as a black box, just as if it had been handed to it over FFI / assembly.

(This is assuming you don't have concurrent modifications that break the aliasing model and you have done the needed memory barriers, obviously.)

Yes, but those are not the issues we are discussing. The other process could indeed make you get a SIGSEGV or SIGBUS if you aren't doing things carefully (or using sealing). But you could also just use kill to send such a signal to the process. None of what you have described here is UB.

Lazy allocation by the OS is standard practise. POSIX semantics must still be respected on a POSIX OS, and similar semantics will be documented for Windows.

Sure, but none of that is UB, you just happen to initially get some non-zero garbage. Not UB (unless you try to interpret the data as a type where that bit pattern is invalid, which wasn't the question here).

I don't think that would be valid semantics for POSIX. As far as the OS is concerned you allocated the page when you mmaped it.

I'm not sure what you're trying to say here. If someone forgot to use fsync the data may not be persisted? Yeah, but that is not really under the purview of the rust AM. Or are you talking about hardware failures or buggy OSes? The AM doesn't consider those either.

3 Likes
17

Please explain the mechanism for this on actual hardware. I'm saying that, yes it is underspecified, but on any actual CM it would be impossible for it to happen, since we must bounce via the CM when two AMs in separate processes communicate. As such there is no optimisation that could detect or take advantage of cross process uninit.

The AM is a mechanism to explain and justity optimising compilers. There is no cross process LTO, as such there must be two separate AMs that talk via a CM.

So yes, it is currently underspecified, and ill defined. But there is no world in which it could be defined otherwise once they get around to it. Or modern multi-tasking operating systems would not work.

18

And thus it would be possible for otherwise-sound C code (sound w.r.t. C’s rules) that you call via FFI to cause UB in Rust code. The operations performed by the called C code, as a whole, are supposed to be equivalent to some sequence of AM instructions that the AM would permit. Even if it’s unknown what exactly those instructions are.

A black box is treated as potentially performing any arbitrary sequence of instructions (in the AM) that could be performed without introducing UB[1], AFAIK.

Edit: I suppose “in an AM” is more correct? I’m currently reading through Where does one instance of the Rust Abstract Machine end and another one start? · Issue #543 · rust-lang/unsafe-code-guidelines · GitHub

  1. Maybe the exact qualifier isn’t exactly correct, but the important part is that it’s still AM instructions. ↩︎

1 Like
19

Comparing with FFI to C was probably not the right choice here: they are in the same process and with cross lang LTO things get tricky. A more apt comparison is a syscall, that truly is a black box.

In this case (as long as you don't do concurrent writes and have the proper memory barriers) the behaviour can be explained by the AM though. But you still can't transfer uninit memory across, there must be an implicit freeze, which was the point I was making.

This will also be true for FFI (excluding cross lang LTO again) though. If I call a binary blob and get a buffer of stuff back (that corresponds to mapped memory that doesn't trap, and that isn't bering concurrently modified) LLVM cannot assume it is uninit. To it, it is just a black box function call that gives back some sort of buffer of stuff.

EDIT: Let's consider a thought experiment: I have an OS with two syscalls: allocate_page_with_whatever_was_there_before and allocate_page_and_fill_with_hw_rng. Both have the same signature, returning a pointer to a newly allocated page. Rust AM cannot tell them apart and say that one is uninit and the other isn't. It will trust whatever signature I give it to describe the situation. If I tell it that the data is uninit, so it will be, if I say it is a *mut [u8] that will work too. From an AM perspective the syscalls are functionally identical.

20

That's just your assumptions. I say they're wrong. Please provide a link to any official statement if you claim otherwise.

Given that CM is pretty much impossible to specify by definition (it could do absolutely anything, and there is a plethora of different CMs), I highly doubt you can find any evidence.

I mean, I have written two lengthy posts which model that behaviour in the real world, and you have even agreed that it can happen in practice. But you just shrug it off, because it's not your real world. In your world, you run on server-grade hardware with a mainstream build of a mainstream OS (could be anything as long as it's Linux), with full POSIX compliance and highly reliable security-hardened hardware. In my real world, the concrete machine could be anything. We could be running on a toaster. Could be running on a GPU. Could be running in a fully virtual world like WASM or RISC-0. POSIX doesn't exist, the hardware could be doing anything, as long as the infra team is willing to accept it as functional.

And does POSIX specify operations on uninitialized memory? I'd like to a see a link for that.

We're really speaking different languages if I explain why stuff is undefined and can fail in all kind of mysterious ways, but you reply with "but it works on my Ubuntu 25 box with DDR5 memory".

Sharing memory is quite simple to explain in the AM: you have to machines which access a single commong pool of memory. If your AM is multithreaded, then a collections of AMs can itself be modelled as an AM with concurrency.

And different aliasing models aren't an issue. It is up to you to provide safe abstractions over concurrently accessed memory, and simply having the memory available to multiple AMs doesn't violate anything on its own. Also, Rust's pointers have basically no aliasing restrictions, which is actually less restrictive than in C, where pointers to different types can never alias, unless one of them is char.

And again, none of the memory model differences matter if the other side can't observe it. Rust's memory model is based on actual execution via small-step semantics and actual memory accesses, not abstract properties. If the actual accesses don't violate aliasing rules at runtime, then everything is fine.

21

What does shared memory have to do with this? It seems like there are three separate UB questions here (mmap, safely reading shared memory IPC, system calls that read uninitialized memory).

Why not just use a representation that doesn't have padding (adding reserved fields if necessary)? If you care about the representation of your structure across a process or disk boundary, how can you rely on the compiler to not change it when you rustup update?

1 Like