Understanding pointer offset/wrapping_offset safety

#1

The offset method on pointers is unsafe. The wrapping_offset method is not. Apparently the difference is in what sort of assumptions LLVM is allowed to make and the former in theory can have better codegen. naturally I want to write the fastest possible code :slight_smile: So I'm trying to wrap my head around when I can get away with using offset.

I'm normally writing code for modern 64-bit machines which have huge address spaces, so I'm not worried about these requirements:

  • The computed offset, in bytes , cannot overflow an isize .
  • The offset being in bounds cannot rely on "wrapping around" the address space. That is, the infinite-precision sum, in bytes must fit in a usize.

But I find this one much harder to reason about:

  • Both the starting and resulting pointer must be either in bounds or one byte past the end of the same allocated object. Note that in Rust, every (stack-allocated) variable is considered a separate allocated object.

Say using unsafe I mmap a large empty memory region, that I want to use as backing memory for many contiguous instances of some T. To construct these instances I'm going to repeatedly take the *mut u8 that mmap gave me, offset by i*size_of<T>::(), cast it to *mut MaybeUninit<T>, and do a ptr::write to initialize. Is that legal? Or do I have to use wrapping_offset? Did mmap create one logical "allocation"? What if I mmap a large region with PROT_NONE, then later change some of the region to be read/write with mprotect?

#2

What this is saying is that if I've got some variable x, I can't use pointer arithmetic to somehow get a pointer to some unrelated variable, y.

A common example of this is when people take pointers to two unrelated variables on the stack (aka "from different allocations"), calculate the difference between the two, and later on try to reuse that difference for offsetting a pointer.

let x = 42;
let y = 0;

let x_ptr = &x as *const u32;
let y_ptr = &y as *const u32;
let difference = y_ptr as isize - x_ptr as isize;

unsafe {
  let ub_pointer_to_y = x_ptr.offset(difference);
}

The reason this is is UB is because it breaks LLVM's ability to reason about aliasing, which in turn means some optimisations will generate broken code. It also opens the door to changing memory you shouldn't have access to (e.g. heap corruption).

The short answer is "yes". You got one big allocation from whatever mmap function you used and can do whatever you want with it, then you'll be passing out smaller allocations to your caller and the caller will only be allowed to touch the memory you gave them.

In general, if some function gives you a pointer to something, you can't use pointer arithmetic to get an address to to something outside the object being pointed to. The one exception is when you are given a pointer to an array, in which case it's fine to use pointer arithmetic to get a pointer one past the last item because that's how for-loops are often implemented in C++ (e.g. in for (auto p = std::start(my_vec); p != std::end(my_vec); p++) { ... }).

Assuming you don't accidentally go out of bounds and only ever store Ts (mixing sizes means your offsetting needs to be smarter), that's perfectly sound :+1:

I'd just use start_of_region.add(i*mem::size_of<T>()). I don't think I've really seen wrapping_offset() used much in practice for much the same reasons you've already mentioned (i.e. 64-bit pointers are huge).

I don't think that has anything to do with pointer offsetting, but it would probably break things downstream. Your consumer will be assuming they can do whatever they want with the memory they've been given until they call some sort of free() function, and that assumption is what makes their unsafe code (e.g. reads and writes) sound.

If you then go and retract that ability without letting them know, you will violate that assumption and they'll have a bad time the next time the caller tries to use that newly mprotect-ed memory.

4 Likes
#3

I guess I'm trying to build my understanding of how rustc/LLVM comes to the conclusion that the pointer I get back from mmap is a single allocation.

  • Is it that LLVM gets a positive indicator, like the function is annotated somehow as returning one logical allocation?

  • Or is this more a case of negative reasoning -- separate local variables are annotated as being distinct allocations, but pointers returned by FFI functions like mmap are not, so the compiler has to assume they may alias anything?

I'm also curious how this works out for something like the mmap mirror trick. There you have two virtual pages mapped to the same physical page, a situation created by two distinct mmap calls. If you have a pointer into the first page and you offset it to a pointer in the other page, you've landed at an address that physically is the same allocation, but if every mmap region is considered distinct, from LLVMs point of view they may be distinct. There may also just be a bigger kettle of fish with Rust probably somewhere having an assumption that objects have a unique memory address.

#4

Be careful! This depends on how you got that pointer into the array!

Any time you have a reference and turn it into the raw pointer, that raw pointer no longer has access to the full allocation, but only to the memory that the reference could have accessed.

It doesn't need to know. The point is that sometimes it knows, and since you have to uphold the guarantee no matter if the compiler knows or not, it knows that in particular, you will uphold the guarantee in the cases where it does know, so in those cases, it can use the knowledge for optimizing code.

It probably doesn't know much about the mmap'ed allocation, but since any kind of access to any index in the mmaped region requires an unsafe block, it doesn't need to know. It can just trust that you are doing the right thing.

This is probably quite dangerous UB-wise. For example, mutable references assume that they have exclusive access, so it wouldn't be valid to create a mutable reference to such a region unless you are sure that the access is exclusive to both mirrors.

1 Like
#5

Yeah when I tried to make the Rust equivalents of a std::start()/std::end() loop I found you had to be really careful about how you get those pointers.

The naive implementation with a start() and end() function would have created aliased pointers (interestingly, stacked borrows didn't complain here) so I needed to do it all in a single function.

That seems super sketchy and a great way to have concurrency issues...

#6

If you run it with

MIRIFLAGS="-Zmiri-track-raw-pointers" cargo +nightly miri run

it will catch the mistake.

2 Likes
#7

It doesn't need to know. The point is that sometimes it knows, and since you have to uphold the guarantee no matter if the compiler knows or not, it knows that in particular, you will uphold the guarantee in the cases where it does know, so in those cases, it can use the knowledge for optimizing code.

The problem I have with this is it's not clear to me what counts as a single allocation, even if I want to uphold it. When I use mmap to allocate a chunk of memory I may use that memory to store arbitrary different types, so it simultaneously feels like one allocation or many allocations.

Say I make a Vec<u8>, and I use casting and MaybeUninit to create a T inside. Once I have a *T, can I do offset on it to arrive at another *T in the vector? Or do I have to do the math on *u8? Certainly it seems based on what everyone is saying I can't do math once it has become a &T.

Using mmap or a backing Vec<u8>, and without casting from reference to pointer is it possible to construct an example of where using offset is UB?

Hypothetically if there were a single system call that took a Vec of desired mmaps, and did them all in one call, and returned to you a Vec of resulting addresses that are not all contiguous, would Rust consider that one allocation or multiple?

#8

What counts as a single allocation for this rule is:

  1. When the raw pointer originally came from an allocator (e.g. malloc, mmap), the allocation is the entire allocation.
  2. If the raw pointer was created from a reference (mutable or immutable), then the raw pointer must stay within the bounds of whatever the reference could have accessed.

So e.g., a raw pointer to a stack variable can only access that stack variable because the only way to get a raw pointer to a stack variable is to first create a reference and cast it to a raw pointer.

As for arrays, if you call slice::as_ptr or slice::as_mut_ptr, you get a raw pointer valid for the entire slice. On the other hand, if you call &arr[i] and cast that to a raw pointer, then it can only access that single array element, since you first created a &T to that single element.

It doesn't matter whether it is a *mut T or a *mut u8 when you do the math. Either both are valid, or both are invalid.

To answer the question in a bit more detail, consider if you use slice::as_mut_ptr to get a raw pointer. Offset it to get a raw pointer to a specific index, then cast the raw pointer to a &mut MaybeUninit<T>, and use this mutable reference to write to the value. In this case, it would still be valid to use the raw pointer for other indexes, because the raw pointer was not created from the &mut MaybeUninit<T>. Of course, if you cast the mutable reference back into a raw pointer, that new raw pointer would only be valid for that element, but this doesn't change anything about the original raw pointer.

As long as you stay within the bounds of the vector, no.

There is no such system call, so no rules have been written to handle that case.

3 Likes