Soundness of replacing pointers with Option references

A single-threaded process will call one of my extern "C" Rust functions and pass it a raw pointer to a structure with nested raw pointers. Here's what I mean.

Null-checking the nested pointers is tedious and error-prone. I could forget to null-check a pointer.

The nullable pointer optimization says the memory layout of a *const T is bitwise-equivalent to a Option<&T>.

If I replace the raw pointers with Option references, then I can null-check by using the ? operator to short-circuit a function that returns an Option. Furthermore, it's impossible for me to accidentally dereference one of the nested pointers without null-checking. Here's what I mean.

I've tested both the raw pointer and the Option reference solutions, and they both produce the same results.

I would like to know whether this conversion is unsound and why.

Yes, that use of Option is sound.

I think your use of 'static is suspect, because it would let you leak one of those references in your Rust code, but you could annotate real lifetimes for that, like so.

There are subtle differences between pointers and references that make this not a 1-to-1 replacement that is always ok.

Variance is the obvious one, though subtle as it is.

A more insidious one is that you can e.g. have *const u32 that you use to access a [u32; N], but doing the same with &u32 is UB; accessing outside the pointee type of a reference is always UB where for pointers it only matters from whence the pointer was derived.

For an easy always-applicable* option, you can use Option<ptr::NonNull<T>> to replace *[const|mut] T.

* assuming you use PhantomData to get the right variance -- though that only matters for variant types (type parameters) and not concrete types

If a pointer is ever at "rest" (stored in some structure) I tend to recommend using ptr::NonNull. If it's "in motion", however, ptr::NonNull lacks a lot of pointer manipulation facilities, meaning you'll be converting back and forth a lot and maybe should stick to raw pointers.

5 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.