Improper_ctypes, NonNull, and Option

[Michael-F-Bryan]

I'm writing some unsafe code and ran into what seems to be a false positive in the improper_ctypes lint given how I think Option and NonNull work, and now I'm wondering if this is a bug in the lint or a bug in my understanding.

Given the following snippet,

#![deny(improper_ctypes_definitions)]

use std::ptr::NonNull;

#[repr(C)]
pub struct Slice<T> {
    ptr: NonNull<T>,
    len: usize,
}

pub extern "C" fn return_string_slice() -> Option<Slice<u8>> {
    None
}

fn main() {
    assert_eq!(
        std::mem::size_of::<Slice<u8>>(),
        std::mem::size_of::<Option<Slice<u8>>>()
    );
}

(playground)

Compiling on stable (and nightly) fails with the following:

   Compiling playground v0.0.1 (/playground)
error: `extern` fn uses type `Option<Slice<u8>>`, which is not FFI-safe
  --> src/main.rs:11:44
   |
11 | pub extern "C" fn return_string_slice() -> Option<Slice<u8>> {
   |                                            ^^^^^^^^^^^^^^^^^ not FFI-safe
   |
note: the lint level is defined here
  --> src/main.rs:1:9
   |
1  | #![deny(improper_ctypes_definitions)]
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
   = help: consider adding a `#[repr(C)]`, `#[repr(transparent)]`, or integer `#[repr(...)]` attribute to this enum
   = note: enum has no representation hint

error: could not compile `playground` due to previous error

My understanding is that when some type, T, contains a "niche" (e.g. the all-zeroes representation in NonNull), Option<T> will use that niche for the enum discriminant and make the layout of Option<T> and T identical. When the niche isn't being used (i.e. there is a non-null pointer in the NonNull), it is also valid to transmute Option<T> to T.

This relationship is also recursive (i.e. Vec<u8> has a niche, so String also gets this optimisation because it's a newtype wrapper around Vec<u8>).

Given my Slice<T> is FFI-safe because of the #[repr(C)] attribute, and the layout of Option<Slice<T>> and Slice<T> are identical, why does Option<Slice<T>> trigger the improper ctypes lint?

[SkiFire13]

Option's niche optimization is not guaranteed in general, only for a couple of types and #[repr(transparent)] wrappers around them (see the documentation). Your Slice type contains an additional len field so the guarantee doesn't apply.

[Michael-F-Bryan]

Ah, so it looks like I'm getting the null pointer optimisation in practice, but the lint is being triggered because it's not guaranteed by the language for structs with more than one member?

It seems like I mis-remembered the #[repr(transparent)] dot-point and thought it was #[repr(C)].