Hi guys,
I'm writing some enums to cross Rust and C FFI boundary. I know that if we specify #[repr(C)], #[repr(u*)] on a Rust enum, it would have a stable C tagged-union layout. However, due to there is no binding between tag value and corresponding union variant in the C code, it's possible for C code to set a tag value with no corresponding Rust enum variant, and match expressions in Rust code assume there is only valid variants(so as tag values), then we got a UB if C code set an invalid tag value.
I have tried #[non_exhaustive] attribute, but unfortunately it doesn't help in this scenario, because it has no effect in defining crate, and in my scenario it's not reasonable to put all such enums into a separate crate for some reasons. Besides, I don't want to define tagged-union in Rust instead of use enums directly. Is there any other solutions?
non_exhaustive doesn't mean that it's not UB to have an unknown tag value. It only affects pattern matching, not the type's validity condition. You cannot use an enum type this way.
You can't do that. For fieldless enums, you need to explicitly pass to/from C raw integer types, or something bitwise-equivalent to them (e.g. #[repr(transparent)] struct MyEnum(u8);). Then you should do an explicit fallible conversion in Rust code from FFI-appropriate pseudo-enum into an actual Rust enum. For field-carrying enums, you should use instead explicit structs with inner union fields.
That is, if you want to program defensively. Another option is to ignore those possibilities, trust the C code to do the right thing, and just pass #[repr(C, u32)] enums to and fro. Depending on the complexity and trustworthiness of C code, this may be a valid option. Of course, it would be painful to debug if C code misbehaves and causes undefined behaviour, but C code has plenty of other ways to do that anyway.