How to implement a C union in Rust


#1

Hi there! I’m trying to create an interface to a C library, but I am stuck with this peculiar situation. As it can be seen here: https://github.com/tbuktu/libntru/blob/16c3a8749bcb96c800a22f475f6582685e3758f8/src/types.h#L37, there is an struct with a union made of two elements. How can I represent that in Rust?


#2

It’s an open issue:

The current approach I use is to substitute the union with a #[repr(C)] struct that is composed to have the same size and alignment, and address possible target variation with alternative definitions, conditionally selected via #[cfg(...)].
Unsafe access to union variants can be provided by transmuting, if needed.


#3

I have implemented a struct to work like the union, but it seems that it does not work properly:

How can I improve that solution to work properly?


#4

Your struct is just a struct, it has two members laid out in sequence.

If the C definition of the union has the two structure variants as provided in your code, they seem to be all filled with varying numbers of 16-bit words. So the C-compatible union stand-in would look like:

// No need for pub, it's an
// implementation detail of another structure
#[repr(C)]
struct PrivUnion {
    data: [uint16_t; UNION_SIZE]
}

, where UNION_SIZE is determined as the minimal number of 16-bit words that fits the longest variant. As the structure content sizes in your case are parameterized with constants, the value should probably be a constant expression over those constants.

Access to variants can be then implemented as (again private, the consumers need only see the safe API):

impl PrivUnion {
    unsafe fn as_tern(&self) -> &NtruTernPoly {
        let p = self as *const _ as *const NtruTermPoly;
        &*p
    }
    unsafe fn as_tern_mut(&mut self) -> &mut NtruTernPoly {
        let p = self as *mut _ as *mut NtruTermPoly;
        &mut *p
    }
    unsafe fn as_prod(&self) -> &NtruProdPoly {
        let p = self as *const _ as *const NtruProdPoly;
        &*p
    }
    unsafe fn as_prod_mut(&mut self) -> &mut NtruProdPoly {
        let p = self as *mut _ as *mut NtruProdPoly;
        &mut *p
    }
}

#5

I think I understand. Doing a sizeof(union) / sizeof(uint16_t) in C gives 3004 words, what means that my struct should have data: [uint16_t; 3004]. Is this correct? the problem is I get an stack overflow :confused:


#6

@Razican Have you tried putting the struct in a Box? This sounds like the perfect use case.

However, the C library might be doing something a bit smarter here. I doubt it would allocate all that memory if it wasn’t going to use it. There might be room here for some cleverness with DSTs, but I’m not sure.


#7

What would change allocating it in heap? About the C library, the variable is initialized in the Rust side, so it shouldn’t be a problem, it should have the correct size.


#8

Boxing it should eliminate the stack overflow, though you may need to use the box keyword instead of Box::new(), since the latter will probably retain the on-stack copy, at least with optimizations off.

C libraries usually provide a way to allocate big structs like this. You’d probably still get a stack overflow if this were in C unless you allocated it on the heap.


#9

Dealing with unions in a common problem when trying to bind certain data structures in the windows header to Rust types. The approach that the winapi-rs crate takes is to use a macro where each variant is accessed by a method.


#10

We are trying to use stable Rust, as far as possible, and I managed to do this: https://github.com/FractalGlobal/ntru-rs/blob/develop/src/types.rs#L167

The problem is that I get a Illegal instruction (core dumped) on runtime.


#11

Your issue is in line 221 of types.rs: in the C version the poly field of NtruPrivPoly is embedded directly rather than being a pointer, so you shouldn’t have a Box in the Rust version.

edit: By the way, an alternate approach would be letting bindgen do some of this for you.


#12

I did what you mentioned, and I even used the bindgen, which suggested using mem::transmute(). Using that I get 3 types of output:

running 1 test
Illegal instruction (core dumped)
running 1 test
Segmentation fault (core dumped)
failures:

---- it_keygen stdout ----
    thread 'it_keygen' panicked at 'assertion failed: `(left == right)` (left: `{ n: 401, coeffs: [1...0] }`, right: `{ n: 401, coeffs: [1...0] }`)', tests/lib.rs:48

With the same compiled source code. The new code: https://github.com/FractalGlobal/ntru-rs/blob/develop/src/types.rs#L168
As you can see in line 212, it’s no more a Box, but continues giving segmentation fault / illegal instruction errors.


#13

The only things I can see are:

  • You probably don’t want to compare all of the data in PrivUnion’s PartialEq implementation, since if the shorter union variant is in use (prod_flag = 0), the remaining space may be uninitialized memory. (Unless I’m missing something in libntru that always initializes it.) Instead you should determine which variant is in use and compare that one.
  • For safety, get_poly_prod and get_poly_tern should verify that prod_flag is actually 1 or 0, respectively, before doing the transmute. I tried adding an assert to this effect and it triggered in the it_keygen test. Not sure why you dropped the conditional in your version of decrypt_poly compared to libntru’s C version.

#14

It’s working now, thanks!!!


#15

I have come across a new issue here. I am trying to create that union from the original types (NtruTernPoly and NtruProdPoly) but I get a segmentation fault with the current script:

unsafe fn new_from_prod(poly: NtruProdPoly) -> PrivUnion {
        let arr: &Box<[uint16_t]> = mem::transmute(&poly);
        let mut data = [0; PRIVUNION_SIZE];

        for i in 0..arr.len() {
            data[i] = arr[i];
        }

        PrivUnion { data: data }
    }

I have an equivalent function for the NtruTernPoly. The segmentarion fault happens at the transmute. The thing is that if I change the box by a slice (&[uint16_t]) the compiler says that transmute called on types with different sizes: &types::NtruTernPoly (64 bits) to &[u16] (128 bits) :confused:

Why is this happening? How could I solve it?


#16

&[u16] is represented as a “fat pointer” which contains both the actual pointer and a length, hence being 128 bits. Instead you should just transmute it to an &[u16; WHATEVER_SIZE] (with the size of NtruProdPoly in u16s), which is represented as a regular pointer since the length is known at compile time.


#17

Didn’ t know that! Thanks, it’s now working perfectly!!