Rust FFI, mut mut and C

tgad · November 12, 2021, 3:10am

I recently generated Rust bindings to some C functionality and ran into an issue where the bindgen generated a function definition that took a *mut *mut c_void. However, passing a *mut *mut c_void returned an error from the function. After some testing I found that passing a *mut c_void did not (I had to change the definition from the bindings). I have attempted to recreate an MVC of the issue below. The following is a bit long, please bear with me.

A representation of the C code. This is not the exact interface as only the API call is documented but the internals are hidden. The following is an approximation of the API functionality. In C you provide a PVOID * and it fills out that pointer with some memory.

// test.h
#pragma once
#include <windows.h>

EXTERN_C DWORD DoublePointerTest(IN OUT PVOID * Address);
EXTERN_C DWORD PointerTest(IN OUT PVOID * Address);

// test.c
#include "test.h"

DWORD DoublePointerTest(IN OUT PVOID * Address) {
	if ( *(DWORD *)Address == 0x0 ) {
		*Address = VirtualAlloc(NULL, 1024, MEM_RESERVE|MEM_COMMIT, PAGE_READWRITE);
		if( *Address != NULL ) {
			return 0;
		}
		else {
			return GetLastError();
		}
	} 
	printf("Error, Address is not NULL: %x\n", *(DWORD *)Address);
	return 0x99000099;
}

DWORD PointerTest(IN OUT PVOID * Address) {
	if ( *(DWORD *)Address == 0x0 ) {
		*Address = VirtualAlloc(NULL, 1024, MEM_RESERVE|MEM_COMMIT, PAGE_READWRITE);
		if( *Address != NULL ) {
			return 0;
		}
		else {
			return GetLastError();
		}
	} 
	printf("Error, Address is not NULL: %x\n", *(DWORD *)Address);
	return 0x99000099;
}

Rust bindgen generated the following bindings.

/* automatically generated by rust-bindgen 0.59.1 */

pub type DWORD = ::std::os::raw::c_ulong;
pub type PVOID = *mut ::std::os::raw::c_void;
extern "C" {
    pub fn DoublePointerTest(Address: *mut PVOID) -> DWORD;
}
extern "C" {
    pub fn PointerTest(Address: PVOID) -> DWORD; // this has been changed manually from *mut PVOID to just PVOID after the bindings were generated 
}

So, in DoublePointerTest, bindgen generated *mut PVOID, which itself is *mut c_void, so its really *mut *mut c_void. I changed the second function, PointerTest, to just take *mut c_void to see if that would work.

Heres the rust code and the test output.

include!("./bindings.rs");

#[cfg(test)]
mod tests {
use core::ffi::c_void;
use crate::{PointerTest, DoublePointerTest};

    #[test]
    fn double_pointer_test() {
        let mut x: usize = 0; 
        let mut x_ptr = &mut x as *mut usize as *mut c_void; // cast &mut x to usize ptr (which it is) then cast that to a void *
        let res = unsafe { DoublePointerTest(&mut x_ptr as *mut *mut c_void)}; // cast to *mut *mut x_ptr
        assert_eq!(res, 0, "Result: {:x}", res); 
    }

    #[test]
    fn pointer_test() {
        let mut x: usize = 0; 
        let x_ptr = &mut x as *mut usize as *mut c_void; // cast &mut x to usize ptr (which it is) then cast that to a void *
        let res = unsafe { PointerTest(x_ptr) };
        assert_eq!(res, 0, "Result: {:x}", res); // x_ptr = *mut c_void
    }
}

     Running unittests (target\debug\deps\test_sys-c05258ef0066ffdb.exe)

running 2 tests
Error, Address is not NULL: 23fff340
test tests::pointer_test ... ok
test tests::double_pointer_test ... FAILED

failures:

---- tests::double_pointer_test stdout ----
thread 'tests::double_pointer_test' panicked at 'assertion failed: `(left == right)`
  left: `2566914201`,
 right: `0`: Result: 99000099', src\lib.rs:13:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    tests::double_pointer_test

test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

error: test failed, to rerun pass '--lib'

In DoublePointerTest I passed in *mut *mut c_void as the bindings expected and received an error. In PointerTest I passed just *mut c_void and it works as expected, no errors. In C you would call the function using something like the following:

void *x = NULL;
DWORD res = PointerTest(&x); // &x = void **x

which does work.
double_pointer_test fails because C tries to dereference the memory and sees that it is not NULL so it returns an error. The real API attempts to allocate memory at this address and then fails (since it is already allocated) and returns an error.

So my questions are:

Did bindgen generate *mut *mut just to be safe because it can't know the implementation details?
Why does *mut *mut behave differently than void**. Is it because Rust passes by reference already and so you dont need an extra layer of inderection?
Would it be better to rewrite the bindings so that it takes a Rust reference to a usize e.g. PointerTest(&mut usize)?

Thanks for reading, I know it was long. And thanks for any help that you might provide.

quinedot · November 12, 2021, 3:31am

    fn double_pointer_test() {
        let mut x: usize = 0; 
        let mut x_ptr = &mut x as *mut usize as *mut c_void; // cast &mut x to usize ptr (which it is) then cast that to a void *
        let res = unsafe { DoublePointerTest(&mut x_ptr as *mut *mut c_void)}; // cast to *mut *mut x_ptr
        assert_eq!(res, 0, "Result: {:x}", res); 
    }

Maybe I missed something, but if you're supposed to be passing in a pointer to a [pointer == null], why are you passing in a pointer to a [pointer to a stack location]? Shouldn't this be like so?

let mut x_ptr = std::ptr::null::<usize>() as *mut usize as *mut c_void;
let res = unsafe { DoublePointerTest(&mut x_ptr as *mut *mut c_void)};

tgad · November 12, 2021, 3:44am

Wow that works! I hadnt thought of that. I was thinking that 0x0 == NULL. However, in the real API implementation, the pointer could be NULL or the requested memory location. My example is a poor approximation of that.

Hyeonu · November 12, 2021, 4:54am

Direct translation of this code would be:

let mut x: *mut c_void= std::ptr::null_mut();
let res = PointerTest(&mut x);

Assuming unsafe context, of course.

H2CO3 · November 12, 2021, 5:55am

On most platforms, this is indeed true. That is not the problem. The problem is that in your code, the inner level of pointer is not null; it points to a 0-value, but it itself isn't null.

Ethindp · November 12, 2021, 6:02am

[quote="H2CO3, post:5, topic:67225"]

On most platforms, this is indeed true.[/quote]
As a note of caution: you definitely don't want to assume this. The C standard does not specify what NULL is, and the implementation is free to use whatever value it sees fit. This is primarily why C++ came up with nullptr -- it very clearly delineates NULL (the value -- which could be 0 or 0xFFFF or whatever) and what is an actual null pointer.
Edit: fixed formatting (apparently you can't mix markdown and BB code like I'd thought...)
Edit2: Or not... Methinks I found a bug in Discourse. I hit quote whole post and, well, this is the result: a half-quoted post.

H2CO3 · November 12, 2021, 6:06am

I'm very well aware, hence my use of the word "most". I was not trying to advise OP to assume this is true, I was trying to point out the conceptual difference between a pointer that is itself null and a (non-null) pointer which points to something that might as well look like a null pointer in memory.

By the way:

tgad · November 12, 2021, 2:46pm

Hey Everyone,

Thanks for responding! I think we're getting hung up on the NULL thing. I've created a smaller example here: Rust Playground. It looks like creating a pointer using std::ptr::null_mut::() and just casting a reference to a usize as *mut c_void (only *mut c_void and not *mut *mut c_void) behave functionally the same. That's what is confusing me.

quinedot · November 12, 2021, 3:21pm

Normal printing of a reference will print the thing behind the reference, which may be adding to the confusion. Use {:p} to print pointers.

(I'm not sure what your confusion actually is though.)

tgad · November 12, 2021, 7:07pm

In the code at the very top are two C functions. DoublePointerTest and PointerTest. They are exactly the same and both functions take PVOID *, which is void**, as an argument. I generated rust bindings using bindgen and got two rust functions out of it. I changed PointerTest to take (PVOID) instead of (PVOID *). I can call the DoublePointerTest using *mut *mut c_void and it works, but I can also call PointerTest using a *mut c_void and it also works. However, both C functions are expecting void **.

DWORD DoublePointerTest(IN OUT PVOID * Address) {
	if ( *(DWORD *)Address == 0x0 ) {
		*Address = VirtualAlloc(NULL, 1024, MEM_RESERVE|MEM_COMMIT, PAGE_READWRITE);
		if( *Address != NULL ) {
			return 0;
		}
		else {
			return GetLastError();
		}
	} 
	printf("Error, Address is not NULL: %x\n", *(DWORD *)Address);
	return 0x99000099;
}

DWORD PointerTest(IN OUT PVOID * Address) {
	if ( *(DWORD *)Address == 0x0 ) {
		*Address = VirtualAlloc(NULL, 1024, MEM_RESERVE|MEM_COMMIT, PAGE_READWRITE);
		if( *Address != NULL ) {
			return 0;
		}
		else {
			return GetLastError();
		}
	} 
	printf("Error, Address is not NULL: %x\n", *(DWORD *)Address);
	return 0x99000099;
}

    #[test]
    fn double_pointer_test() {
        let mut x_ptr = std::ptr::null_mut::<usize>() as *mut usize as *mut c_void; 
        let res = unsafe { DoublePointerTest(&mut x_ptr as *mut *mut c_void)}; // &mut x_ptr == *mut *mut c_void
        assert_eq!(res, 0, "Result: {:x}", res);
    }

    #[test]
    fn pointer_test() {
        let mut x: usize = 0; 
        let x_ptr = &mut x as *mut usize as *mut c_void;
        let res = unsafe { PointerTest(x_ptr) };
        assert_eq!(res, 0, "Result: {:x}", res); // x_ptr = *mut c_void
    }

This is what is confusing me. Bindgen wants *mut *mut c_void, but I can just send in *mut c_void if its pointing at a variable and it works. In fact, I can even change the Rust definition to

pub fn PointerTest(Address: &mut usize) -> DWORD;

and call it like this

#[test]
    fn pointer_test() {
        let mut x: usize = 0; 
        //let x_ptr = &mut x as *mut usize as *mut c_void;
        let res = unsafe { PointerTest(&mut x) };
        assert_eq!(res, 0, "Result: {:x}", res); // x_ptr = *mut c_void
    }

And that works too. So should I change the definition to take &mut usize instead of PVOID * because I know that I'm supposed to get a pointer back?

so to recap, this C function:

DWORD PointerTest(IN OUT PVOID * Address) {
	if ( *(DWORD *)Address == 0x0 ) {
		*Address = VirtualAlloc(NULL, 1024, MEM_RESERVE|MEM_COMMIT, PAGE_READWRITE);
		if( *Address != NULL ) {
			return 0;
		}
		else {
			return GetLastError();
		}
	} 
	printf("%x\n", *(DWORD *)Address);
	return 0x99000099;
}

Can be called from Rust these ways:

extern "C" {
    pub fn PointerTest(Address: *mut PVOID) -> DWORD; // PVOID = *mut c_void
}
extern "C" {
    pub fn PointerTest(Address: PVOID) -> DWORD;
}
extern "C" {
    pub fn PointerTest(Address: &mut usize) -> DWORD;
}

    #[test]
    fn pointer_test() {
        let mut x_ptr = std::ptr::null_mut::<usize>() as *mut usize as *mut c_void; 
        let res = unsafe { DoublePointerTest(&mut x_ptr as *mut *mut c_void)}; // &mut x_ptr == *mut *mut x_ptr
        assert_eq!(res, 0, "Result: {:x}", res);
    }

    #[test]
    fn pointer_test() {
        let mut x: usize = 0; 
        //let x_ptr = &mut x as *mut usize as *mut c_void;
        let res = unsafe { PointerTest(&mut x) }; // x_ptr = &mut usize
        assert_eq!(res, 0, "Result: {:x}", res); 
    }

    #[test]
    fn pointer_test() {
        let mut x: usize = 0; 
        let x_ptr = &mut x as *mut usize as *mut c_void; // cast &mut x to usize ptr (which it is) then cast that to a void *
        let res = unsafe { PointerTest(x_ptr) }; // x_ptr = *mut c_void
        assert_eq!(res, 0, "Result: {:x}", res); 
    }

quinedot · November 12, 2021, 8:15pm

Best to preserve the semantics and keep *mut *mut c_void. Even ignoring things like provenance, consider that a c_void has alignment of 1 and *mut c_void has the alignment of a pointer.

system · February 10, 2022, 8:16pm

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.

Topic		Replies	Views
Scope for structs and raw pointers for FFI/C library help	4	611	May 10, 2021
How to pass `mut mut ::std::os::raw::c_char` to C function?	12	2404	December 22, 2021
C FFI Questions: rust methods in callbacks help	6	1159	August 20, 2019
Interior mutability and C API help	4	548	January 15, 2022
FFI：return *mut c_void to c++ project help	9	963	February 28, 2022

Rust FFI, *mut *mut and C

Related topics

Rust FFI, mut mut and C