Creating a struct from a raw byte buffer given by FFI

help
1

I'm staring some system programming in Rust and am playing with the windows create.
As a simple starter solution I want to get some system information via the NtQuerySystemInformation function call.

To get the buffer I use the following function:

fn query_system_information_buffer(information_class: SYSTEM_INFORMATION_CLASS) -> Option<Vec<u8>> {
    let mut size: u32 = 1 << 20; //  1 MB
    let max_size: u32 = 1 << 24; // 16 MB

    let mut buffer = vec![0u8; size as usize];

    let mut status: windows::core::Result<()>;
    loop {
        let mut return_length: u32 = 0;

        status = unsafe {
            NtQuerySystemInformation(
                information_class,
                buffer.as_mut_ptr() as *mut c_void,
                size,
                &mut return_length,
            )
        };

        // In case the system information returned Ok, then we can safely beark from the loop and transform the byte
        // buffer into the required structure. If we have an error, we null the buffer (safety!) and resize it to the
        // desired new size.
        match &status {
            Ok(_) => break,
            Err(e) => {
                let h_status = e.code();

                if h_status == STATUS_INFO_LENGTH_MISMATCH.to_hresult() {
                    size = if return_length > size {
                        return_length
                    } else {
                        size * 2
                    };

                    buffer.fill(0);
                    buffer.resize(size as usize, 0);
                }
            }
        };

        // If the resized buffer exceeds the maximum size, we just break from the loop, since we can't get anything
        // useful anymore.
        if size >= max_size {
            break;
        }
    }

    // If the status returned an error, we simply return None (Rust's memory system will deal with the buffer), else we
    // transform the buffer into the info struct and return that to the caller.
    if status.is_err() {
        None
    } else {
        Some(buffer)
    }
}

This all works fine and well, but as soon as I try to convert the byte buffer into a struct (called SYSTEM_HANDLE_INFORMATION_EX) I run into problems. As far as I can tell, the C struct definition uses the C89 struct hack, at least in the definitions I could find online.

My first approach was to recreate the struct verbatim like

use std::ffi::{c_ulong, c_ushort, c_void};

#[derive(Debug)]
#[repr(C)]
struct SYSTEM_HANDLE_INFORMATION_EX {
    pub num_handles: usize,
    reserved: *const c_ulong,
    pub handles: [SYSTEM_HANDLE_TABLE_ENTRY_INFO_EX ; 1],
}

#[derive(Debug)]
#[repr(C)]
struct SYSTEM_HANDLE_TABLE_ENTRY_INFO_EX {
    pub object: *const c_void,
    pub unique_process_id: *const c_ulong,
    pub handle_value: *const c_void,
    pub granted_access: c_ulong,
    pub creator_back_trace_index: c_ushort,
    pub object_type_index: c_ushort,
    pub handle_attributes: c_ulong,
    reserved: c_ulong,
}

fn query_system_information(information_class: SYSTEM_INFORMATION_CLASS) -> Option<SYSTEM_HANDLE_INFORMATION_EX> {
    let buffer = query_system_information_buffer(information_class);
    let info: SYSTEM_HANDLE_INFORMATION_EX = unsafe { 
        std::ptr::read(buffer.as_ptr() as *const _) 
    };

    Some(info)
}

However, since I want to loop over the handles later on, this compiles, but obviously won't work (and on top of that is UB).
Then I read around a bit and possible solutions pointed me to replace the handles field with a slice, i.e.

#[derive(Debug)]
#[repr(C)]
struct SYSTEM_HANDLE_INFORMATION_EX {
    pub num_handles: usize,
    reserved: *const c_ulong,
    pub handles: [SYSTEM_HANDLE_TABLE_ENTRY_INFO_EX],
}

But now the size is unknown and it won't compile. More searching led me to THIS thread here.

I understand the concept behind it, but somehow I cannot get it to run on my side. And pointers (heh!) to what I'm doing wrong get me going?

2

You can use the [ElementType; 1] trick, as long as you only access the struct as a pointer (not as a reference; a reference "shrinks" the allowed access to the bounds of the struct). To illustrate, we can use pointer arithmetic to construct a safe slice of table entries out of the buffer:

#![allow(nonstandard_style)]

use std::{
    ffi::c_void,
    mem,
    os::raw::{c_ulong, c_ushort},
    ptr::addr_of,
    slice,
};
use windows::Win32::System::WindowsProgramming::{
    NtQuerySystemInformation, SYSTEM_INFORMATION_CLASS,
};

// signatures adapted from https://www.geoffchappell.com/

const SystemExtendedHandleInformation: SYSTEM_INFORMATION_CLASS = SYSTEM_INFORMATION_CLASS(0x40);

struct SYSTEM_HANDLE_INFORMATION_EX {
    NumberOfHandles: usize,
    _Reserved: usize,
    Handles: [SYSTEM_HANDLE_TABLE_ENTRY_INFO_EX; 1],
}

struct SYSTEM_HANDLE_TABLE_ENTRY_INFO_EX {
    Object: *mut c_void,
    UniqueProcessId: usize,
    HandleValue: usize,
    GrantedAccess: c_ulong,
    CreatorBackTraceIndex: c_ushort,
    ObjectTypeIndex: c_ushort,
    HandleAttributes: c_ulong,
    _Reserved: c_ulong,
}

fn query_system_information_buffer(information_class: SYSTEM_INFORMATION_CLASS) -> Option<Vec<u8>> {
    let mut buffer: Vec<u8> = vec![];
    loop {
        let mut return_length = 0;
        let result = unsafe {
            NtQuerySystemInformation(
                information_class,
                buffer.as_mut_ptr().cast(),
                buffer.len() as c_ulong,
                &mut return_length,
            )
        };
        let return_length = return_length as usize;
        match result {
            Ok(()) => return Some(buffer),
            Err(_) if return_length > buffer.len() => {
                buffer.clear();
                buffer.reserve_exact(return_length);
                buffer.resize(return_length, 0);
            }
            Err(_) => return None,
        }
    }
}

// the `repr(packed)` is necessary, the methods are just for convenience
#[repr(packed)]
struct HandleEntry(SYSTEM_HANDLE_TABLE_ENTRY_INFO_EX);

impl HandleEntry {
    fn object(&self) -> *mut c_void {
        self.0.Object
    }
    fn unique_process_id(&self) -> usize {
        self.0.UniqueProcessId
    }
    fn handle_value(&self) -> usize {
        self.0.HandleValue
    }
    fn granted_access(&self) -> c_ulong {
        self.0.GrantedAccess
    }
    fn creator_back_trace_index(&self) -> c_ushort {
        self.0.CreatorBackTraceIndex
    }
    fn object_type_index(&self) -> c_ushort {
        self.0.ObjectTypeIndex
    }
    fn handle_attributes(&self) -> c_ulong {
        self.0.HandleAttributes
    }
}

fn get_handle_table(buffer: &[u8]) -> &[HandleEntry] {
    let info_ptr: *const SYSTEM_HANDLE_INFORMATION_EX = buffer.as_ptr().cast();
    // get a pointer to `Handles`, and cast it to `*const HandleEntry`
    let table_ptr: *const HandleEntry = unsafe { addr_of!((*info_ptr).Handles).cast() };
    // read the length as an unaligned `usize`
    let len = unsafe { addr_of!((*info_ptr).NumberOfHandles).read_unaligned() };
    // ensure that the table fits within `buffer`
    let table_offset = unsafe { table_ptr.cast::<u8>().offset_from(buffer.as_ptr()) };
    let max_len = (buffer.len() - table_offset as usize) / mem::size_of::<HandleEntry>();
    assert!(len <= max_len);
    // create a `&[HandleEntry]` from the pointer and length
    unsafe { slice::from_raw_parts(table_ptr, len) }
}

fn main() {
    let buffer = query_system_information_buffer(SystemExtendedHandleInformation).unwrap();
    let table = get_handle_table(&buffer);
    for (i, entry) in table.iter().enumerate() {
        println!("entry {i}:");
        println!("  Object = {:p}", entry.object());
        println!("  UniqueProcessId = {}", entry.unique_process_id());
        println!("  HandleValue = {}", entry.handle_value());
        println!("  GrantedAccess = {:#x}", entry.granted_access());
        println!(
            "  CreatorBackTraceIndex = {}",
            entry.creator_back_trace_index()
        );
        println!("  ObjectTypeIndex = {}", entry.object_type_index());
        println!("  HandleAttributes = {:#x}", entry.handle_attributes());
    }
}

(As a side note, it's possible to avoid the repr(packed) silliness, but only if the alignment of the buffer is increased to the alignment of the struct. To do this without copying would require storing the buffer as a Vec<usize>, or as a custom type altogether.)

3

Shouldn't this be undefined behaviour? I man, sure the [i] access is just a nice wrapper around a pointer dereference with some arithmetic, but as far as I could find it's a common "hack", but at least in C with strict standard interpreting, yet still UB.

So I'd just create a new Vec<usize>/Newtype, that has the size of my struct and then fill it with the buffer?
Also: What's this "repr(packed) silliness" you're talking about. I only found that packed structs can be dangerous and cause massive headaches. Any reason to use it over #[repr(C)] here?

4

Rust has different rules for UB than C/C++ have. In get_handle_table(), buffer.as_ptr() creates a *const u8 that can be used to read any data in buffer. The addr_of!((*info_ptr).Handles) creates a pointer into buffer that can still be used to read any data in buffer. If we were to write &(*info_ptr).Handles instead, then it would be UB to read past the first element. (Also, now that I've thought about it, it may be better to write Handles: [SYSTEM_HANDLE_TABLE_ENTRY_INFO_EX; 0] instead. Otherwise, it would be UB to perform the pointer-to-field projection, if the table has no elements.)

So I'd just create a new Vec<usize>/Newtype, that has the size of my struct and then fill it with the buffer?

Yes, that would be sufficient to align the data, so that the HandleEntry and read_unaligned() become unnecessary.

Also: What's this "repr(packed) silliness" you're talking about. I only found that packed structs can be dangerous and cause massive headaches. Any reason to use it over #[repr(C)] here?

Without copying into an aligned buffer, the data will be unaligned, and it would be UB to access the fields directly from the struct. Therefore, either the read_unaligned()/write_aligned() pointer methods must be used, or the fields must be accessed via a #[repr(packed)] struct, which has no alignment requirements.

5

You might want to check how it is done in this article. In short, one can:

  • make an (unsized) repr(C) struct with [u8] as the last field;
  • create a wrapper, which:
    • holds internally a Vec,
    • is filled via passing vec.as_mut_ptr() to FFI,
    • and then Derefs to the struct by casting the reference.
6

The header type (SYSTEM_HANDLE_INFORMATION_EX) doesn't contain anything except for the length of the table. So I doubt that using a custom DST here would be preferable to just generating a &[PackedWrapper<SYSTEM_HANDLE_TABLE_ENTRY_INFO_EX>].

7

As a side note, you can use https://crates.io/crates/bytemuck to make it a lot easier and safer to parse structs like this: it will handle checking you have no padding, unaligned values, etc.

8

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.