How do you allocate an `fn` from scratch?

1

Context: I'm aware the whole "unsafe" portion of the language exists precisely to avoid all the issues that come into existence the moment one decides to venture into this level of detail and manually manage all the underlying size/alignment/lifetimes involved. To me, this is a learning exercise.

After dabbling in and out of the standard safety guarantees I came into a much closer contact with zero-guarantees, hands-on-memory, nail-or-crash type of code, such as:

fn main() {
    let array: [u16; 1] = [1];
    let tuple = array.cast::<(bool, bool)>();
    println!("{:?}", tuple);
}

trait Castable<T: Copy> {
    fn cast<X: Copy>(&self) -> X; 
}

impl<T: Copy> Castable<T> for T {
    fn cast<X: Copy>(&self) -> X {
        unsafe { *(self as *const T as *const X) }
    }
}

Without (significant) prior experience with C/C++ and their static/const/reinterpret_cast - as well as virtually complete and utter (though, in retrospect, somewhat willfully blind) trust in "all unsafe is bad" - something I might have taken on board long before realizing what unsafe ops actually are - the code above "clicked" in a somewhat better/deeper level of understanding.

The underlying memory management mechanisms for the regular variables seem rather straightforward enough, so far. Yet I'm struggling quite a bit to wrap my head around the way functions are represented in it. Reading about stdcall and the rest of calling conventions seems to have raised even more questions. Initialized registers? Stack/return pointers? Utter confusion.

Can someone write up the most basic example of a fn, declared completely from scratch - which does absolutely nothing, as in fn test() {} - yet allocated from ground up in pure lower level tuples / arrays / structs, showing what's going on beneath it all, when it comes to memory?

Or would that involved a great deal of platform-specific asm declarations as well?

2

This is a roundabout way of implementing std::mem::transmute_copy. Use that instead.

There are different levels on which this can be answered. Your hypothetical test, per se, is primarily something that implements the Fn trait. If you step into unstable features, you can just do that:

#![feature(unboxed_closures)]
#![feature(fn_traits)]

struct test;

impl std::ops::Fn<()> for test {
    extern "rust-call" fn call(&self, _args: ()) -> Self::Output {
        // nothing
    }
}
impl std::ops::FnMut<()> for test {
    extern "rust-call" fn call_mut(&mut self, _args: ()) -> Self::Output {
        // nothing
    }
}
impl std::ops::FnOnce<()> for test {
    type Output = ();
    extern "rust-call" fn call_once(self, _args: ()) -> Self::Output {
        // nothing
    }
}

But, of course, that says nothing about the actual implementation of function code, since it’s just one call turning into another. If you coerce test to a function pointer type (fn()), which is something only actual function items can do, then you have something “closer to the metal”: a function pointer is simply the address of some machine code. Nothing more.

If you wanted to generate a function from scratch (say, as part of a JIT compiler), you would:

  1. Allocate some memory.
  2. Write machine code in that memory that obeys the platform’s calling convention.
  3. Call the operating system to mark the page containing that memory as executable and not writable. (If you don't do this, you'll get a segfault when you try to use the pointer. This is protection against accidental or malicious memory corruption. We're somewhat defeating it by doing run-time code generation at all.)
  4. Cast the regular pointer to a function pointer. (On some platforms, this might be entirely prohibited.)

Then, when the function pointer is used, the compiler simply generates code that jumps to that address (plus whatever preliminary steps the calling convention requires).

Or would that involved a great deal of platform-specific asm declarations as well?

So, yes and no. No, you don't need to write any asm! to do this. But you do need to know the platform’s machine language and calling convention.

The big-picture thing to understand is that there is no more structure than the machine instructions and calling convention. There is no “data structure” that is a function, beyond the code itself.

5 Likes
3

Much appreciated. Last few clarification points:

Which would have to be done with a call to the mprotect of libc - as there is no alternative to it in the std itself, correct? Or via crates such as region-rs?

Prohibited by the compiler, or the OS itself? Why would only a select few allow or prohibit this?

4

it's not clear what you mean by "allocate". can you clarify more?

each function in rust has a unique type, which can be coerced to a function pointer type of the same signature. but neither functions nor function pointers need to be "allocated".

1 Like
5

Prohibited by the hardware. Not every processor treats code and data the same. On some devices, code is kept in special-purpose read-only memory, and the processor can only execute that memory.

1 Like
6

On systems/processes where forward-edge control flow integrity mechanisms (e.g., Control Flow Guard, Intel CET, Arm pointer authentication) are enabled, you additionally need to mark runtime-defined functions as valid branch targets.

1 Like
7

Either, or both, or neither. For example, on a Harvard architecture, it may be completely meaningless (eg. on AVR microcontrollers, no internal software can write the program memory, only an external piece of hardware can, therefore it's impossible to dynamically generate executable code).

8

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.