Is there a way to get all concrete implementations of Traits/generic functions into a compiled library? (So I can access them dynamically at run-time)

[ANombre]

I’m currently implementing a toy language and library in Rust, dealing with data flows of lists and matrices. Still figuring out what the best way to implement this, but I’m currently thinking of generating all relevant functions up front and then picking the right one at runtime.

Say I have some data types: {f32, Vec2, Vec3 etc.} (where the Vec2/Vec3 are just [f32; n], like those in Glam). I’m actually using Glam, so I have the correct implementations of Add/Mult etc. at my disposal. I can do f32 + Vec3 -> Vec3, Vec3 + Vec3 -> Vec3 etc.

Now I'm making a generic function that will accept slices of them, so that if I can do A+B=C I can do Multi<A> + Multi<B> = Multi<C> by doing c[i] = a[i] + b[i] (and later on combinations of this, such as Multi<A> + B = Multi<C>):

pub fn add_inplace<A, B>(a: &[A], b: &[B], c: &mut [<A as Add<B>>::Output])
    where 
    A: Copy, 
    B: Copy, 
    A: Add<B>, 
{ 
    assert_eq!(a.len(), b.len());
    for i in 0..a.len() {
        c[i] = a[i] + b[i];
    }
}

But this function is generic, meaning that it doesn't actually exist yet, although rustc knows how to generate the code for it. Rust generates the concrete implementation whenever I call the function in my code somewhere (the so-called monomorphization).

Now my use-case is a bit special. At runtime, I want to lookup the actual function that I want to call and call it using some unsafe trickery (comparable to a dylib). This means that I need Rust to somehow generate concrete versions of all possible implementations of this function:

// Obviously pseudo-code
let mut functions_library = HashMap::new(); 
for Type1 in <f32, Vec2, Vec3> { // Iterate over all types
    for Type2 in <f32, Vec2, Vec3> { 
        let key = ("+", Type1, Type2); // Lookup consists of the function name and the input types 
        // let return_type = add_inplace<Type1, Type2>::ReturnType // question 2: does something like this exist?? 
        let func_ptr = add_inplace<Type1, Type2>; // The function ptr, e.g. add_inplace<Vec3, f32> 
        functions_library.insert(key, func_ptr as *const c_void); 
    } 
}

The goal being a function pointer to the monomorphized function stored in a hashmap. Are there ways to use Rust (or perhaps rustc directly) as a code generator in this fashion, so that all possible instantiations end up compiled and retrievable in some way? Do you know of any blog posts or crates that do something like this?

Second question, is there a way to get the return type of an generic function, so something like add_inplace<Type1, Type2>::ReturnType?

[kpreid]

No matter what, you will need some listing of the cases that you want to include, such that the compiler sees a need to monomorphize it. But this is possible without anything esoteric. Proof of concept:

use std::ffi::c_void;

/// Placeholder trait for example
trait Foo<T> {
    fn do_thing();
}
impl<T, U> Foo<T> for U {
    fn do_thing() {
        // some code to make each function actually different
        dbg!(std::any::type_name::<(T, U)>());
    }
}

type Entry = (&'static str, *const c_void);

fn main() {
    let mut library = Vec::new();
    generate_library_1(&mut library);
    println!("{library:#?}");
}

fn generate_library_1(library: &mut Vec<Entry>) {
    // This could be macro-generated code
    generate_library_2::<f32>(library);
    generate_library_2::<[f32; 2]>(library);
    generate_library_2::<[f32; 3]>(library);
}
fn generate_library_2<T>(library: &mut Vec<Entry>) {
    generate_library_3::<T, f32>(library);
    generate_library_3::<T, [f32; 2]>(library);
    generate_library_3::<T, [f32; 3]>(library);
}
fn generate_library_3<T, U>(library: &mut Vec<Entry>) {
    library.push((
        std::any::type_name::<(T, U)>(),
        <U as Foo<T>>::do_thing as *const c_void,
    ));
}

Second question, is there a way to get the return type of an generic function, so something like add_inplace<Type1, Type2>::ReturnType ?

Write the function that wants the information to be generic over an additional type parameter for the return value:

fn uses_return_type<A, B, R, F>(_: F) -> &'static str
where
    F: Fn(A, B) -> R,
{
    std::any::type_name::<R>()
}

fn main() {
    dbg!(uses_return_type(<String as std::ops::Add<&str>>::add));
}

It's also sometimes possible to write an extension trait that exposes an associated type for the return value, but that sometimes makes the compiler sad.

[ANombre]

Ah thank you very much. I was already afraid that I was gonna have to go the macro route, but was hoping there might be some obscure possibility (perhaps using nightly). Not really sure how you'd use an extension trait, perhaps you have a link of that? Still, your reply gave me a clean solution for the second problem, which also turns out to be more general. But this snippet actually works, and that is... uhh pretty cool I guess?

pub fn create_function_generic<A: 'static, B: 'static, R: 'static>(
    function_name: &'static str,
    library: &mut HashMap<(&'static str, [TypeId; 2]), (TypeId, *const c_void)>,
    func: fn(A, B) -> R,
) where
    A: Copy,
    B: Copy,
    R: Copy,
{
    let (name, input_types, return_type, function_ptr) = (
        function_name,
        [TypeId::of::<A>(), TypeId::of::<B>()],
        TypeId::of::<C>(),
        func as *mut c_void,
    );

    library.insert((name, input_types), (return_type, function_ptr));
}

// Actually instantiate all the functions (in this case a+b) and add them to the "library"
create_function_generic::<f32, f32, _>("+", &mut functions, std::ops::Add::add);
create_function_generic::<f32, Vec4, _>("+", &mut functions, std::ops::Add::add);
create_function_generic::<Vec4, f32, _>("+", &mut functions, std::ops::Add::add);
create_function_generic::<Vec4, Vec4, _>("+", &mut functions, std::ops::Add::add);

[ANombre]

Also, do you know of a way to further improve it by solving this:

Once I have a function of fn(A, B) -> R I want to generate 4 functions out of it, so that I can add Single<T>/Multi<T> + Single<T>/Multi<T> and write it to a third buffer:

fn (&A, &B, &mut R);       // Single single assign
fn (&[A], &B, &mut [R]);   // Multi single assign
fn (&A, &[B], &mut [R]);   // Single Multi assign
fn (&[A], &[B], &mut [R]); // Multi Multi assign

I'm trying something like this (for the Multi<T> + Multi<T> option):

fn apply_multi_multi<A: 'static, B: 'static, R: 'static>(
    func: fn(A, B) -> R,
) -> fn(&[A], &[B], &mut [R])
where
    A: Copy,
    B: Copy,
    R: Copy,
{
    |a: &[A], b: &[B], c: &mut [R]| {
        for i in 0..a.len() {
            c[i] = func(a[i], b[i]);
        }
    }
}

Of course this doesn't work because the returned closure captures func and can therefor not be coerced into a function pointer, but I feel this idea should somehow be possible as func is a function that is known at compile time? Even something like this doesn't work fn apply_multi_multi<A: 'static, B: 'static, R: 'static, F: Fn(A, B) -> R>(func: F), while I would say that F is a generic type there and that should be possible.

[kpreid]

In principle it is possible to compose functions that way and have the compiler generate code and thus function pointers for them, but in current Rust it cannot be done as a higher-order function. Perhaps someday const-computed function compositions could support this — but first we'll need support for higher-order functions in const fn at all.

You will have to use macros to generate such functions, instead.