Can I build a function name from arguments to a macro_rules!

I have the same function, written in various different ways that I was testing and bench marking. Each of those functions lives in it's own module that is named after it's author.

So I ended up writing the same test and the same bench many times, one for each version of the function.

Today I thought it would be a nice exercise to learn how to write macro_rules! macros generate those tests instead. This what I came up with:

    macro_rules! make_test {
        ($test_name:ident, $mod_name:ident, $fn_name:ident) => {
            #[test]
            fn $test_name () {
                assert_eq!($mod_name::$fn_name(&SAMPLE, &COEFF), EXPECTED);
            }
        };
    }

    make_test!(test_zicog_convolution, zicog, convolution);
    make_test!(test_zicog_convolution_safe, zicog, convolution_safe);
    make_test!(test_zicog_convolution_fast, zicog, convolution_fast);
    make_test!(test_alice_convolution_serial, alice, convolution_serial);
    make_test!(test_alice_convolution_parallel, alice, convolution_parallel);
    ...
    ...

This works fine. But I thought it would be nice to get rid of that $test_name argument and build it out of concatenating the $mode_mane and $func_name arguments.

As you see this is required because some modules/authors have more versions of the same function with slightly different names.

After a couple of ours of scouring docs, googling around and experimenting I have not managed find a way to do this.

1 Like

As far as I understand, the hygeine rules for macro_rules! prevents this, but it is possible with a proc macro. The paste crate might be able to help you here, but I haven’t used it myself: https://docs.rs/paste/0.1.18/paste/

OK Thanks. I'll take that as a no then.

I don't think I am going anywhere near proc macros until I have a couple of years of free time or someone writes a proc macros for dummies book.

That is all far to meta for me.

Can we use the C pre-processor with Rust?

:slight_smile: Joking, honest, only joking :slight_smile:

You don't need to write a proc macro for this, you can just use paste. With paste, you could do the macro_rules! macro roughly (untested) as

macro_rules! make_test {
    ($mod_name:ident, $fn_name:ident) => {
        paste::item! {
            #[test]
            fn [< test_ $mod_name _ $fn_name >] () {
                assert_eq!($mod_name::$fn_name(&SAMPLE, &COEFF), EXPECTED);
            }
        }
    };
}

paste basically adds [< $($ident)* >] syntax for pasting multiple identifier fragments into one identifier. The one tricky bit is the split between paste::item! for item position and paste::expr! for expression position, due to different requirements for macro expansion in those positions.

This also produces a new hygenic identifier, so as usual with macro_rules!, you can't call it from outside the macro invocation unless you pass the name in rather than constructing it. However, this isn't problematic for #[test] functions, as the test runner discovers them, you don't call them directly.

4 Likes

Wow, awesome, worked first time!

I now also have:

    macro_rules! make_bench {
        // Arguments are module name and function name of function to test bench
        ($mod_name:ident, $fn_name:ident) => {
            // The macro will expand into the contents of this block.
            paste::item! {
                #[bench]
                fn [< bench_ $mod_name _ $fn_name >] (b: &mut Bencher) {
                    let sample: Vec<f32> = vec![0.33333; 1024];
                    let coeff: Vec<f32> = vec![0.33333; 16];
                    b.iter(|| $mod_name::$fn_name(&sample, &coeff))
                }
            }
        };
    }

I'm a bit disturbed though. That is pretty inscrutable, certainly not easy on the eyes.

Is there anyway to at least create $test_name using paste and then use "fn $test_name (..." ?

That would at least bring some high level language comfort to what otherwise looks like line noise.

FWIW, there does not seem to be any hygiene when dealing with "global" items such as a functions, modules, types, macros, consts and statics, etc.. It's rather the lack of a built-in operator to perform the concatenation (but what paste! enables back), or the fact that macros can only expand to items, expressions, types and statements, and nothing finer-grained such as "the name of a new function".


Shameless plug: ::paste depends on the long to compile ::syn crate, so if your project is not depending on syn already, and if you are sensitive about the time it takes to compile your crate from scratch, then you can use ::mini_paste, which features the same concat-related functionality but without using syn nor quote.

  • EDIT: Nevermind, depending on paste does not make you pull syn, got that wrong.

The constraints mentioned above lead to it not really being possible to define new meta-level variables unless nested macros are used, which I would say lead to even worse ergonomics.

Luckily for you, we don't have to speculate about it: while researching for a syn-free alternative to paste!, I had come up with the following experimental design:

//! ```toml
//! [dependencies.with_concat_ident]
//! git = "https://github.com/danielhenrymantilla/with-concat-idents"
//! rev = "d8c15957ed74bf4a61f37da241737343fc86840e"
//! ```

#[macro_use]
extern crate with_concat_ident;

macro_rules! make_bench {(
    $mod_name:tt, $fn_name:tt $(,)?
) => (
    // nested helper macro
    macro_rules! helper {(
        $bench_fn_name:ident
    ) => (
        #[bench]
        fn $concat_ident (b: &mut Bencher)
        {
            let sample: Vec<f32> = vec![0.33333; 1024];
            let coeff: Vec<f32> = vec![0.33333; 16];
            b.iter(|| $mod_name::$fn_name(&sample, &coeff));
        }
    )}
    with_concat_ident! {
        // $bench_fn_name := concat!(bench _ $mod_name _ $fn_name)
        concat!(bench _ $mod_name _ $fn_name) => helper!
    }
)}
  • (Such crate is very experimental right now, do not use it in production).

  • That being said, if you feel like this pattern is not that bad, especially knowing that I think it should be possible to avoid having to provide helper! explicitely, thus allowing the following syntax:

    macro_rules! make_bench {(
        $mod_name:tt, $fn_name:tt $(,)?
    ) => (
        with_concat_idents! {
            concat_idents!(bench _ $mod_name _ $fn_name), |$bench_fn_name| {
            // or something like:
            // $bench_fn_name := concat_idents!(bench _ $mod_name _ $fn_name) in
                #[bench]
                fn $bench_fn_name (b: &mut Bencher)
                {
                    let sample: Vec<f32> = vec![0.33333; 1024];
                    let coeff: Vec<f32> = vec![0.33333; 16];
                    b.iter(|| $mod_name::$fn_name(&sample, &coeff));
                }
            }
        }
    )}
    

    Then know that I could keep tinkering on that crate to polish it (e.g., no more typo in the name of the crate :sweat_smile:), make it support that syntax and then publish it (I am personally used to paste syntax by now, so I know I'd rather use paste instead, so I personally don't see any reason to further develop this experiment unless other people feel otherwise).

3 Likes

That's an interesting experiment.

At first sight it seems as unreadable as the paste syntax. So I might agree and just get on with paste/mini_paste.

Paste does not depend on syn. It has dev-dependencies that use syn but you would only build those when running the test suite of the paste crate, not when using it as a library.

1 Like

There is a different crate which works more like that; see https://github.com/dtolnay/mashup.

#[macro_use]
extern crate mashup;

macro_rules! make_bench {
    ($mod_name:ident, $fn_name:ident) => {
        mashup! {
            m["bench" $mod_name $fn_name] = bench_ $mod_name _ $fn_name;
        }
        m! {
            #[bench]
            fn "bench" $mod_name $fn_name(b: &mut Bencher) {
                let sample: Vec<f32> = vec![0.33333; 1024];
                let coeff: Vec<f32> = vec![0.33333; 16];
                b.iter(|| $mod_name::$fn_name(&sample, &coeff))
            }
        }
    };
}
1 Like

Oh, I always thought proc-macro-hack did, my bad!

I kind of like the mashup approach. It separates the name mashing part from the actual function definition part nicely.

I thought proc macro hack used syn and quote? I recall making the decision to use both of those because they were already in my dependencies tree.

Only for dev dependencies.

I've never messed with macros before, but what about changing the macro to avoid the need for a new unique function name?

it worked in a simple test I did.

macro_rules! make_test {
    ($mod_name:ident, $fn_name:ident) => {
        mod testing {
            mod $mod_name {
                use super::super::*;
                #[test]
                fn $fn_name() {
                    assert_eq!($mod_name::$fn_name(&SAMPLE, &COEFF), EXPECTED);
                }
            }
        }
    };
}

Edit: I see where this goes wrong...nevermind.

Where does it go wrong?

Macros give me headache.

Duplicate module definition. It works fine as long as you only use it once!

Ah yes.

So let’s only use it once:

    macro_rules! make_test_mod {
        ( $( $mod_name:ident { $( $fn_name:ident ),* } )* ) => {
            #[cfg(test)]
            mod testing { $(
                mod $mod_name {
                    use super::super::*;
                    $(
                        #[test]
                        fn $fn_name () {
                            assert_eq!($mod_name::$fn_name(&SAMPLE, &COEFF), EXPECTED);
                        }
                    )*
                }
            )*}
        };
    }

    make_test_mod!{
        zicog { convolution, convolution_safe, convolution_fast }
        alice { convolution_serial, convolution_parallel }
    }
1 Like

OK, that's pretty smart.

But let's stop this now, it's getting out of hand. At some point the solution imposes more complexity than the problem one started with. I only wanted to concatenate two strings!

:slight_smile:

I had a feeling there would be a way to do it.

Like I said, I've never messed with macros before; but they are starting to look more interesting.