Invoke macro to create identifier for constant (inconsistent behavior between let/const in macro)

help
1

I am writing a macro_rules macro, and I want users to be able to provide a custom way to "format" the identifiers I create.

I came across this Stack Overflow answer that pointed out that you could pass in the name of a macro into a macro, and have it evaluate that macro. I figured this was exactly what I needed.

macro_rules! create_fn_let {
    ($create_ident:ident) => {
        fn created_fn_let() {
            let $create_ident!() = 1;
            let $create_ident!(): u8 = 1;
        }
    };
}

macro_rules! create_foo_ident {
    () => {foo};
}

create_fn_let!(create_foo_ident);

This successfully expands into:

fn created_fn_let() {
    let foo = 1;
    let foo: u8 = 1;
}

However, if I try to do the same thing with const instead of let, it doesn't work!

macro_rules! create_fn_const {
    ($create_ident:ident) => {
        fn created_fn_const() {
            const $create_ident!(): u8 = 1;
        }
    };
}

macro_rules! create_foo_ident {
    () => {foo};
}

create_fn_const!(create_foo_ident);

This produces the following error:

error: expected one of `:`, `;`, `<`, `=`, or `where`, found `!`
  --> src/macros.rs:71:32
   |
71 |             const $create_ident!(): u8 = 1;
   |                                ^ expected one of `:`, `;`, `<`, `=`, or `where`
...
80 | create_fn_const!(foo; create_foo_ident);
   | --------------------------------------- in this macro invocation
   |
   = note: this error originates in the macro `create_fn_const` (in Nightly builds, run with -Z macro-backtrace for more inf>

...snip...

error: missing type for `const` item
  --> src/macros.rs:71:32
   |
71 |             const $create_ident!(): u8 = 1;
   |                                ^
...
80 | create_fn_const!(foo; create_foo_ident);
   | --------------------------------------- in this macro invocation
   |
   = note: this error originates in the macro `create_fn_const` (in Nightly builds, run with -Z macro-backtrace for more inf>
help: provide a type for the item
   |
71 |             const $create_ident: <type>!(): u8 = 1;
   |                                ++++++++

Is there any reason that expanding a macro into a const identifier should work differently than into a let identifier? Is there some way to allow users to pass in a custom "formatting" macro while invoking my macro that I can use to define a const?

2

I also tried evaluating the "formatting" macro and passing the result to a "helper" macro, but that didn't work either.

macro_rules! create_fn_const {
    ($create_ident:ident) => {
        create_fn_const_helper!($create_ident!());
    };
}

macro_rules! create_fn_const_helper {
    ($ident:ident) => {
        fn created_fn_const() {
            const $ident: u8 = 1;
        }
    };
}

macro_rules! create_foo_ident {
    () => {foo};
}

create_fn_const!(create_foo_ident);

(by the way, the real formatting function would take a parameter and presumably use the paste crate, not just provide a constant identifier)

3

Yes, for one, these are expected to work differently w.r.t. hygiene. let identifiers (and other local variables, e.g. function parameters, values introduced in other pattern-matching constructs such as match) offer hygiene in macros, whereas “top-level” items, such as modules, types, traits, and also static variables and const items, don’t.

The other difference that you’re running into here is with the syntax. A const item syntactically

ConstantItem :
   const ( IDENTIFIER | _ ) : Type ( = Expression )? ;

supports only _ and identifiers, whereas let statements

LetStatement :
   OuterAttribute * let PatternNoTopAlt ( : Type )? (= Expression † ( else BlockExpression)? )? ;

† When an else block is specified, the Expression must not be a LazyBooleanExpression, or end with a }.

support arbitrary patterns syntactically (with the PatternNoTopAlt nonterminal) – which is what includes the possibility for macro invocations.

Something you could try to avoid this issue is: structuring your macro in some callback style.

In your (presumably oversimplified) example, this could look like

macro_rules! create_fn_const {
    ($create_ident:ident) => {
        $create_ident!(create_fn_const_helper);
    };
}

macro_rules! create_fn_const_helper {
    ($ident:ident) => {
        fn created_fn_const() {
            const $ident: u8 = 1;
        }
    };
}

macro_rules! create_foo_ident {
    ($callback:ident) => { $callback!{foo} };
}

create_fn_const!(create_foo_ident);

Especially given you already mention paste, note that in my experience that crate generally is designed to replace/avoid the need of such – harder to use – constructions in the first place; it’s hard to give more concrete help on how to do that though, without seeing the more concrete / actual use-case you’re working with :wink:

4

Thanks for the thorough response!

Oh yes, of course! I had even read that earlier in my investigation but it didn't occur to me when I was having this issue.

So now I have a little more background for why const would be treated differently than let. Now I need to figure out how to actually get this to work with const.

This is a little closer to what I'm actually doing (hopefully we're not running into an X/Y problem):

macro_rules! rename_constants {
    ($rename_macro:ident; $($tail:tt)*) => {
        rename_constants_helper!(0; $rename_macro; $($tail)*);
    };
}

macro_rules! rename_constants_helper {
    ($state:expr; $rename_macro:ident;) => {
        // Done
    };
    ($state:expr; $rename_macro:ident; $vis:vis const $name:ident : $ty:ty = $value:expr; $($tail:tt)*) => {
        // This is what I want to do, but I can't invoke $rename_macro from within paste!
        paste! {
            $vis const $rename_macro!($ident) : $ty = $value;
        }
        rename_constants_helper!($state + 1; $suffix; $($tail)*);
    };
}

macro_rules! custom_renamer {
    ($ident:ident) => {
        paste! {
            [<CUSTOM_BEGIN_ $ident _CUSTOM_END>]
        }
    };
}

rename_constants! {
    custom_renamer;
    pub const X1: u8 = 1;
    pub const X2: u8 = 3;
    pub const X3: u8 = 8;
}

I can't figure out how to adjust this to use a "callback" approach that doesn't require the user to do all of the work to define a constant in their own "callback" macro (which would also require their macro to take in at least 4 arguments).

My first approach was to use the output of the paste! macro to return an identifier and use that to create a const declaration, but that ran into the problem that const identifiers can't be the output of macro invocations.

My second approach (in this post) was to invoke paste with the output of the user's "formatting" macro, but that doesn't work due to how callbacks work:

Due to the order that macros are expanded in, it is (as of Rust 1.2) impossible to pass information to a macro from the expansion of another macro:

from this

So if the output of paste! can't be used for a const identifier, and I can't use the output of a user's custom macro inside a paste! invocation, how do I do this?

...

I think I figured something out!

use paste::paste;

macro_rules! rename_constants {
    ($rename_macro:ident; $($tail:tt)*) => {
        rename_constants_helper!(0; $rename_macro; $($tail)*);
    };
}

macro_rules! rename_constants_helper {
    ($state:expr; $rename_macro:ident;) => {
        // Done
    };
    ($state:expr; $rename_macro:ident; $vis:vis const $name:ident : $ty:ty = $value:expr; $($tail:tt)*) => {
        $rename_macro!(custom_renamer_callback($vis const $name : $ty = $value), $name);
        // This is what I want to do, but I can't invoke $rename_macro from within paste!
        // paste! {
        //     $vis const $rename_macro!($ident) : $ty = $value;
        // }
        rename_constants_helper!($state + 1; $rename_macro; $($tail)*);
    };
}

macro_rules! custom_renamer_callback {
    ($vis:vis const $name:ident : $ty:ty = $value:expr, $paste_args:tt) => {
        paste! {
            #[allow(dead_code)]
            $vis const $paste_args : $ty = $value;
        }
    };
}

macro_rules! custom_renamer {
    ($callback:ident($($args:tt)*), $name:ident) => {
        $callback!($($args)*, [<CUSTOM_BEGIN_ $name _CUSTOM_END>]);
    };
}

rename_constants! {
    custom_renamer;
    pub const X1: u8 = 1;
    pub const X2: u8 = 3;
    pub const X3: u8 = 8;
}

pub fn main() {
    println!("Custom X1 = {CUSTOM_BEGIN_X1_CUSTOM_END}");
    println!("Custom X2 = {CUSTOM_BEGIN_X2_CUSTOM_END}");
    println!("Custom X3 = {CUSTOM_BEGIN_X3_CUSTOM_END}");
}

Playground link

Although I can't take the output of a user's paste! invocation and use it as an identifier for a const, I can take raw paste! arguments from a user's macro invocation and pass them into an invocation of paste...although it does require the user to implement a macro with less-than-ideally-simple inputs.

If anybody has any suggestions on any way to improve/simplify this approach, I'd love to hear it :+1:

Thanks again to @steffahn!

5

You don’t need to hard-code usage of paste, the user just has to write their call-back correctly:

macro_rules! custom_renamer_callback {
    ($vis:vis const $name:ident : $ty:ty = $value:expr, $paste_args:tt) => {
-       paste! {
            #[allow(dead_code)]
            $vis const $paste_args : $ty = $value;
-       }
    };
}

macro_rules! custom_renamer {
    ($callback:ident($($args:tt)*), $name:ident) => {
+       paste! {
            $callback!($($args)*, [<CUSTOM_BEGIN_ $name _CUSTOM_END>]);
+       }
    };
}

Admitted: of course this makes writing the callback even more verbose. But the callback can then also be arbitrarily complex, even a proc-macro.

But I still don’t really have a good feeling for the practical use-case you’re after here. As far as this example goes, the user might as-well have simply provided the CUSTOM_BEGIN_ and _CUSTOM_END prefex/suffix directly, instead of making a custom_renamer macro out of it..

6

Hmm… so one thing for simplification can be to replace the $(…)*-wrapped args with a simple/single token-tree; e.g. (I’m staying with the version where paste is used by the user):

macro_rules! custom_renamer {
    ($cb:ident$t:tt $name:ident) => {
        paste! {
            $cb!{$t [<CUSTOM_BEGIN_ $name _CUSTOM_END>]}
        }
    }
}

You just need to wrap up all the extra stuff that needs to be piped through the callback in some set of parentheses to make this workable. (playground).

Thinking about potential further simplification, I do suppose one could offer another helper-macro for callbacks? :thinking:

So in theory, this is somewhat “simpler” even, at least as far as writing out that $cb:ident$t:tt part goes (even though that was already easier than $callback:ident($($args:tt)*),)… really we can pass an arbitrary amount of stuff in a single $__:tt – we just can’t call that directly anymore, without unpacking… but I guess:

Here’s a fun design idea for making this – perhaps – more “ergonomic”

macro_rules! return_ {
    ([$outer:ident, $($outer_args:tt)*] $($inner_result:tt)*) => {
        $outer!{$($inner_result)* $($outer_args)*}
    }
}

then the callback definition could do

macro_rules! custom_renamer {
    ($__:tt $name:ident) => {
        paste! {
            return_!{$__ [<CUSTOM_BEGIN_ $name _CUSTOM_END>]}
        }
    }
}

I suppose, we can make the call-site more ergonomic as well… e.g. with something like

macro_rules! call {
    ($outer:ident!{$inner:ident!{__ $($inner_args:tt)*} $($outer_args:tt)*}) => {
        $inner!{[$outer, $($outer_args)*] $($inner_args)*}
    }
}

it can look like

call!{custom_renamer_callback!{$rename_macro!{__ $name}, $vis, $ty, $value}}

superficially “simulating” some inside-out evaluation, if you will…, the custom_renamer_callback’s definition here is matching that call expression’s interior as if $rename_macro!{__ $name} simply expanded to some tokens (a single ident in this case):

macro_rules! custom_renamer_callback {
    ($name:ident, $vis:vis, $ty:ty, $value:expr) => {
        #[allow(dead_code)]
        $vis const $name: $ty = $value;
    };
}

(playground)

Moving back to a paste-in-the-custom_renamer_callback approach is supported, too, of course! Simply

macro_rules! custom_renamer_callback {
    ($pasteable_name:tt, $vis:vis, $ty:ty, $value:expr) => {
        paste! {
            #[allow(dead_code)]
            $vis const $pasteable_name: $ty = $value;
        }
    };
}

now expects an arbitrary tt (expected to contain a paste-evaluable [<…>]-style token-tree). And the user’s call-back becomes

macro_rules! custom_renamer {
    ($__:tt $name:ident) => {
        return_!{$__ [<CUSTOM_BEGIN_ $name _CUSTOM_END>]}
    }
}

(playground)

1 Like