How can I invoke a macro as an argument to another macro

[Gonkalbell]

I'm having a problem that that has to do with using a macro invocation as an argument to a separate proc-macro where it expects a literal token. The specific macro I'm dealing with is include_spirv from https://github.com/PENGUINLIONG/inline-spirv-rs, but I think this issue I'm facing can be generalized with this example:

// suppose I have a macro_rules or proc-macro from another crate like this
macro_rules! outer {
    ($inner:literal) => {$inner}
}

// and I want to invoke it like this, using a macro like concat that expands into a literal
dbg!(outer!(concat!("inner")));

However, I get the error:

   Compiling playground v0.0.1 (/playground)
error: no rules expected the token `concat`
  --> src/main.rs:27:13
   |
23 | macro_rules! outer {
   | ------------------ when calling this macro
...
27 | dbg!(outer!(concat!("inner")));
   |             ^^^^^^ no rules expected this token in macro call

error: aborting due to previous error

error: could not compile `playground`

To learn more, run the command again with --verbose.

It looks like this is because rust lazily evaluates it's macros. Is there some trickery I can do to eagerly evaluate the inner macro first, so that this will work?

[jjpe]

Consider your definition of outer!: it defines $inner as a literal, which a macro/intrinsics call decidedly is not.

To fix this, you can have outer! accept an $inner:expr syntax fragment rather than an $inner:literal, which in turn will likely require modifying the macro body.

If the definition of outer! is not in your crate then all you can do is a fork-discuss-PR style change.

[Gonkalbell]

I think for my specific problem, I can't just switch it from expecting literals to expecting expressions. The include_spirv macro C style preprocessor substitution, where I can specify preprocessor macros in the call to include_spirv, and those will be substituted in the source code before it gets compiled.

In other words, I want to replace

include_spirv!("path/to/shader.glsl", vert, D SOME_PREPROCESSOR_SUBSTITUTION="foo");

with

include_spirv!("path/to/shader.glsl", vert, D SOME_PREPROCESSOR_SUBSTITUTION=some_other_macro!());

So even if I were to fork this crate and change this proc-macro to expect an expression, I'd need to make it actually evaluate that macro somehow. Otherwise, it would define SOME_PREPROCESSOR_SUBSTITUTION as the string "some_other_macro!()". And I'm not sure if there's a way to evaluate macro expressions from inside a proc-macro, since it's just treating everything as a bunch of tokens.

[Yandros]

That is indeed not possible currently, and there aren't even plans to support this in the mid-term. Basically we'd need, generally, one of two mechanisms:

• either an intrinsic function for proc-macros that would be able to eval a macro on a given TokenStream,

• or a macro with the same API as:

with_builtin_macros

• which, by virtue of being compiler blessed (contrary to the one in that crate), would be able to eval any macro.

With the latter, for instance, one would be able to write:

with_macro!(let $expansion = some_other_macro!() in {
    include_spirv!(..., D SOME_PREPROCESSOR_SUBSTITUTION=$expansion);
});

---

In the meantime, you'll need to have some_other_macro! be written by you so that it can cooperate:

macro_rules! some_other_macro_include_spirv {() => (
    include_spirv!(..., D SOME_PREPROCESSOR_SUBSTITUTION=<expansion...>);
)}

or, to generalize, you can use the callback pattern:

macro_rules! with_some_other_macro {(
    let $__:tt ($metavar:ident : tt)* = some_other_macro!() in {
        $($expansion:tt)*
    }
) => ( /* wrap in extra braces if you want to expand to an expression. */
    macro_rules! __ret {(
        $__ ( $metavar : tt )*
    ) => (
        $($expansion)*
    )}
    __ret! { <your actual expansion here> }
)}

• Usage:

  with_some_other_macro! { let $($expansion:tt)* = some_other_macro!() in {
      include_spirv!(..., D SOME_PREPROCESSOR_SUBSTITUTION=$($expansion)*);
  });
  

As you can see, nothing pretty

[jjpe]

Indeed nothing pretty. But the usage pattern wouldn't be any prettier for a compiler-blessed macro/intrinsic version of this.
Isn't there some sort of better solution, at least at the conceptual level?

[Yandros]

@jjpe

Isn't there some sort of better solution, at least at the conceptual level?

Conceptually, given how macros and parser passes have been designed, it is hard to imagine any other solution for macro_rules! macros: a preprocessor would be needed.

I have personally suggested the with syntax since it makes it quite obvious, but there is a big degree of freedom regarding the syntax, given that we are dealing with a preprocessor. We could for instance, borrow from the syntax of the most widespread preprocessor out there, paste!, and use, say, [< … >] to query an early expansion there and then:

with_eager_expansions! {
    include_spirv!(..., D SOME_PREPROCESSOR_SUBSTITUTION=[< some_other_macro!() >])
}

But yet again, this is conceptually, we still need:

• either a compiler-provided magic macro for this,

• or that some_other_macro! feature some kind of callback pattern such as showcased in my previous post.

For procedural macros, however, the alternative I mentioned:

@Yandros

either an intrinsic function for proc-macro s that would be able to eval a macro on a given TokenStream ,

would be pretty straight-forward:

//! proc-macro crate
extern "intrinsic" {
    fn expand_macro (
        path_to_macro: TokenStream, // ≈ ::syn::Path
        whence: Span, // to resolve relative paths to macros
        macro_input: TokenStream,
    ) -> Result<
        TokenStream, // expansion
        (Span, String), // ≈ ::syn::Error
    >;
}

With it, ::syn would be able to offer a .expand_inner_macros() convenience method on Expr, Item, Patern, or Type.

• (Although I don't know how ::proc_macro2 would do to polyfill that; I guess it would just always fail in that case)