"Specializing" macros-by-example recursively

I have a macro in which I want to specialize on one of the arguments:

macro_rules! specialized {
    (u8, $e:expr) => {
        $e as u8
    };
    ($t:ty, $e:expr) => {
        $e as $t + 1
    }
}

This works fine if I call it directly with u8. But if I call it from another macro like so:

macro_rules! not_specialized {
    ($t:ty, $e:expr) => {
        // This never matches the (u8, $e:expr) rule in `specialized`
        specialized!($t, $e)
    }
}

The "specialization" fails -- when calling not_specialized!(u8, 7), the u8 is "lost" in the call to specialized!($t, $e). (In my actual use case, there are more macros, deeper layers, and a missed specialization is a compiler error due to type/trait mismatches.)

Is there a way to make this work without also specializing the outer macro? Does anyone have a reference where I can read more about this?

Playground.

I think that only tokens matched with a tt matcher will be re-analyzed in recursive macro calls. For this example, you can do this:

macro_rules! not_specialized {
    ($t:tt, $e:expr) => {
        // But this never matches the (u8, $e:expr) rule in `specialized`
        specialized!($t, $e)
    }
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=5b1630461e04f3e8a35a87ba3dd6666e

In a real situation, you probably can't get away with matching just a single tt for a type, so you'd probably need to enclose it somehow.

2 Likes

See the documentation / motivation for ::defile :

  • Besides the :tt and :literal captures (and maybe :lifetime? 🤔), that is, :expr, :item, :ty, :pat, :path (haven't checked with :block), the capture gets bound to a metavariable that wraps the captured token within invisible delimiters ("parenthesis").

    • In the case of :expr, this is justified so that if one captures 2 + 3 as $five:expr, and then does 4 * $five, the expected result is 20, and not the C-preprocessor-like behavior of 4 * 2 + 3 = 11.
  • The issue is that, AFAIK, one cannot write such invisible delimiters as the input of a macro_rules! to either inspect its contents or "match-compare" against them.

    • In your example, for instance, you had u8 captured by $t:ty, and thus $t expands to ⟨ u8 ⟩, hence not matching the literal u8 you had on your specialized rule.
  • Playground to better see this behavior.

Solutions

The two available solutions are to:

  • either avoid using high-level captures for the "dispatcher macro" (such as not_specialized in your example), a.k.a., "only use :tts", but then you lose the powerful parsing capabilities of such captures, and you end up having to use tt-munching;

  • or use proc-macros since they, on the other hand, can identify and inspect such things –as None-delimited Groups–, and even more conveniently, ::syn's special cases ParseStream<'_> so that calls to .parse() and .peek() transparently see through these delimiters 👌

Hence ::defile, with a good in between solution, much like with the problem of concatenating identifiers: expose a simple helper proc-macro to perform targeted None-delimiter ungrouping, so that users of it can remain in thr macro_rules! realm 🙂

2 Likes

Thanks! I'll play with :tt and decide if defile is the way to go instead.

For anyone who finds this thread in the future, here's the relevant part of the reference:

When forwarding a matched fragment to another macro-by-example, matchers in the second macro will see an opaque AST of the fragment type. The second macro can't use literal tokens to match the fragments in the matcher, only a fragment specifier of the same type. The ident , lifetime , and tt fragment types are an exception, and can be matched by literal tokens.

2 Likes