Re-parsing $var:expr (or tt) in recursive macros

Hello,

Running the code below (Rust Playground) the first macro invocation below "A" matches the "nested" arm, while the second macro invocation below "B" first matches the split ("car") arm, but then the recursive call ends up matching the "non-nested" arm. I'm trying to get a deeper understanding of why.

My guess is that, somewhere in the process, the second expression gets parsed as a tuple, and then becomes a single token literal instead of a token-tree.

Is this guess right? And how do I stop that or somehow re-parse the code? Or do I just have a dumb bug in the macro.

Thank you.

macro_rules! expr {
    (($car:tt, $($cdr:tt $(,)?)+)) => {
        {
            println!("nested {:?}", $car);
        }
    };
    ($car:tt $(,)?) => {
        {
            println!("non-nested {:?}", $car);
        }
    };
    ($car:tt, $($cdr:tt $(,)?)+) => {
        {
            println!("car {:?}", $car);
            expr!($($cdr,)*)
        }
    };
}

fn main() {
    println!("A");
    expr!((0, 1));
    
    println!("B");
    expr!(0, (0, 1));
}

The fact is that tt can represent stuff in delimiter or stuff including the delimiters.

DelimTokenTree :
( TokenTree * )
| [ TokenTree * ]
| { TokenTree * }

TokenTree :
Tokenexcept delimiters | DelimTokenTree

See: Macros By Example - The Rust Reference

  • tt: a TokenTree (a single token or tokens in matching delimiters (), [], or {})

And you have a , in the nested expr!($($cdr,)*) which is matched against the second branch because nothing appears after the comma. Rust Playground
You may remove the comma and the expansion is what you expect it to be. Rust Playground

P.S. obviously, if you want to specify tt to match tokens except (), you should use ( $($stuff:tt)* )

Thanks for the reply.

The absence of the comma in the second non-nested branch is because I want to also match:

    println!("C");
    expr!("zero");

I want to keep the repetition (or some other greedy construct) in the third branch because I also want to match something like this:

    println!("D");
    expr!(0, 1, 2);

and even

    println!("E");
    expr!(0, ('A', 'B'), 2, ('A', ('a', 'b', 'c')));

Here is an updated playground with the additional cases, switching tt to expr, and it would do what I want if I could understand why the recursive invocation won't go down the "nested" path.

Thanks again

You mean this? https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=07a1d14ef7869930bb1dfcbc176f4f2a

expr!(0, ('A', 'B'), 2, ('A', ('a', 'b', 'c')));

// expansion
car 0
nested 'A'
non-nested 'B'
car 2
nested 'A'
nested 'a'
car 'b'
non-nested 'c'
1 Like

Yes exactly. Beautiful!

It seems my cardinal misstep was how I handled the trailing comma in the repeating statement.

Thank you for setting me straight.