Why is the innermost meta-variable expansion impacted by the outmost one?

help
1 
fn get<T>(_:T){}
fn post<T>(_:T){}
pub mod a{
	pub mod b{
		pub struct C;
	}
}
macro_rules! call_methods {
	($($m:ident), + => $($a:ident)::*) => {
		{
			$($m($($a)::*));+
		}
	};
}
fn main(){
	call_methods!(get, post => a::b::C);
}

I expect that the macro invocation call_methods!(get, post => a::b::C); will be expanded to

{
   get(a::b::C); post(a::b::C);
}

However, the compiler cannot compile this code and reports an error:

error: meta-variable m repeats 2 times, but a repeats 3 times

Why are these meta-variables not expanded on their own separately?

2

I'm on mobil, so I'll skip explaining the intricacies of repetitions and nested repetition in macros, but for your use-case you can probably just use the :path fragment specifier to accept the whole of a::b::C as a single metavariable without repetitions. See here for a list of fragment specifiers..

3

The reason why I used $($a:ident)::* is because I want to extract the last identifier in the sequence separated by ::, then I want to use the whole "path" as the argument to invoke the methods.

4

Looks like a bad error message and the real limitation is that you have to expand the variable at the depth it was matched at.

1 Like
5

the real limitation is that you have to expand the variable at the depth it was matched at.

What does this mean? In my understanding, $($a)::* should expand meta-variable $a, and $($method(...))+ should expand meta-variable $method, why do they associate with each other?

6

The TLDR is that you cannot do the kind of combinatorial expansion you want, at least not directly. Instead, you will need to recurse, with one iteration per $m or per $a.

I find it tricky to explain exactly what's going on here. The macro processor treats all of the repetitions as being processed in lock-step. When you enter one of these, you are entering all of them simultaneously. So, what you think you're doing is nesting two for loops, something akin to:

for $m_entry in $m {
    print!("{$m_entry}(");
    for (i, $a_entry) in $a.enumerate() {
        if i > 0 { print!("::"); }
        print!("{$a_entry}");
    }
    print!(")");
}

But what is actually happening is more akin to:

assert_eq!($m.len(), $a.len());
for i in 0..$m.len() {
    let $m_entry = $m[i];
    let $a_entry = $a[i];
    print!("{$m_entry}({$a_entry})");
}

(I don't know that the compiler gets as far as noticing the over-nesting of $a here. My assumption is that it trips and falls over when it first notices that the lengths of $m and $a don't match, hence the error message.)

I think of it as the macro processor descending in "layers", where each layer unlocks a new depth of nested repetitions.

Whether you think this is a good idea or not is irrelevant: this is how it works, and it ain't changing.

4 Likes
7

Here's the documentation.

https://doc.rust-lang.org/reference/macros-by-example.html#repetitions

A metavariable must appear in exactly the same number, kind, and nesting order of repetitions in the transcriber as it did in the matcher.

3 Likes
8

(To be clear, I am not criticising you for linking this, or suggesting you shouldn't. My beef here is with the documentation itself. Edit: and yes, I realise this is the reference, not the book. I feel the point still stands.)

To be honest, I don't think that specific explanation is terribly helpful. I understand the behaviour it's trying to describe, and even I can feel my brain going "no, that's crazy, it can't mean that". It doesn't really convey that this is a "global" effect.

Like, yes, if I want to access the elements of a Vec<Vec<i32>>, I need two levels of indirection. Obviously. Now imagine if indexing into one array also implicitly indexed into all other arrays in scope. You would never expect that behaviour in normal, imperative code.

Maybe that's the disconnect? Rust is an imperative language... but macro_rules! is a weird array language with very different semantics around processing collections.

The description is precise and correct, but... fails to sufficiently establish context? I don't know. Documentation is hard.

9

Here's one workaround for the OP.

3 Likes
10

Does this mean all metavariables wrapping in the outermost same $(...) that would be expanded are always required to have the same number of elements?

11

Depending on exactly what you mean by that, yes. I'm sorry for being vague, but the devil is in the details here, and I'm not 100% sure you mean what I think you mean.

Here's an example that might help:

macro_rules! puom {
    (0: $($a:literal),*; $($b:literal),*) => {
        [$($a,)* $($b,)*]
    };
    (1: $($a:literal),*; $($b:literal),*) => {
        [$($a, $b,)*]
    };
}

fn main() {
    println!("{:?}", puom!(0: 1, 2, 3; 4, 5, 6));
    println!("{:?}", puom!(1: 1, 2, 3; 4, 5, 6));
    println!("{:?}", puom!(0: 1, 2, 3; 4, 5, 6, 7));
    // error: meta-variable `a` repeats 3 times, but `b` repeats 4 times
    //println!("{:?}", puom!(1: 1, 2, 3; 4, 5, 6, 7));
}

This outputs:

[1, 2, 3, 4, 5, 6]
[1, 4, 2, 5, 3, 6]
[1, 2, 3, 4, 5, 6, 7]

When $a and $b are in different repetitions, they expand independently, and can have different lengths. When they are in the same repetition, they must have the same length, and expand together.

2 Likes
12

I'm curious why this would be an error even though the number is the same

    (2: $($a:literal),*; $($b:literal),*) => {
        //[$($a, $b,)*]
        ($(($a,$($b),*)),*)
    };
    println!("{:?}", puom!(2: 1, 2, 3; 4, 5, 6));
1 Like
13

To quote the compiler:

If you count the "depth" of each metavar based on the repetition groups it's inside of (i.e. how many layers of $( ... )), $a is one deep, but $b is two deep. When you matched those metavars, they were both matched one deep.

That's what it's complaining about: you matched $b one deep, but are trying to expand it two deep.

It's a bit like trying to index b: Vec<i32> by doing b[i][j].

2 Likes
14

Can I understand the repetition group as that: a repetition group conceptually corresponds to a for iterator and a meta-variable in repetition resembles an array? $($metavariable) resembles to access the element of the array.

So, $($($m)) means to iterate the element acquired in the first depth(for loop).

15

Let's say you have a macro match that looks something like:

($a:ident; $($b:ident),*; $($c:ident => $($d:ident),*;)

You then have the following structure in the expansion (I'm ignoring how to represent the actual output):

$a, $($b, $c, $($d)*)*

Conceptually, the repetition groups would look something like:

fn expand(a: Ident, b: &[Ident], c: &[Ident], d: &[&[Ident]]) {
    for i in 0..witness_len(b, c, d) {
        let b = b[i];
        let c = c[i];
        let d = d[i];
        for j in 0..witness_len(d) {
            let d = d[j];
        }
    }
}

witness_len is a magic function that takes one or more collections, makes sure they all have the same length, then returns that length.

Note that although the collections are independent (and can be accessed independently), if they appear anywhere "inside" of a repetition group, they need to all have the same length as all other collections at that depth.

Also note that each repetition group "shadows" the metavar as it indexes into them. So the $d in the expansion is a single term, but its presence within the repetition groups causes it to be indexed into. It's a little bit as though you could do the following in Rust:

let a = vec![1, 2, 3];
for {
    println!("a is: {}", a);
}

... and have it output:

a is: 1
a is: 2
a is: 3

Technically, that wouldn't do anything because there are no repetition symbols. That is, it should be something like: $($($m)*)*.

Assuming that's what you meant, it would iterate over a collection of a collection of elements. So it would be akin to m: &[&[T]], and involve two nested loops.

2 Likes
16

This is also my understanding of your second last answers. This mental model is very helpful to understand what declarative macro does. Incidentally, in your last answers, Does $a, $($b, $c, $($c)*)* intend to mean $a, $($b, $c, $($d)*)*?

$($($m)*)*.

Yes, this is my intent in that comment.

Back to this question: why does ($(($a,$($b),*)),*) cause an error. With the mental model, this repetition looks something like this:

for i in 0..witness_len(a,b){
   let a = a[i];
   let b = b[i]; 
  for j in 0..witness_len(b){ // this causes the error
    /// ****
   /// let b = b[j];
  }
}

As indicated in the comment, the error is due to that b[i] is not a collection, both witness_len and b[j] can initiate that error

17

Yes. Sorry about that.

Yes. It seems the compiler checks the lengths first, and only if they match will it complain about the incorrect nesting/depth/indexing a non-collection (however you want to think about it).

I am relieved that we appear to have found an explanation that makes sense. :slight_smile:

1 Like
18

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.