Macro generating complete match

I'm trying to write a macro generating a match for all combinations of tuples.
I want this code:

matcher!(
    (db_url.scheme(), outname)
    :
    "mysql" => MySql,
    "postgres" => Postgres
    ;
    "json" => JSON,
    "xlsx" => XLSX
    ;
);

using the following macro:

macro_rules! matcher {
    (($pat1:stmt, $pat2:stmt) : $($str1:literal => $typ1:ty),* ; $($str2:literal => $typ2:ty),* ;) => {
        match ($pat1, $pat2) {
            matcher!(@branch $($str1 => $typ1),* ; $($str2 => $typ2),* ;),
            _ => ()
        }
    };
    (@branch $hds1:literal => $hdt1:ty $(,$tls1:literal => $tlt1:ty)* ; $($str2:literal => $typ2:ty),* ;) => {
        $(
            ($hds1, $str2) => { $typ2::<$hdt1>::write(&db_fetch::<$hdt1>(&db_url, &args.query).await?, &args.output)?; },
        )*
        matcher!(@branch $($tls1 => $tlt1),* ; $($str2 => $typ2),* ;)
    };
    (@branch ; ;) => {};
}

to generate the following:

match(db_url.scheme(),outname) {
    ("mysql","json") => {
        JSON::<MySql>::write(&db_fetch::<MySql>(&db_url, &args.query).await?, &args.output)?;
    },
    ("mysql","xlsx") => {
        XLSX::<MySql>::write(&db_fetch::<MySql>(&db_url, &args.query).await?, &args.output)?;
    },
    ("postgres","json") => {
        JSON::<Postgres>::write(&db_fetch::<Postgres>(&db_url, &args.query).await?, &args.output)?;
    },
    ("postgres","xlsx") => {
        XLSX::<Postgres>::write(&db_fetch::<Postgres>(&db_url, &args.query).await?, &args.output)?;
    },
    _ => ()
}

but I'm obtaining this one instead:

match(db_url.scheme(),outname) {
    ("mysql","json") => {
        JSON::<MySql>::write(&db_fetch::<MySql>(&db_url, &args.query).await?, &args.output)?;
    },
    ("mysql","xlsx") => {
        XLSX::<MySql>::write(&db_fetch::<MySql>(&db_url, &args.query).await?, &args.output)?;
    },
    matcher!(@branch "postgres" => Postgres; "json" => JSON,"xlsx" => XLSX;),
    _ => ()
}

recursive macro expansion seem to work only the first time (for "mysql").
the weirdest thing is that if remove the match from the macro the expansion works;
the macro modified as follows:

macro_rules! matcher {
    (($pat1:stmt, $pat2:stmt) : $($str1:literal => $typ1:ty),* ; $($str2:literal => $typ2:ty),* ;) => {
        matcher!(@branch $($str1 => $typ1),* ; $($str2 => $typ2),* ;),
    };
    (@branch $hds1:literal => $hdt1:ty $(,$tls1:literal => $tlt1:ty)* ; $($str2:literal => $typ2:ty),* ;) => {
        $(
            ($hds1, $str2) => { $typ2::<$hdt1>::write(&db_fetch::<$hdt1>(&db_url, &args.query).await?, &args.output)?; },
        )*
        matcher!(@branch $($tls1 => $tlt1),* ; $($str2 => $typ2),* ;)
    };
    (@branch ; ;) => {};
}

produces an (obviously) not working but correctly expanded code:

    ("mysql","json") => {
        JSON::<MySql>::write(&db_fetch::<MySql>(&db_url, &args.query).await?, &args.output)?;
    },
    ("mysql","xlsx") => {
        XLSX::<MySql>::write(&db_fetch::<MySql>(&db_url, &args.query).await?, &args.output)?;
    },
    ("postgres","json") => {
        JSON::<Postgres>::write(&db_fetch::<Postgres>(&db_url, &args.query).await?, &args.output)?;
    },
    ("postgres","xlsx") => {
        XLSX::<Postgres>::write(&db_fetch::<Postgres>(&db_url, &args.query).await?, &args.output)?;
    },

I'm sure How to generate => in macro is related but I can't figure out how to modify my macro to solve the problem.

1 Like

First of all: this doesn't work because macros can only expand to complete instances of specific syntax elements. Last I checked, match arms aren't on that list, which is hinted at by one of the errors you get:

error: unexpected `,` in pattern
  --> src/main.rs:4:74
   |
4  |               matcher!(@branch $($str1 => $typ1),* ; $($str2 => $typ2),* ;),

That macro invocation is at the point in a match where the compiler is expecting a pattern. That's why the comma is confusing it.

(There's also the fact that you're taking $pat1 etc. as stmt, and then using them in expression context. You can't do that.)

The trick with generating match expressions is that you need to expand the entire match in a single step. You can do this by pushing the partial output into the invocation of the next step, then substituting the accumulated output all at once.

Here's a partially working version of the macro.

I had to fudge some stuff because I don't have the rest of the code, and your write calls didn't make sense syntactically, so I'm guessing as to how they're supposed to look.

2 Likes

Thank you Daniel,
I understood the problem was about match branches unsupported by macros,
what I didn't grasp is what you said in a few simple words:

you need to expand the entire match in a single step

I'll adapt my code to your solution and I'll post a working version.

Thank you very much!

This macro is a nightmare: your solution worked but I can't figure out how to use turbofish on generic struct inside macro.
I modified the line

<$typ2>::write::<$typ1_head>(todo!());
$typ2::<$typ1_head>::write(&db_fetch::<$typ1_head>(&db_url, &args.query).await?,

because in my code $typ2 (JSON and SQLX) are generic structs:

pub struct XLSX<DB: Database> {
    phantom: PhantomData<DB>,
}

after this modification the compiler yells out:

error: expected identifier, found `<`
   --> src/main.rs:66:32
    |
66  | $typ2::<$typ1_head>::write(&db_fetch::<$typ1_head>(&db_url, &args.query).await?, &args.output)?;
    |        ^ expected identifier
...

the weirdest thing is that if "Expand macro recursively" the result is correct:

match ((db_url.scheme()), outname) {
    ("mysql", "json") => {
        JSON::<MySql>::write(&db_fetch::<MySql>(&db_url, &args.query).await?, &args.output)?;
    }
    ("mysql", "xlsx") => {
        XLSX::<MySql>::write(&db_fetch::<MySql>(&db_url, &args.query).await?, &args.output)?;
    }
    ("postgres", "json") => {
        JSON::<Postgres>::write(&db_fetch::<Postgres>(&db_url, &args.query).await?, &args.output)?;
    }
    ("postgres", "xlsx") => {
        XLSX::<Postgres>::write(&db_fetch::<Postgres>(&db_url, &args.query).await?, &args.output)?;
    }
    _ => (),
}

even using the "inline macro" function of VSCode the generated code compiles and works flawlessly.

Thank you again

1 Like

Hey, it's not the macro's fault! It's doing it's best.

This is why context matters.

TLDR: don't use the type capture. Try path instead. If that also fails, you may need to use ident and limit yourself to things that are in scope.

Given the following:

JSON::<MySql>::write

JSON is not a type. I mean, JSON refers to a type, but from the parser's perspective, the JSON token is not being parsed as a type. I think it's technically a "path": a sequence of identifiers and turbofish used to refer to a specific symbol (which could be a function, or a type, or a constant, or a variable).

This is one of those cases where being too specific in a macro causes the macro expansion to fail to parse. Yes, it looks correct, but the parser doesn't work based on what the macro output looks like. It works based on what you tell it a set of tokens are. It's a bit like the difference between duck-typing in something like JavaScript or Python, and strong-typing in Rust. Just because you have a String with value "24", that doesn't mean you can do arithmetic on it.

Expanding the macros and compiling the result can work because the act of turning the expanded syntax trees into text destroys this "strong typing". Fun fact: there are actually cases where this can break the output of macros that would otherwise be correct.

This specific circumstance (a "type name" where I can use it as a type or a path) is one where I don't know a good solution. I've tried path before, but it didn't work particularly well (I suspect you can't chain paths). The only reliable solutions I've found is to either stick to ident (which limits your "type names" to a single component) or $($name:tt)* (which gives you arbitrary type names, but complicates matching because this matches everything).

P.S. I literally just woke up, so the above might not be entirely coherent. Very rambly, at the least.

Would you mind sharing a working version of it so we can learn from it? Thank you.

I tried path before asking you but it didn't work :upside_down_face:
ident works as you suggested but it needs a refactoring to bring needed variables in scope.

Thank you very much!

Sure, I'll publish the github repo as soon as code work (and even I write a two-line README :slightly_smiling_face: ).

1 Like

Hi @jymchng,
I published a working version of the sql2any project:

is quite raw and simple but works :slight_smile:

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.