Expanding macro into macro invocation

[jacg]

TLDR

Is it possible to nest invocations of macro_rules! macros, using the inner one to generate arguments passed to the outer one?

In other words, can some variation on the following line ever compile?

outer!(inner!(arg))

Put another way, does something akin to quasiquotation / unquoting exist in Rust macros?

Specific example

How would you formulate test use_generated below?

// A macro that uses two inputs
#[macro_export]
macro_rules! use {
    ($name:ident $operator:tt) => {
        format!("{} {}", stringify!($name), stringify!($operator))
    }
}

// Verify what the `use` macro received
#[test]
fn use_hand_given() {
    assert_eq!(use!(Add +=), "Add +=");
}

// A macro that generates different sets of symbols depending on input
#[macro_export]
macro_rules! generate {
    (+) => (Add +=);
    (-) => (Sub -=);
    (*) => (Mul *=);
    // etc.
}

// Try to use the generated symbols as arguments to `use!`
#[test]
fn use_generated() {
    assert_eq!(use!(generate!(+)), "Add +=");
}

Background

In the context of implementing arithmetic operators

Implementing ops traits

I am writing a type which should support an arithmetic operator via both Op and OpAssign. Let's take + and += as a concrete example. My natural instincts make me want to ensure that all the following cases work: a + b; &a + b; a + &b; &a + &b; a += b; a += &b Moving the LHS of += makes no sense, so there are only 2 (rather than 4) cases for += and the borrow is implicit on the LHS. It's pretty clear to me how to implement support for any single one of these cases in isolation, but…

I am exploring designs of macros which write all the required trait implementations.

Implementing a full set of arithmetic operators for a single operation requires use of six symbols:

• For +: +, +=, Add, AddAssign, add and add_assign
• For *: *, *=, Mul, MulAssign, mul and mul_assign
• ... and so on

I can write a macro with an interface approximately like this

make_ops(+ += Add AddAssign add add_assign for MyType (self, other) {
   ...
})

but burdening the user with having to write out the six symbols each time is naff. I would like the interface to be something like

make_ops(+ for MyType (self, other) {
  ...
})

The idea is to generate the remaining 5 symbols using a macro like generate! above.

Can this ever work?

Is there some better solution?

paste-based solution

The macro could use add and add_assign instead of + and += in its implementation, so using the paste crate I see how to implement an interface like

make_ops(add for MyType (self, other) {
  ...
})

which could be a viable solution to this specific problem, but I'd be interested to learn what is possible more generally.

[jethrogb]

@jacg

Is it possible to nest invocations of macro_rules! macros, using the inner one to generate arguments passed to the outer one?

In other words, can some variation on the following line ever compile?

outer!(inner!(arg))

This is not possible. But you can do the opposite: invoke a named macro from within a macro. I'm not sure if this solves your problem:

inner!(outer, arg)

See Callbacks

[Michael-F-Bryan]

What about writing one macro who's whole purpose is to match on the operator, fill in the extra symbols, then invoke another macro to actually write the impl?

Here's what it looks like to use:

use std::ops::{Add, AddAssign, Sub, SubAssign};

implement_operator!(impl Add for Foo, |left, right| Foo(left.0 + right.0));
implement_operator!(impl Sub for Foo, |left, right| Foo(left.0 - right.0));

#[derive(Debug, Copy, Clone, PartialEq)]
struct Foo(u32);

fn main() {
    let a = Foo(1);
    let b = Foo(2);
    
    println!("{:?} + {:?} = {:?}", a, b, a + b);
    
    let mut c = Foo(4);
    c += a;
    println!("{:?}", c);
}

My implement_operator!() macro isn't overly interesting. It just matches on the second word (Add and Sub) and fills in the blanks.

macro_rules! implement_operator {
    (impl Add for $target_type:ty, |$left:ident, $right:ident| $body:expr) => {
        actually_implement_operator!($target_type, Add, add, AddAssign, add_assign, |$left, $right| $body);
    };
    ...
}

And my actually_implement_operator!() macro looks pretty much identical to what you'd write by hand:

macro_rules! actually_implement_operator {
    (
        $target_type:ty,
        $op_trait:ident,
        $op_method_name:ident,
        $op_assign_trait:ident,
        $op_assign_method_name:ident,
        |$left:ident, $right:ident| $body:expr
    ) => {
        impl $op_trait<$target_type> for $target_type {
            type Output = $target_type;
            
            fn $op_method_name(self, $right: $target_type) -> Self::Output {
                let $left = self;
                $body
            }
        }
        
        impl $op_assign_trait for $target_type {
            fn $op_assign_method_name(&mut self, other: $target_type) {
                *self = <$target_type as $op_trait>::$op_method_name(self.clone(), other);
            }
        }
    }
}

... Is that kinda what you were looking for?

(playground)

[jacg]

@jethrogb

This is not possible. But you can do the opposite: invoke a named macro from within a macro. I'm not sure if this solves your problem:

inner!(outer, arg)

See Callbacks

Yes, it can be used to solve the problem. The solution I came up with is still a bit too verbose and the signal-to-noise ratio is lower than I would like, but it works: playground.

[jethrogb]

@jacg

Yes, it can be used to solve the problem.

You didn't use this though? You're just calling some macros the standard way.

[jacg]

@Michael-F-Bryan

... Is that kinda what you were looking for?

I think that a crucial part of my question (though maybe I didn't express it too clearly) is about mapping single items of information into sets of multiple items, and letting a macro use those multiple items as arguments[*].

In my specific case it's this sort of mapping:

+  -> + += Add AddAssign add add_assign
-  -> - -= Sub SubAssign sub sub_assign
...

and I don't see how your code addresses these mappings.

OTOH, I think that your code is structurally very similar to what I wrote here: but my version has an extra level of indirection precisely for the purpose of creating these mappings.

[*] Actually, I don't really care about them being arguments that the macro receives, it's just that I don't see any way of creating local variables in these macros other than through their parameters. At a high level, what I want is something like the following pseudocode:

let ops = hashmap! {
   + => (+= Add AddAssign add add_assign),
   - => (-= Sub SubAssign sub sub_assugn),
   ...
}

macro_rules! make_ops {
   ($op:tt $Type:ty ...) => {
      // Bind macro-local variables: NB this is not part of the expanded result
      let ($opa:tt $T:ident $TA:ident $fn:ident $fna:ident) = ops.get(+);
      // *THIS* is the result:
      impl<RHS> $T for $Type {
        fn $op(self, other) { $($body)* }
      }
      // more impls ...
   }
}

[jacg]

@jethrogb

You didn't use this though?

Hmm, yes, you're right. I started off with it as a starting point, but in order to get the dispatch/mapping to work I transitioned into something that doesn't really have that structure any more.

Am I missing some way in which using this callback pattern may make the implementation less convoluted?

[steffahn]

I’ve used ⟶ this pattern before of adding two additional arguments encoding a "continuation" and a call to that continuation to a helper macro (compare read_instruction and read_instruction_cps in the linked playground) that’s supposed to generate any kind of token tree the caller can inspect and use afterwards.

Edit: You can do fairly complicated stuff with this approach, but you might have to iterate through things recursively instead of just using $(...)* in some places.

[jacg]

Thanks for the CPS sample.

In trying to understand exactly what problem CPS is solving in this case (and, secondarily, how CPS solves it) I've boiled it down to this:

macro_rules! inner_works { () => { 2 } }
macro_rules! outer_works { () => { inner_works!() + 3 } }

macro_rules! inner_fails { () => { + } }
macro_rules! outer_fails { () => { 2 inner_fails!() 3 } } // ERROR if used

macro_rules! inner_fixed { ($l:tt $r:tt) => { $l + $ r } }
macro_rules! outer_fixed { () => { inner_fixed!(2 3)} }

macro_rules! cont      { ($op:tt $l:tt $r:tt) => { $l $op $r } }
macro_rules! inner_cps { ($k :tt $l:tt $r:tt) => { $k!(+ $l $r) } }
macro_rules! outer_cps { () => { inner_cps!(cont 2 3) }}

#[test]
fn hmm() {
    assert_eq!(outer_works!(), 5);
    //assert_eq!(outer_fails!(), 5); // Compile-time ERROR
    assert_eq!(outer_fixed!(), 5);
    assert_eq!(outer_cps!(),   5);
}

From this I infer that, inside a macro-expansion

• It is possible to use a macro which expands to an expression (inner_works)
• It is not possible to use a macro which expands to an operator (inner_fails)

(and the Lisper in me weeps)

This is pure phenomenology: my google-fu is failing me when I try to find the relevant section of the manual. Where can I read about exactly what is going on here?

How CPS helps

The macro which produces the operator, rather than returning the operator, passes the operator to the continuation, and lets the continuation combine the operator with its operands, to create the full expression, which can be the result of a macro expansion.

In other words, the problem is circumvented by passing the operator as an argument to a macro, rather than returning the operator as the result of a macro. Turning return values into function call arguments is a fundamental characteristic of CPS.

In this simple case CPS is overkill, as illustrated by {inner,outer}_fixed. Its mechanism is the same: turn a return value into an argument.

Remaining doubts

Where can I read about what exactly can and can't be "returned" by macro_rules! macros?

[2e71828]

@jacg

Where can I read about what exactly can and can't be "returned" by macro_rules! macros?

Macro invocations are only legal as patterns, types, statements, expressions, items, and extern blocks.

Although it hasn’t been updated in a while, the Little Book of Rust Macros still does a pretty good job of describing some of the more advanced techniques.

[jacg]

So it's not what the macro can expand to, rather it's the context in which the invocation can appear ... which probably amounts to the same, but it explains why I miserably failed to find what I was looking for.

Thanks.

[Yandros]

You should also, if you haven't yet, read:

The Little Book of Rust Macros

• It's a bit outdated, so there may be a few features the book doesn't know about (such as the $( )? Kleene operator, or, expanding to types, and some other minor things) but it's otherwise a very complete guide.