Rust Declarative Macros and Scoping of tt

I encountered this problem while trying to apply a function to expressions inside a macro. I condensed it down to this somewhat silly minimal working example, where we have an expression inside curly braces and apply a function to it like so:

macro_rules! foo {
    ( {$expression:expr} .apply($($f:tt)+)) => {
        {
            let apply_func = $($f)+;
            apply_func($expression)
        }
    }
}

For my use case I am actually applying the function to multiple expressions, but that is not important here. The thing is that I bind a local variable apply_func to the tokens inside the apply(...). I was pretty sure that the local let binding would do exactly what I want and I am happy to say that the following expressions all work as intended:

//WORKS: simple use case
println!("{}", foo!({10}.apply(|x|x*x)));
//WORKS: nested use case
println!("{}", foo!({ foo!({10}.apply(|x|x*x)) }.apply(|x|x+3)));
//WORKS: using some function to pass to the macro
let some_func = |x|x+10;
println!("{}", foo!({ foo!({10}.apply(some_func)) }.apply(|x|x+3)));
//WORKS: calling a function in this scope `apply_func` and using it
//from the inner macro invocation
{
    let apply_func = |x|x*3;
    println!("{}", foo!({ foo!({10}.apply(apply_func)) }.apply(|x|x+3)));
}

The resulting output is

100
103
53
33

as expected.

However, the problem manifests if I give apply_func as the argument of the inner macro invocation like so:

// FAILS: trying to access the apply_func of the outer macro from
// the inner macro without defining it in this scope
println!("{}", foo!({ foo!({10}.apply(apply_func)) }.apply(|x|x+3)));

I feared that this could expand to

{
    let apply_func = |x| x + 3;
    apply_func({
        let apply_func = apply_func;
        apply_func(10)
    })
}

which I would not have liked one bit, because I want no interaction between apply_func instances of the nested scopes. But on my machine (with rustc 1.48.0 (7eac88abb 2020-11-16)) it gives a compile error

 |     println!("{}", foo!({ foo!({10}.apply(apply_func)) }.apply(|x|x+3)));
 |                                           ^^^^^^^^^^ not found in this scope

which is fantastic. Try it on the Rust Playground.

My questions are: Can I rely on that behavior? Is that actually expected and can someone explain that to me?

I know that Rust macros are not C-style text replacement. Is it the case that the tokens passed to the macro need to be valid code before being "pasted" into the macro expansion? (I asked this question on SO as well)

Yes, you can rely on declarative macros being hygenic. I.e. any identifiers in function scope will not be leaked outside the macro. Even for recursive calls to said macro

Thanks, just for my understanding: does that mean that the inner macro does not see the apply_func binding from the outer macro? And thus the compile error occurs when trying to expand let apply_func = apply_func in the inner macro?

Yep, that's exactly it. You can think of it as each invocation creating new identifiers for all local variables. Even if they look the same to you and me.

1 Like

Fantastic, thanks for clarifying this. That solves my problem. One follow up question: would the same be true for procedural macros?

Only if they use Span::mixed_site() (or the unstable Span::def_site()) to emit the identifier used in the binding.


Note, more generally, that the current hygiene of macro_rules! macros, i.e., that of mixed_site, only works for local variables (and lifetime names, and the special behavior of $crate). Anything else, such as:

  • constants and function names (and statics): non-local "value" namespace;

  • modules, external crates, crate:: or type names: "type" namespace;

  • macros (a namespace as well).

are not hygienic.

Meaning that if we take a simplification of your example:

macro_rules! let_x_42 {() => (
    let x = 42;
)}

let x = 0;
let_x_42!();
assert_eq!(x, 42); // Error, `x` refers to the zero-init one

then, you can have things such as:

macro_rules! let_x_42 {() => (
    let x = 42;
    macro_rules! x {() => ( x )}
)}

let_x_42!();
assert_eq!( x!(), 42 );

"Full" hygiene is what Span::def_site() is for, and is also the one used by the unstable macro macros.

Note that historically, and even currently by many proc-macros, Span::call_site() has been and may still be used, which makes using proc-macros quite error-prone hygiene-wise.

3 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.