Why does a nested proc macro in a declarative macro make the ident have a hygiene issue?

help
1 
macro_rules! test_me {
    ($name:ident) => {
       proc_crate::just_ret_let!($name);
    };
}

fn main(){
   test_me!(ddd);
   println!("{ddd}");
}

The definition in just_ret_let in proc_crate is simply:

#[proc_macro]
pub fn just_ret_let(v:TokenStream)->TokenStream{
    let ident = v.to_string();
    format!(r#"let {ident} = "just_ret_let";"#).parse().unwrap()
}

The compile reports an error that:

  --> src/main.rs:11:16
   |
11 |     println!("{ddd}");
   |                ^^^ not found in this scope

If change proc_crate::just_ret_let! to another declarative macro, for example:

#[macro_export]
macro_rules! model_proc_m {
    ($name:ident) => {
        let $name = "model_proc_m";
    };
}

This is ok. Why does the nested proc macro make the hygiene apply here? According to the clause Hygiene - The Little Book of Rust Macros, the substituted $name should have the same syntax context as the identifier ddd in the call site, why are they not considered to have the same syntax context after the proc macro expansion?

2

Because they are not from the same context, they are from the context of a string inside the proc-macro. You are not preserving the Spans of original tokens, you're producing a new token stream from scratch.

3

if set the span of the identifier in just_ret_let to call_site, it still does not work.

#[proc_macro]
pub fn just_ret_let(v: TokenStream) -> TokenStream {
    let ident = v.to_string();
    let r = [
        TokenTree::Ident(Ident::new("let", Span::call_site())),
        TokenTree::Ident(Ident::new(&ident, Span::call_site())),
        TokenTree::Punct(Punct::new('=',Spacing::Alone)),
        TokenTree::Literal(Literal::string("just_ret_let")),
        TokenTree::Punct(Punct::new(';',Spacing::Alone))
    ];
    TokenStream::from_iter(r.into_iter())
}

Does it mean, by the interpretation of the little book of Rust Macros, the call_site denotes the syntax context just_ret_let will have in the macro expansion of test_me?

742×356 5.12 KB
, the syntax context denoted by call_site is that green, which is not the same as ddd, right?

4

I think the below would work

use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, Ident};

#[proc_macro]
pub fn just_ret_let(v: TokenStream) -> TokenStream {
    let ident = parse_macro_input!(v as Ident);
    let expanded = quote! {
        let #ident = "just_ret_let";
    };
    TokenStream::from(expanded)
}

Converting the input v to a raw string would lose the input TokenStream span information, in this case in the scope of main(). Instead, parse_macro_input would create a new Ident which is constructed with a span. This span is maintained by quote!.

1 Like
5

One other thing to note is, to verify the spans, you can also run the below

// Inside proc macro crate
use proc_macro::TokenStream;

#[proc_macro]
pub fn just_ret_let(v: TokenStream) -> TokenStream {
    let ident = v.to_string();
    let ts: proc_macro2::TokenStream = format!(r#"let {ident} = "just_ret_let";"#).parse().unwrap();
    for token in ts.clone().into_iter() {
        println!(
            "Token: {:?}, Span Start: {:?}, Span End: {:?}, Source File: {:?}",
            token,
            token.span().start(),
            token.span().end(),
            token.span().source_file(),
        );
    }
    ts.into()
}

// `main.rs` Inside main crate
macro_rules! test_me {
    ($name:ident) => {
        proc_macro_test::just_ret_let!($name);
    };
}

fn main() {
    test_me!(ddd);
    // Commented out since this will not compile
    // println!("{ddd}");
}

with the command (using nightly features and configurations to obtain the span locations and source file),

RUSTFLAGS='--cfg procmacro2_semver_exempt' cargo +nightly run

This will give an output like

Token: Ident { sym: let, span: #6 bytes(54..91) }, Span Start: LineColumn { line: 3, column: 8 }, Span End: LineColumn { line: 3, column: 45 }, Source File: SourceFile { path: "main/src/main.rs", is_real: true }
...

which in this case illustrates that the span of the tokens in ts resolve to line 3 in main.rs which is inside the test_me! macro. This means the ddd identifier is not in scope in main() due to macro hygiene.

You can then run a similar command with the alternative suggestion to see the spans resolve inside the main() function scope.

1 Like