Proc-macro-attribute parse_macro_input!: error: expected literal

help
1

I'm obviously struggling with parsing in the context of a proc-macro-attribute....
I've also had difficulty getting a fully working example in the playground, so please excuse the code pasting.

Context:
Take an attribute with these forms:

  1. #[trace],
  2. #[trace("own")],
  3. #[trace(other=true)],

and after parsing have an argument list as Vec<NestedMeta> that would be equivalent to:

  1. #[trace(name="__default")],
  2. #[trace(name="own")],
  3. #[trace(name="__default", other=true)]

Status:

I am able to take the TokenStream and rewrite or append the name parameter. The relevant code for this is:

    let parser = syn::punctuated::Punctuated::<syn::Expr, syn::Token![,]>::parse_terminated;
    let mut stream = syn::parse::Parser::parse2(parser, (*args).clone()).expect("Token stream");
    let mut name_pair: syn::Expr = syn::parse_str::<syn::Expr>("name = __default").unwrap();
    stream.iter_mut().for_each(|e| {
        #[allow(clippy::single_match)]
        match e {
            syn::Expr::Lit(syn::ExprLit {
                lit: syn::Lit::Str(lit_str),
                attrs: _,
            }) => {
                let value = (*lit_str).value();
                let name = format!("name = {}", value);
                name_pair = syn::parse_str::<syn::Expr>(name.as_str()).unwrap();
            }
            _ => {} //?
        }
    });
    stream.push(name_pair);

However, this 'new' TokenStream, say ts, breaks parse_macro_input!(ts as AttributeArgs) with the error, error: expected literal

I'm obviously missing something obvious, or going about this all wrong.

The attribute code is:

#[proc_macro_attribute]
#[proc_macro_error]
pub fn trace(
    args: proc_macro::TokenStream,
    items: proc_macro::TokenStream,
) -> proc_macro::TokenStream {

    println!("args: {:#?}", args);

    let nm_args = parse_macro_input!(args as AttributeArgs);

    let validated = trace::validate(&args.clone().into(), &items.clone().into());
    let va: proc_macro::TokenStream = validated.into();
    println!("va: {:#?}", va);

    let nm_vargs = parse_macro_input!(va as AttributeArgs);
    
    items
}

Given the test case is:


#[trace(enter_on_poll = true)]
fn f(a: u32) -> u32 {
    a
}

fn main() {
    f(1);
}

When I run this test case, then the error is:

┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
error: expected literal
 --> tests/trace/ui/ok/002-has-enter_on_poll-ident.rs:3:1
  |
3 | #[trace(enter_on_poll=true)]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: this error originates in the attribute macro `trace` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0425]: cannot find function `f` in this scope
 --> tests/trace/ui/ok/002-has-enter_on_poll-ident.rs:9:5
  |
9 |     f(1);
  |     ^ not found in this scope
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈

If I comment out the parse_macro_input!(va as AttributeArgs), the print statements show these 2 data structures:

  1. args: TokenStream [
    Ident {
        ident: "enter_on_poll",
        span: #0 bytes(31..44),
    },
    Punct {
        ch: '=',
        spacing: Alone,
        span: #0 bytes(44..45),
    },
    Ident {
        ident: "true",
        span: #0 bytes(45..49),
    },
]

  2. va: TokenStream [
    Ident {
        ident: "enter_on_poll",
        span: #0 bytes(31..44),
    },
    Punct {
        ch: '=',
        spacing: Alone,
        span: #0 bytes(44..45),
    },
    Ident {
        ident: "true",
        span: #0 bytes(45..49),
    },
    Punct {
        ch: ',',
        spacing: Alone,
        span: #4 bytes(23..51),
    },
    Ident {
        ident: "name",
        span: #4 bytes(23..51),
    },
    Punct {
        ch: '=',
        spacing: Alone,
        span: #4 bytes(23..51),
    },
    Ident {
        ident: "__default",
        span: #4 bytes(23..51),
    },
]

Appreciate any hints or tips about what I might be doing incorrectly.

2

true and __default are not literals, they are identifiers (as it is also evident from the Debug representation of the token stream). Didn't you mean "true" and "__default", which would indeed be literals?

1 Like
3

Thank you - I had indeed missed that subtlety:

    let parser = syn::punctuated::Punctuated::<syn::Expr, syn::Token![,]>::parse_terminated;
    let mut stream = syn::parse::Parser::parse2(parser, (*args).clone()).expect("Token stream");
-    let mut name_pair: syn::Expr = syn::parse_str::<syn::Expr>("name = __default").unwrap();
+    let mut name_pair: syn::Expr = syn::parse_str::<syn::Expr>(r#"name = "__default""#).unwrap();
    stream.iter_mut().for_each(|e| {
        #[allow(clippy::single_match)]
        match e {
            syn::Expr::Lit(syn::ExprLit {
                lit: syn::Lit::Str(lit_str),
                attrs: _,
            }) => {
                let value = (*lit_str).value();
-                let name = format!("name = {}", value);
+                let name = format!(r#"name = "{}""#, value);
                name_pair = syn::parse_str::<syn::Expr>(name.as_str()).unwrap();
            }
            _ => {} //?
        }
    });
    stream.push(name_pair);
4

Btw, instead of constructing and re-parsing raw representations as strings, you should use quote!() / parse_quote!().

1 Like
5

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.