Proc macro fails to compile but expanded code compiles

help
1

Hi All,

I'm having some trouble with a pretty significant proc macro and have had no luck trying to debug it. As a preface, I'll note that the code (macro and generated) is not something that I can release publicly in it's entirety, and I have not been able to repro the problem in a small standalone project partially since I don't even know what the problem is.

Anyways, the problem manifests itself as:

error: expected identifier, found `,`
  --> tests/test_compile.rs:23:1
   |
23 |    some_proc_macro! {
   |   _^
   |  |_|
   | ||
24 | ||     name: foobar,
25 | ||
26 | ||     some_type: int,
...  ||
37 | ||     some_other_macro_field: {},
38 | || }
   | || ^
   | ||_|
   | |__expected identifier
   |    while parsing this struct

Which I can only interpret as having a duplicate comma in a struct, or a comma before any fields. I've verified that parsing the macro and generating code itself succeeds by getting the macro to dump it's tokenstream in a panic right before returning it.

Looking at the structs generated by the dump (not by cargo-expand), I see:

type int = i32;
struct _AssertSized                                 
           where          
               int: ::std::marker::Sized;
struct _AssertDefault                                               
           where                                                     
               int: ::std::default::Default;
struct _AssertAny                                
           where                           
               int: ::std::any::Any;     
//...
struct foobar_type<T: Sized> {
    _dummy: std::marker::PhantomData<T>,
    bar: u16,
    foo: u16,
    foobar: std::num::NonZeroU16,
}
struct foobar_type2<T: Sized> {
    _dummy: std::marker::PhantomData<T>,
    foo: u16,
    bar: u16,
}

#[repr(align(8))]
struct foobar_refs<'a> {
    __dummy_phantom_noname__: std::marker::PhantomData<&'a u32>,
    ref1: &'a int,
    ref2: foobar_nested_refs<'a>,
}
struct foobar_nested_refs<'a> {
    _dummy: std::marker::PhantomData<&'a u32>,
    ref2: &'a int,
    ref3: foobar_type2<int>,
}

All of this looks like valid code, and in fact when looking at the entire cargo-expand output, it compiles with a few minor tweaks (getting rid of the $crate variables, adding the right feature flags, adding a main function). It also compiles when I copy struct definitions from the panic dump.

So, my two main questions are:

  • Are there errors cases which I'm not considering?
  • Does the expanded code have the offending code stripped somehow?
  • Am I insane and misreading this syntax, or somehow changing it to compile
  • Is rustfmt somehow changing the code or fixing an error so I can't see it in the formatted macro expansions?
  • Is this some weird proc_macro bug that I've run into?

Thanks for your help!

2

Can we get a link to the proc_macro?

3

unfortunately, I can't really include the proc macro here or the whole expanded code since it contains some fairly sensitive ip. It's also in total ~900 lines including all the helper functions.

I'm trying to get a smaller macro with the IP stripped, but I'm not really sure where or what the error is which makes it hard.

4

Ok, I figured it out - the jist of the matter seems to be that a sufficiently confused quote!{...} invocation will simply not emit any code while the rest of the expansion continues. Sufficient confusion can be caused by a single stray comma.

In this case, the resulting expansion was still valid, leading to this confusion. Knowing this I think I can update with a small test case to post here for reference, and possibly a test case of sorts to improve macro diagnostics.