The trait `ToTokens` is not implemented for `Map<Filter<syn::punctuated::Iter<'_, Variant>,

Hello, everyone! Thanks in advance.

I am trying to implement a Procedural Macro for an Enum. The idea is very simple: Different Variants within the Enum are classified into three groups (i.e., physical, personal, operational). I want to derive(ElementKind) using attributes(personal, physical, operational). Now, I am having some trouble iterating Variants... this is my first time developing proc_macros, but I did not find this issue when iterating Fields in a Struct.

Anyway, I am doing the following:

#[proc_macro_derive(ElementKind, attributes(personal, physical, operational))]
pub fn derive_kind(input: TokenStream) -> TokenStream {
    // Parse the input tokens into a syntax tree
    let ast = parse_macro_input!(input as DeriveInput);

    match ast.data {
        syn::Data::Struct(_) => derive_struct_kind(ast), // This is unimplmented!()
        syn::Data::Enum(_)=>derive_struct_enum(ast),
        _ => {
            panic!("#name ::: can only be implemented in Structs and Enum");
        }
    }
}

Then, just to show one case, the function derive_struct_kind() is the following:

fn derive_struct_enum(ast: syn::DeriveInput) -> TokenStream {
    
    let enum_name = &ast.ident;

    // Get structure fields or panic (THIS WORKS)
    let variants = if let syn::Data::Enum(syn::DataEnum {
        ref variants,
        ..
    }) = ast.data
    {
        variants
    } else {
        panic!("THIS IS A BUG: Expecting an Enum");
    };

    // Sort variants    (THIS WORKS)
    let mut physical = variants.iter().filter(|v|{
        let att_names : Vec<String> = v.attrs.iter().map(|a|{
            format!("{}",a.path.segments[0].ident)
        }).collect();
        att_names.contains(&"physical".to_string())
    }).map(|x|{x});
    
    let is_physical_doc = format!("Checks whether a {} is of kind Physical", enum_name);
    
    TokenStream::from(quote!{
        impl #enum_name {
            #[doc = #is_physical_doc]
            pub fn is_physical(&self)->bool{
                #physical  // THIS FAILS!!!!!! ... I know this will not compile. I will fix this syntax when I get this to expand, at least
            }
        }
    })
}

The error I am getting is the trait 'ToTokens is not implemented for Map<Filter<syn::punctuated::Iter<'_, Variant>, [closure@src/lib.rs:21:53: 26:10]>, [closure@src/lib.rs:26:16: 26:22]>'.

What am I doing wrong? When the map to be tokenized is Map<Filter<syn::punctuated::Iter<'_, Field>,... it seems to work. What am I missing?

I would appreciate your help. (I have tried reading stuff, searching in this forum, and several "patterns" of implementing the same Macro, and I always end up with a similar error...)

Best!

The syntax #var_name inside a quote!-like macro is used to interpolate the variable var_name as the source code it is expected to represent, and that "mapping" / definition is defined/provided by the ToTokens implementation.

But in your case you are not dealing with one "standalone" element to tokenize, but with a "sequence" of such / several such: an impl IntoIterator<Item = impl ToTokens>.

In that case, the syntax that will always work is to use the #( … #iterable … )* syntax: while some iterables may already provide a convenience ToTokens implementation allowing you to directly use the #iterable syntax as a shorthand for #(#iterable),* (for instance), this is not true of all iterables. So do use the general syntax:

    TokenStream::from(quote!{
        impl #enum_name {
            #[doc = #is_physical_doc]
            pub fn is_physical(&self)->bool{
-               #physical  // THIS FAILS!!!!!! ... I know this will not compile. I will fix this syntax when I get this to expand, at least
+               #(#physical),*
            }
        }
    })

Extra tips: precisely because of that syntax, in order for the code to be readable, I've found that naming the iterable each_… makes stuff way more readable. Moreover, in your case, you are dealing with variants.

So I'd further do:

-   let mut physical = …
+   let each_physicial_variant = …;

so as to write:

    TokenStream::from(quote!{
        impl #enum_name {
            #[doc = #is_physical_doc]
            pub fn is_physical(&self)->bool{
                #(#each_physical_variant),*
            }
        }
    })

Note now that this expansion, which works, quote!-wise, and which is readable, human-wise (the first two issues of your OP), still makes no sense, generated-code-wise.

Indeed, imagine the following enum:

#[derive(ElementKind)]
enum SomeEnum {
    #[physical]
    Foo { a: bool },
    #[physical]
    Bar,
    Baz {},
    #[physical]
    Quux(String),
}

then, the generated code would be:

        impl SomeEnum {
     //   missing ` ` –     missing `[` `]` 
     //        v                v      v
            ///Checks whether a SomeEnum is of kind Physical
            pub fn is_physical(&self)->bool{
                 Foo { a: bool }, Bar, Quux(String)
            }
        }

I assume you'd rather write something like:

pub
fn is_physical (self: &'_ Self)
  -> bool
{
    ::core::matches!(*self, Foo { .. } | Bar | Quux(_))
}

or, to keep the code generation / proc-macro logic simple:

pub
fn is_physical (self: &'_ Self)
  -> bool
{
    ::core::matches!(*self, Foo { .. } | Bar { .. } | Quux { .. })
}
  • Actually, it is better to directly use a match rather than matches! to handle the case of there not being any physical variant whatsoever.

Hence,

The final code

use ::syn::*;

fn derive_struct_enum (type_def: DeriveInput)
  -> TokenStream
{
    #![allow(nonstandard_style)]

    let EnumName @ _ = &type_def.ident;

    let variants = match &type_def.data {
        | Data::Enum(DataEnum { variants: it, .. }) => it,
        | _ => unreachable!(),
    };

    let EachPhysicalVariant =
        variants
            .iter()
            // Only keep the `#[physical]`-annotated variants
            .filter(|variant| {
                variant
                    .attrs
                    .iter()
                    .any(|attr| attr.path.is_ident("physical"))
            })
            // From `Foo { … }` get `Foo`
            .map(|Variant { ident: VariantName @ _, .. }| VariantName)
    ;
    
    let is_physical_docstring =
        format!(" Checks whether a [{}] is of kind `Physical`", EnumName)
    ;
    
    TokenStream::from(quote!(
        impl #EnumName { // <- Assumes there being no generics.
            #[doc = #is_physical_docstring]
            pub
            fn is_physical (self: &'_ Self)
              -> ::core::primitive::bool
            {
                #[allow(unreachable_patterns)]
                match *self {
                #(
                    | Self::#EachPhysicalVariant { .. } => true,
                )*
                    | _ => false,
                )
            }
        }
    ))
}
1 Like

Excellent tips, thanks a million!

Although, I had to change one thing. This code:

let EachPhysicalVariant =
        variants
            .iter()
            // Only keep the `#[physical]`-annotated variants
            .filter(|variant| {
                variant
                    .attrs
                    .iter()
                    .any(|attr| attr.path.is_ident("physical"))
            })
            // From `Foo { … }` get `Foo`
            .map(|Variant { ident: VariantName @ _, .. }| VariantName)
    ;

was keeping the attributes, which ended up being written in the match statement later on, failing. So, I used quote! to return a clean verion of it as follows:

 let each_physical_variant = variants.iter().filter(|v|{
        contains_attr(v,&"physical") // This checks whether the variant has an attribute called "physical"
    }).map(|x|{
        let v_ident = x.ident.clone();
        quote!{
            Self::#v_ident{..}
        }        
    });

And then

TokenStream::from(quote!(
        impl #enum_name { // <- Assumes there being no generics.
            #[doc = #is_physical_docstring]
            pub fn is_physical (self: &'_ Self) -> ::core::primitive::bool{                                
                match self {
                    #(
                        | #each_physical_variant => true,
                    )*
                    | _ => false,
                
                }
            }
}

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.