Derive macro: trait ToTokens not implemented for Map<... syn::Field>

help
1

I want to write a macro to generate a new function for a struct, so that it runs get_field_<type_name> for each field.

#[derive(Debug, AutoStruct)]
struct MyStruct {
	geometry: Point,
	x: i64
}

should output code like this:

impl AutoStruct for MyStruct {
	fn generate<L: Layer>(layer: &L) -> Result<Self, Box<dyn Error>> {
		Ok(Self {
			geometry: match layer.get_field_Point("geometry")?.unwrap(),
			x:        match layer.get_field_i64("x")?.unwrap()
		})
	}
}

Here's the macro I'm writing and it fails:

imports 
use proc_macro::{TokenStream};
use proc_macro2::{Span};
use quote::quote;
use syn::{Data, DataStruct, Fields, parse, Type, Ident};


#[proc_macro_derive(AutoStruct)]
pub fn autostruct_macro_derive(input: TokenStream) -> TokenStream {
	// Construct a representation of Rust code as a syntax tree
	// that we can manipulate
	let ast:syn::DeriveInput = parse(input).unwrap();

	let name = &ast.ident;
	println!("buliding: name: {:?}", name);

	let (fields, types, method_names) = match &ast.data {
		Data::Struct(DataStruct { fields: Fields::Named(f), ..}) => (
				f.named.iter().map(|f| &f.ident),
				f.named.iter().map(|f| &f.ty),
				f.named.iter().map(|f|
					match &f.ty {
						Type::Verbatim(ts) => format!("get_field_{}", ts),
						_ => panic!("type must be concrete")
					}
				)
			),
		_ => panic!("only structs with named fields are supported")
	};

	let methods = method_names.map(|m| Ident::new(&m, Span::call_site()));

	let gen = quote! {
		#input

		impl AutoStruct for #name {
			fn generate<F: Layer>(reader: &F) -> Result<Self, Box<dyn Error>> where Self: Sized {
				Ok(#name {
					#(
						#fields: reader.#methods(#fields)?.unwrap(),
					)
				})
			}
		}
	};
	gen.into()
}
Full error message

(this is at quote! invocation) ^ the trait ToTokens is not implemented for Map<syn::punctuated::Iter<'_, syn::Field>, [closure@...src/lib.rs:17:24: 17:27]> required by a bound introduced by this call.

I'm lost on what else should I do to produce a token for each Ident?

2

Iterators and "iterables" (collections) require you to explicitly specify iteration in the quote. This is the same restriction that you find in regular, declarative macros – you can't paste a matcher $($fragment)* or $($fragment)+ using simply the name $fragment, you have to specify at what level the repetition occurs.

So instead of

#(
    #fields: reader.#methods(#fields)?.unwrap(),
)

you likely wanted

#(
    #fields: reader.#methods(#fields)?.unwrap(),
)*

By the way, many other things stand out as wrong in your code:

  1. You re-emit the whole type declaration. Maybe you are thinking that this is necessary? It isn't – the output of a derive macro is appended after the item it is invoked on. (Otherwise, if every derive macro had to re-emit the whole declaration, then it would be 1. extremely redundant and thus error-prone, and 2. impossible to apply more than one derive to a given item.)
  2. …?.unwrap() seems wrong. Are you sure you need both the ? and the unwrap? You should probably just handle any nested errors/Nones in a Result-returning function. Your users won't expect a function to panic if it already returns Result.
  3. The generated code is not robust. The generated type parameter F might easily clash with a type parameter on a generic type holding a function, for example. The Box<dyn Error> return type will only compile if std::error::Error is in scope.
  4. Putting a type name directly in the method name will yield mixed-case method names, which is un-idiomatic. Method names should be lower_snake_case in all user-facing situations.
1 Like
3

I tried it, but now the error is "the trait ToTokens is not implemented for proc_macro::TokenStream".

If by this you mean f.named.iter().map(|f| &f.ty),, this is a leftover (I thought I'd just need types), I should probably remove it.

The functions return Result<Option<T>>', because the methods I'm wrapping with get_field_ return Option(the GDAL one, IIRC). I think I'll reuse this for optional fields, rather than dounwrap()` under the hood.

Do you mean someone may use F for a function generic, rather than a struct generic, or I get you wrong?

4

No you're including the input token stream in your output quote before anything else, which is also where the error you mention is coming from. quote only implements ToTokens on proc_macro2's types, not proc_macro's

2 Likes
5

Indeed. I picked that up at some forum, and thought it should have been this way. Thanks! Now the code works.

6

Which is true. quote operates on the proc_macro2 shim types only.

No. By this, I meant that the type itself may have a type parameter named F, in which case your functions won't compile due to the name clash. For example, a generic type Foo<F> will have its impl would be derived in the following way:

struct Foo<F>(F);

impl<F> AutoStruct for Foo<F> {
    fn generate<F: Layer>(reader: &F) -> Result<…> {
        …
    }
}

and thus the type parameter on generate clashes with the type parameter on the type itself.

But nevermind, because your code won't even compile for any generic type in the first place. You only include the #name in the generated impl; you omit any type, lifetime, and const generics, so what your macro currently generates is something like

struct Foo<F>(F);

impl AutoStruct for Foo {
    fn generate<F: Layer>(reader: &F) -> Result<…> {
        …
    }
}

which is invalid, as Foo is not a type in itself, only Foo<F> is for some specific F.

7

I see. I'm far from failing there yet.

I found that type annotation Point is not Type::Verbatim, but Type::Path, and it's a deep nested structure.

8

Path has a convenience getter if you know it's just a simple type name with no leading path or type args

2 Likes
9

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.