pub struct Schema<T>{
name: String,
origin: T, // the struct to be parse, like the User struct above.
fields: Vec<Field>, // the struct's fields
}
// the struct's fields info
pub struct Field {
field_name: String,
field_type: String,
field_attrs: FieldAttribute
}
// the field's attr info
pub struct FieldAttribute {
unique: bool,
default: String,
index: Vec<String> // Index of the database
}
Suppose I use json to express, its result should be as follows:
I know it can be achieved using derive macro, but I don’t know how to write it.
I have only recently begun to learn derive macro and have not fully mastered it yet. It is difficult to find useful tutorials about derive macro on the Internet.
Usually, the syn crate is used for parsing input to derive macros into an AST. Then you can process the resulting AST manually, or use higher-level creates, like darling.
Have a look at the examples in the darling repository. As far as I can see, all four use attributes (both struct and field) so that should give you somewhere to start.
If you're interested in spending some more time learning these APIs, https://github.com/dtolnay/proc-macro-workshop is invaluable. It's a set of tutorials/workshops on writing proc macros which start out simple, and get progressively more complex. By the end, I think you should be able to write something like what you're wanting to do.
I don't think what you're doing is going to be simple enough to be answered in a single post here, but hopefully we can provide some of the pieces.
Also unfortunately, there's not a lot of "documentation-like" documentation for proc macros that I can point to. Most of the details are really only documented through tutorials, like in the rust book or that workshops repo. Macros - The Rust Programming Language is shorter than the workshops, and you can just read it, so maybe it'd be worth a read? It's a canonical source for at least some of this information. It's also a beginner source, though, so maybe not.
As for actually writing what you want to write, I think you're close-ish, but you'll need a lot more machinery.
Next, you'll want to have some way to walk the AST, and then some way to generate and combine data from that. Walking the AST is relatively simple, it just requires looking into syn rustdocs, finding the data structure you have, and looking at its fields. Most things have iterators, and you can just iterate through the fields and whatnot.
An easier alternative to that is to use a helper crate. I'd recommend synstructure for easily walking through fields and collecting stuff. It gets rid of most or all of the boilerplate, so you just write how to turn a single item
Field { field_name: "name".to_owned(), field_type: "String".to_owned(), ... }
Lastly, you'll probably want to be constructing your result piece-by-piece, rather than trying to do it all at once. This applies whether you're using synstructure or manual iteration. I think the easiest way to do this is using ToTokens::to_tokens to append to an existing stream. For example,
let mut fields = proc_macro2::TokenStream::new();
(quote! { Field { field_name: #name, ... }, }).to_tokens(fields);
(quote! { Field { field_name: #name, ... }, }).to_tokens(fields);
let constructor = quote! {
Schema {
name: #schema_name,
origin: self,
fields: vec![ #fields ],
}
};
Took me forever to figure out how to do that nicely! There's probably a better way, but that should work for everything you need to be constructing.
Hopefully this gives you some tools to use! Let me know if there's anything else in particular that seems hard to figure out?
@daboross
Thanks, this gives me a better understanding of derive macro.
But since I have only recently begun to learn derive macro, my understanding of derive macro is very fragmented, and I can only write part of the code. For example, ToTokens you mentioned, I am very new to it.
I uploaded my code to github, here is my code repository: parse-struct
I wrote part of the code, but I don't know how to continue writing.
@bestia.dev
Although I did something similar to yours, there are still differences, and I encountered some problems.
The structure I need to parse is different from yours:
My logic only needs to implement the parse_schema method, which returns the parsed Schema.
extern crate proc_macro;
use crate::proc_macro::TokenStream;
use quote::quote;
use syn::{
Lit,
Meta,
Data,
Fields,
FieldsNamed,
DataStruct,
DeriveInput,
parse_macro_input
};
#[proc_macro_derive(ParseStruct, attributes(field_attr))]
pub fn parse_struct_derive(input: TokenStream) -> TokenStream {
let ast = parse_macro_input!(input as DeriveInput);
let name = &ast.ident;
let _fields = if let Data::Struct(DataStruct {
fields: Fields::Named(FieldsNamed { ref named, .. }),
..
}) = ast.data { named } else {
unimplemented!();
};
let builder_fields = _fields.iter().map(|f| {
let field_name = &f.ident;
let field_type = &f.ty;
let mut unique: bool = false;
let mut default: Option<String> = None;
let mut index: Option<String> = None;
let attribute = get_field_attr(&f);
match attribute.parse_meta() {
Ok(meta)=>{
if let Meta::NameValue(m) = meta {
if m.path.is_ident("unique") {
if let Lit::Bool(LitBool) = &m.lit {
unique = LitBool.value;
}
}
if m.path.is_ident("default") {
if let Lit::Str(LitStr) = &m.lit {
default = Some(LitStr.value());
}
}
if m.path.is_ident("index") {
if let Lit::Str(LitStr) = &m.lit {
index = Some(LitStr.value());
}
}
}
quote! {
::parse_struct::Field {
field_name: #field_name,
field_type: #field_type,
unique: #unique,
default: #default,
index: #index
}
}
},
Err(err)=> quote! { err }
}
});
let empty = Vec::new();
let gen = quote! {
impl ParseStruct for #name {
fn parse_schema() -> ::parse_struct::Schema {
::parse_struct::Schema {
name: #name,
fields: #builder_fields // get an error
}
}
}
};
gen.into()
}
fn get_field_attr(field: &syn::Field) -> &syn::Attribute {
let attrs: Vec<&syn::Attribute> = field
.attrs
.iter()
.filter(|at| at.path.is_ident("field_attr"))
.collect();
attrs[0]
}
I get an error:
the trait `quote::to_tokens::ToTokens` is not implemented for `std::iter::Map<syn::punctuated::Iter<'_, syn::data::Field>, [closure@parse_struct_derive/src/lib.rs:27:45: 66:6]>
I don't understand why this is, I hope you can help me.
The end result you're trying to construct is a token stream, a series of tokens like "if", "x" , "let" and so on. The error is saying that quote doesn't know how to turn your iterator into a list like that, basically. See Repetition; you want to do #(#builder_fields,)* to insert each field separated by commas. Of course, what you actually want is a vec and not just a series of fields so do vec![#(#builder_fields,)*].
Another reoccurring issue in the macro is that you're using tokens where you want strings. For example, name: #name will output name: User which isn't valid syntax. You want the result to be name: String::from("User"), or something like that. I'm not sure if there's some shortcut for doing so, but you could do
let name_string = name.to_string();
...
name: String::from(#name_string),
Same applies to the field name and type when you're generating Fields. Unfortunately they don't seem to have a to_string method, so I did it like this: let name_string = quote::quote! (#field_name).to_string();. Might be that there's an easier way but it works.
The conversion of the Attribte to Meta also fails. It seems that index=["@hash", "@count"] is not valid syntax for parsing into Meta, so you'll have to express it in some other way.
Inserting an Option into a quote also doesn't do what you're looking for. A Some(string) will be inserted as string and a None will be inserted as (). Again I'm not sure what the best way to tackle this is, but using quote::quote!(None) and quote::quote!(Some(lit_str.value())); seems to work.
Edit: Debugging proc macros can be pretty tricky. If you're not using cargo expand you should definitely give it a try. Additionally, eprintln!("some log message") messages will be printed in the compile process and can be used to help out in the similar way as regular debug prints.
I think you've created an iterator over token streams, and that itself isn't a token stream. You'll want to collect the iterator into a token stream first - with something like
let builder_fields = _fields.iter().map(|f| {
...
}).collect::<TokenStream>();
The error is from attrs[0]. Your code assumes that every field has a field_attr property and panicks when it tries to parse the age field because it has none. You'll need to handle fields without attributes, add them to age or remove the field. Doing so will cause a lot more errors, but they can all be fixed with what I said in the previous post.
I get a different error, did you change something else in your code? You should try cargo expand to see what the macro is producing and look for invalid syntax there to see where the issue might be (install with cargo install cargo-expand if you don't have it yet).
