I am trying to write a functional macro that part of its feature is
to parse code matching in such style: pub(crate) var:var_type; var2:var2_type;... in a trait, there could be as many vars as possible, and no need to be at the same line.
the core code is like this:
/// Define the enum to represent different kinds of trait variable types.
/*↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓*/
struct TraitVarType {
name: TokenStream, // the whole type name, including generics, like `HashMap<K, V>`, `i32`, `T`, etc.
generics: Vec<String>, // the generic type elements in the trait type, like `K, V` in `HashMap<K, V>`
}
impl ToTokens for TraitVarType {
fn to_tokens(&self, tokens: &mut TokenStream) {
tokens.extend(self.name.clone());
}
}
impl Parse for TraitVarType {
fn parse(input: ParseStream) -> syn::Result<Self> {
let mut name = TokenStream::new();
let mut generics = Vec::new();
// 1. Parse until a semicolon is found, indicating the end of a type definition
while !input.is_empty() {
if input.peek(Token![;]) {
println!("the EXIT token is: `{}`", input.parse::<TokenTree>()?);
break;
}
let token = input.parse::<TokenTree>()?;
println!("the token is:{}", token.to_string());
name.extend(Some(token));
}
println!("finally， the name is:{}", name.to_string());
// 2. Parse generics
if let Ok(type_parsed) = syn::parse2::<Type>(name.clone().into()) {
let mut visitor = GenericTypeVisitor {
generics: Vec::new(),
};
visitor.visit_type(&type_parsed);
generics.extend(visitor.generics);
}
// 3. Return
Ok(TraitVarType { name, generics })
}
}
#[test]
fn test_trait_var_type() {
let raw_code = quote! { Vec<T, HashMap<K, V>>; x}; // `x` is put on purpose
println!("the raw code is:`{}`", raw_code);
let parsed = parse2::<TraitVarType>(raw_code.clone()).expect("Failed to parse");
println!("the raw code is:`{}`", raw_code);
assert_eq!(
parsed.name.to_string(),
"Vec < T , HashMap < K , V >>".to_string()
);
assert_eq!(
parsed.generics,
vec!["T".to_string(), "K".to_string(), "V".to_string()]
);
}
/*↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑*/
/// Define the struct to represent a single trait variable field.
struct TraitVarField {
var_vis: Visibility,
var_name: Ident,
type_name: TraitVarType,
}
impl Parse for TraitVarField {
fn parse(input: ParseStream) -> syn::Result<Self> {
println!("the orig input is:{}", input);
let var_vis: Visibility = input.parse().expect("Failed to Parse to `var_vis`");
println!("the input is:{}", input);
let var_name: Ident = input.parse().expect("Failed to Parse to `var_name`");
println!("the input is:{}", input);
let _: Token![:] = input.parse().expect("Failed to Parse to `:`");
println!("the input before TraitVarType is:{}", input);
let type_name: TraitVarType = input.parse().expect("Failed to Parse to `type_name`");
println!("the input after TraitVarType is:{}", input);
let _: Token![;] = input.parse().expect("Failed to Parse to `;`");
Ok(TraitVarField {
var_vis,
var_name,
type_name,
})
}
}
The struct TraitVarField is for handling each var definition, and TraitVarType
is specific for parsing attributes of the type.
The core issue is related to TraitVarType when running with test_trait_var_type test, it panics with msg thread 'test_trait_var_type' panicked at src\lib.rs:111:59: Failed to parse: Error("unexpected token"),
which is caused by line let parsed = parse2::<TraitVarType>(raw_code.clone()).expect("Failed to parse"); ---simply speaking, the issue is due to it hits x after ;, which is not expected. But practially,
there wound be tokens that should NOT be parsed in this parse() invocation (just like what the obj of TraitVarType is used in impl Parse for TraitVarField block).
So, what should I do to fix this issue with Syn V2.0?
(Ps: for simplicity, I didn't put the GenericTypeVisitor code here,
since it's not the root cause of the issue. I've tested it with no error)