Proc macro parsing groups

I am trying to parse something similar to SQL syntax and check column names on compile time.
my syntax is select Foo::{a,b} from {Foo}
The problem is that syn crate doesn't seem to provide any utilities to parse this in these cases what do people use? The easiest method seems to be converting the query into a string and using regex on it

Ps. Perhaps using <> instead of {} is a better idea in case to be able to use format macro with the output

This is because the syn crate is intended to parse (snippets of) Rust code, which this isn't.

One viable option is to consider the input TokenStream as if it is the output of a lexer, and then use that TokenStream as an input for a custom-written parser. It's likely to be a fair amount of work though, so be prepared for that.

Yeah... Assuming you want to be able to do stuff like nested queries, that's not going to work for the same reason you can't parse HTML reliably with regexes: they're the wrong tool for the job. In particular, you need a more powerful version of state machine that adds a stack. That model is known as a pushdown automaton (PDA), of which parsers are a good example.

1 Like

the thing is that my syntax is very similar to the UseGroup which is rust syntax and somehow the syn crate parses it but Parse is not implemented for it

From what I can see only the parts in curly braces resemble a UseGroup. So if that's all that is in the input TokenStream, then syn could parse it. But in the select snippet you gave, the code around it prevents that. Syn doesn't scan the input looking for parts it can parse. It tries to parse the input in its entirety, and it either succeeds or fails.

3 Likes

You'd implement Parse yourself:

#[allow(unused_imports)] // typical / pervasive syn imports
use ::syn::{*,
    parse::{Parse, Parser, ParseStream},
    punctuated::Punctuated,
    spanned::Spanned,
    Result, // explicitly shadow it
};

mod kw {
    ::syn::custom_keyword!(select);
    ::syn::custom_keyword!(from);
}

struct Input {
    _select: kw::select,
    base: Ident,
    elems: Punctuated<Ident, Token![,]>,
    _from: kw::from,
    origin: Ident,
}

/// `select Foo::{a,b} from {Foo}`
impl Parse for Input {
    fn parse (input: ParseStream<'_>)
      -> Result<Self>
    {
        Ok(Self {
            _select: input.parse()?,
            base: input.parse()?,
            elems: {
                let _: Token![::] = input.parse()?;
                let sub_input; braced!(sub_input in input);
                Punctuated::parse_terminated(&sub_input)?
            },
            _from: input.parse()?,
            origin: {
                let sub_input; braced!(sub_input in input);
                sub_input.parse()?
            },
        })
    }
}

fn main ()
{
    println!("{:#?}", ::syn::parse2::<Input>(::quote::quote!(
        select Foo::{a,b} from {Foo}
    )))
}

for instance, yields:

Ok(
    Input {
        _select: Keyword [select],
        base: Ident {
            sym: Foo,
        },
        elems: [
            Ident {
                sym: a,
            },
            Comma,
            Ident {
                sym: b,
            },
        ],
        _from: Keyword [from],
        origin: Ident {
            sym: Foo,
        },
    },
)
1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.