Enum conversion using macros with attributes

help
1

I am following along this blog series for writing lexers and parsers Part Ten: Starting Again · Make A Language

I have two enums

#[derive(Debug, Copy, Clone, PartialEq, Logos)]
pub enum TokenKind {
    #[regex("[ \n]+")]
    Whitespace,

    #[regex(r"\$-?[_a-zA-Z][_a-zA-Z0-9-]*")]
    Variable,

    #[regex("-?[_a-zA-Z][_a-zA-Z0-9-]*")]
    Ident,

    #[token(":")]
    Colon,

    #[regex("[0-9]+")]
    Number,

    #[token("+")]
    Plus,

    #[token("-")]
    Minus,

    #[token("*")]
    Star,

    #[token("/")]
    Slash,

    #[token("=")]
    Equals,

    #[token("{")]
    LBrace,

    #[token("}")]
    RBrace,
}

and then a SyntaxKind with additional variants

#[derive(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, FromPrimitive, ToPrimitive)]
pub enum SyntaxKind {
    Whitespace,
    Variable,
    Ident,
    Colon,
    Number,
    Plus,
    Minus,
    Star,
    Slash,
    Equals,
    LBrace,
    RBrace,
    Error,
    Root,
    VariableDecl,
}

impl From<TokenKind> for SyntaxKind {
    fn from(token_kind: TokenKind) -> Self {
        match token_kind {
            TokenKind::Whitespace => Self::Whitespace,
            TokenKind::Variable => Self::Variable,
            TokenKind::Ident => Self::Ident,
            TokenKind::Colon => Self::Colon,
            TokenKind::Number => Self::Number,
            TokenKind::Plus => Self::Plus,
            TokenKind::Minus => Self::Minus,
            TokenKind::Star => Self::Star,
            TokenKind::Slash => Self::Slash,
            TokenKind::Equals => Self::Equals,
            TokenKind::LBrace => Self::LBrace,
            TokenKind::RBrace => Self::RBrace,
        }
    }
}

Now every time I add a new token type I have to also add it to SyntaxKind and the from implementation.

I know I can just write it out, but I am wondering if there is a convenient ways using macros to solve this?

2

If SyntaxKind is a superset of TokenKind, storing the TokenKind subset in a single variant might be more convenient?

enum SyntaxKind {
    Token(TokenKind),
    Error,
    Root,
    VariableDecl,
}

impl From<TokenKind> for SyntaxKind {
    fn from(token_kind: TokenKind) -> Self {
        Self::Token(token_kind)
    }
}

Otherwise I wouldn't use a macro to optimize a few lines of easily readable boilerplate code. The strum crate might have something that could help you though. I was thinking of crimes like Self::from_str(token_kind.as_ref()).unwrap() using strum::AsRefStr and strum::EnumString.

2 Likes