Amount of nesting in pest parsing

Hi people,
I have used pest to define the following grammar:

space_sep = @{ (" "+ | "\t"+) }

range = @{ ASCII_DIGIT+ ~ ("-" | ":") ~ ASCII_DIGIT+ }
numlist_element = @{ (range | ASCII_DIGIT+) }

numlist = @{ (numlist_element ~ (",")?)+ }
namelist = @{ (ASCII_ALPHA+ ~ (",")?)+ }

float = @{ (ASCII_DIGIT+ | "." ~ ASCII_DIGIT+ | ASCII_DIGIT+ ~ ".") }
sphere_values = @{ ASCII_DIGIT+ ~ space_sep ~ float }

id_str = @{ ^"id" | ^"resid" }
name_str = @{ ^"name" | ^"resn" | ^"resname" }
sphere_str = @{ ^"sphere" | ^"s" | ^"ressphere" | ^"rs" }

idsel = @{ id_str ~ space_sep ~ numlist }
namesel = @{ name_str ~ space_sep ~ namelist }
sphere = @{ sphere_str ~ space_sep ~ sphere_values }

selection = { (idsel | namesel | sphere) }
conjunction = { (("and" | "&") | ("or" | "|")) }

selection_chain = { SOI ~ selection ~ (conjunction ~ selection)* ~ EOI }

WHITESPACE = _{ " " }

When I match this on a string, it will result in an iterator of Pairs as expected.
The following input

    let success = SelectionParser::parse(Rule::selection_chain, "s 2589 .5 or id 1")
        .unwrap()
        .next()
        .unwrap()
        .into_inner();

Results in:

[
    Pair {
        rule: selection,
        span: Span {
            str: "s 2589 .5",
            start: 0,
            end: 9,
        },
        inner: [
            Pair {
                rule: sphere,
                span: Span {
                    str: "s 2589 .5",
                    start: 0,
                    end: 9,
                },
                inner: [],
            },
        ],
    },
    Pair {
        rule: conjunction,
        span: Span {
            str: "or",
            start: 10,
            end: 12,
        },
        inner: [],
    },
    Pair {
        rule: selection,
        span: Span {
            str: "id 1",
            start: 13,
            end: 17,
        },
        inner: [
            Pair {
                rule: idsel,
                span: Span {
                    str: "id 1",
                    start: 13,
                    end: 17,
                },
                inner: [],
            },
        ],
    },
    Pair {
        rule: EOI,
        span: Span {
            str: "",
            start: 17,
            end: 17,
        },
        inner: [],
    },
]

Now I'm wondering why the Pairs are not nested further. As can be seen, the inner field of the sphere and idsel rules contain empty vecs even though they are defined in terms of nested rules in my grammar. I would have liked to use the further nested rules to parse the results into user-defined structs. With output like this I'd have to take the strings and "reparse" them which seems to defeat the purpose of a dedicated parsing library.
Am I missing something here?

From the Pest Book:

pest has two kinds of atomic rules: atomic and compound atomic . To make one, write the sigil before the left curly bracket { .

atomic = @{ ... }
compound_atomic = ${ ... }

Both kinds of atomic rule prevent implicit whitespace: inside an atomic rule, the tilde ~ means "immediately followed by", and repetition operators (asterisk * and plus sign + ) have no implicit separation. In addition, all other rules called from an atomic rule are also treated as atomic.

The difference between the two is how they produce tokens for inner rules. In an atomic rule, interior matching rules are silent. By contrast, compound atomic rules produce inner tokens as normal.

So you probably meant to declare most of your rules with a leading $ rather than @.

Oh dear, I somehow managed to not read that part. Thanks for the clarification!

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.