Hi all — sharing a project that might be useful to folks here, since "is there a maintained ANTLR for Rust?" comes up from time to time.
antlr-rust-runtime (crate) is a pure-Rust implementation of the ANTLR v4 runtime, plus a .g4 → Rust code generator. No JVM or ANTLR jar is needed at build time, and it has just two runtime dependencies (memchr, thiserror).
Highlights:
Passes the official ANTLR runtime conformance suite.
Typed parse trees + listener/visitor, with semantic predicates and actions wired through a SemanticHooks trait.
Performance-focused. On the cross-runtime MySQL benchmark (Oracle's grammar), run on a single machine, it edges out the C++ runtime overall and is ~2.4× faster than the TypeScript runtime — methodology and numbers are in this PR (single-box comparison, so treat it as indicative rather than a leaderboard).
It's already used in practice (a generated Gremlin parser; an Oracle-MySQL grammar port), but still pre-1.0 — so feedback, grammars that break it, and performance reports are all very welcome. Happy to field ANTLR-in-Rust questions here too.
This is very interesting but, can I ask you how much code was written by LLM?
It may seem a rude question but let me explain: the problem I've noticed lately is that writing lots of code with the help of LLMs is easy while maintaining it is a very different story. I know firsthand how much work it takes to maintain a library (I am the original author of psycopg2, and I maintained it for about 10 years before giving it away to the actual maintainer). My impression, which may well be wrong, is that a lot of developers using LLMs don't fully appreciate that and abandon the code after some months. It is not that i dislike LLMs, it is just that I don't want to depend on libraries that end up being abandoned or poorly maintained and the "how much" question is the only metric I have right now.l
Not rude at all - it's the right question to ask, and coming from psycopg2's author it carries real weight (that's a library a huge slice of the Python world relied on for years).
Honest answer: if "written" means physically typed, then most of this code is LLM-written. What I bring is ~20 years of software engineering (currently at one of the big-tech companies) applied to directing it - designing the testing framework, sourcing the conformance fixtures, and running competing implementation/optimization experiments to keep the best result. As a code writer, I think frontier models are already better than most of us; the leverage is in knowing what to build and how to verify it.
You're right that a dependency is always a bet on future maintenance. But I'd argue this case cuts the other way. A Rust ANTLR runtime is exactly the kind of large, thankless, complex effort where the human-written attempts have historically stalled - the most widely-used one sat unmaintained for years. That's the gap I'm trying to close.
Two things make me more comfortable here, not less:
It's the only Rust ANTLR runtime I'm aware of with a committed, reproducible performance benchmark - so "did this change regress?" is answerable by anyone, not a vibe.
It passes ANTLR's official runtime conformance suite, so correctness has an objective gate on every commit.
For a project like this I've come to see "LLM-written, conformance-tested" as a better maintenance warranty than "one person's spare evenings." When issue #777 comes in, prompting a steadily-improving frontier model against a full test suite and a benchmark is a more durable loop than hoping any single maintainer finds unpaid time to debug parser-generator internals. The tests are the warranty; the author being a model I can re-invoke is the feature, not the risk.
And the strongest maintenance signal I can offer: I didn't build this in a vacuum - it's the parsing engine under mehen, a code-metrics tool I actively develop and depend on (it drives its Java and Kotlin analyzers). So the runtime isn't a weekend curiosity I'll wander away from; it stays maintained because something I rely on breaks if it doesn't. (There's a nice irony there, too - mehen exists partly to flag low-quality AI-generated code, and it runs on an AI-written parser that has to pass conformance to ship.)
Skepticism is fair, of course - but that's the bet, and the conformance suite + benchmark are there precisely so it's checkable rather than "trust me."
Mainly lineage and focus. antlr4rust and its forks (the revived one, dbt's dbt-antlr4) all descend from the 2020 codebase; this is a clean-room rewrite built around two things - ergonomics and performance.
DX: two dependencies (memchr, thiserror), no JVM or jar at build time. Tree traversal is idiomatic Rust iterators (children(), descendants(), tokens()) rather than manual walking; visitor support is in progress (#138).
Performance: there's a committed benchmark in the repo, and it's in the shared cross-runtime suite (PR merged) - on the MySQL grammar it edges out the official C++ runtime overall (C++ still wins parse-only; we win on lexing) and is ~2.4× faster than TS. Recently added SIMD lexer scanning (#120, #127 ), more perf work ongoing. All of it sits on top of the official conformance suite.
Not a knock on antlr4rust - it's capable and maintained again. Just a different bet: minimal-dependency pure Rust, with performance I can actually point at.
No JVM and Java at build time, that I understand, and it's always better if we can stay in the same ecosystem. On the flip side, your project is disconnected from the original ANTLR one, but I'm not sure Parr will keep updating it; he seems to have changed his research entirely. He also seemed pretty much against re-integrating a Rust version into the main project, which I understand too. But that must mean there's more than a runtime? That's what puzzled me; are you keeping the generator in the same crate as the runtime?
I assume "SIMD lexer scanning" is the optimization of UTF characters decoding? Not sure there's much parallelism to extract from the lexer itself, since it's a state machine.
Anyway, good to see ANTLR is still appreciated. For me, decoupling the grammar from the source code is vastly superior to other systems that mimic Yacc and mix both.
Thanks @blonk — yes: developer experience, and TypeScript.
On re-integration: agreed it's unlikely, and I'm not chasing it - the reasoning is on #13. But I'd push back on "disconnected." It runs against ANTLR's official conformance suite on every PR, so it tracks the reference runtimes' behavior whether or not it ever lives in their repo - arguably a tighter coupling than a fork that drifts.
And yes, there's more than a runtime, all in one crate: the runtime is the library, antlr4-rust-gen s a binary beside it. Today it consumes the .interp (serialized ATN) the ANTLR tool emits, which is the one spot Java shows up - run once, not part of cargo build. Removing even that is the plan: #141 is about parsing .g4 directly and reimplementing that slice of the tool in Rust.
On SIMD you're right, and that's the caveat. A general DFA step is serial - each state depends on the last - and it isn't UTF decoding. What it exploits is self-loops: a lexer sits in one state across a whole identifier, whitespace run, or comment body, and while parked there the next state is constant until a byte leaves the class. "Find the first byte not in this range" is a vectorized scan, so you skip the run instead of stepping byte-by-byte. The parallelism is in the run length, not the state machine - and on real code that's most of the characters. Details in #79
On ANTLR itself - this is exactly why. I needed to parse Kotlin for mehen (code metrics over 1000+ file codebases), tried the tree-sitter Kotlin grammars, and the best I found covered ~60% of the semantics. Meanwhile JetBrains publishes the official Kotlin spec as an ANTLR grammar - decoupled from any implementation, exactly your point. The missing piece was a Rust runtime fast enough to run it across a whole codebase. That gap is the whole reason this project exists. And Kotlin's not alone - the official-grammar list is long enough that I started curating Awesome ANTLR to point at the good ones.
How much was actually written by LLM is not the important metric. The correct one is how much code was reviewed and understood by human. If no one understand the code (and LLMs could't do that by definition) then it's already dead.
You may say that you don't need to understand that code now, but would understand it later, when it would be needed, but you would be fooling yourself: if you couldn't even find time now, when there are no time pressure then chances of finding time later is closer to zero.
Frontier models are still the same stockastic parrots they always were. They may write amazing 1000 lines of code and then inject absolutely stupid error that no one would ever do after half-year of experience right in the middle of it. And then it tries to patch it in the entirely different place — precisely because LLM doesn't understand (and doesn't try to understand) what code does, it works with “feelings”: how similar code worked in other projects that it saw in its training set.
It doesn't work like that with LLMs. Precisely because they don't understand anything and rely on tests they tend to introduce convoluted and sometimes outright crazy “solutions” that, nonetheless, make tests pass. If no one looks on the generated code at some point they overwhlem the ability of tests to keep code in somewhat coherent state and if no one one stop them at this exact point then you continue to build castles on a beach. When problem is discivered you end up with another abandoned project.
Situation with ANTRL is a bit different because it doesn't change much and thus there are a chance that natural bitrot of anything AI-generated wouldn't destroy the codebase quickly… but this also negates the “other crates are not updated” argument: if we rely on specification staying essentially unmodified and code staying unchanging then there are nothing wrong with using code that's not upgraded in last N years, too.
I'm just curious, if you can disclose, how much it will cost to give AI to rewrite an app? I have several old apps < 1 million lines of code. There is a full spec for the apps, and if I right understand, I can feed the specs to AI and get the app decently coded. So problem can be only in the cost.
@khimru Fair - and I won't argue AI in the abstract with you; how well it works genuinely varies by domain and by how it's driven. Let me answer for this specific case, because it wasn't blank-page generation.
ANTLR is unusually good ground for it. It's a decade old and deep in every frontier model's pretraining, so the architecture - ATN, ALL(*), DFA caching - is something these models reason about and explain, not pattern-match blindly. But that's not the argument; the setup is. The 20 years I mentioned went into a validation model built to catch exactly the failure you name - the plausible error dropped in the middle:
The oracle isn't mine to edit. ANTLR's official conformance suite is third-party expected outputs. On top of it, generated parsers for Kotlin, TypeScript, JavaScript, Java and Trino SQL are diffed tree-for-tree against the reference ANTLR runtimes (Java, Python, Go). Convoluted code doesn't help you reproduce an oracle you don't own - only correctness does.
Clean-room, and the direction matters. The template that renders ANTLR's conformance descriptors into Rust was authored blind - modeled on an ideal Rust runtime, without reading the actual code - and then the runtime was adapted to match that ideal, not the reverse. You can't accuse code of being contorted to pass a test it authored when the reference was written without seeing it. (design notes)
Quality watched, not assumed. Strict Clippy (nursery + pedantic, warnings-as-errors) and copy/paste detection on every PR; three independent AI reviewers per PR; the Rust itself watched for complexity by mehen.
Exercised against real history. Those generated Java/Kotlin parsers run inside mehen across ~1000 real metric snapshots I've kept since the previous parser - regressions surface against real codebases, not toy inputs.
Can a bug still slip through? Of course - it slips past the best humans too. The difference isn't origin, it's the net, and mine is public. The honest test is whether bugs get root-caused or papered over - that's checkable in the issue tracker, not something you have to take on faith.
Thank you very much for your complete and honest answer.
I agree with @khimru when he says «The correct one is how much code was reviewed and understood by human.» (and that's also why I don't particularly like TDD, it's too easy to have a spaghetti monster that passes all tests, but I am digressing...) so I'll probably have a look at the source myself and try to understand that before using it in any project.
Said that, I always loved ANTLR and one more library to work with it in Rust is good. Thank you again.
