Trying to write parser, experiencing huge compile times


#1

Hello. I got myself into Rust recently and started my first project: parser for a scripting language.

I found a parser combinators library and was very excited, however after first successful steps compile times for my simple parser, which can currently only parse include statements and class declarations, are over 5 minutes.

I heard Rust has big compile times, but I didn’t expect to experience several minutes long at this stage. I’m afraid implementing more grammar will increase them even more.

I’m using macros extensively, since lack of anonymous return types, closure cloning and inference outside of function bodies makes it very painful, if not impossible at some times, to write composable parsers.
Here is my code.
I’m wondering if I’m doing it horribly wrong. What is the idiomatic way to write a simple parser in Rust? Are the compile times I get normal, will they be much bigger if I continue to implement parser this way?

Thank you in advance :flushed:


#2

Which library are you using? I’m working on a parser combinator library myself and ran into the same problem. Once a parser gets to a certain complexity, the compile time exponentially increases. I can’t tell from your gist alone if it’s the exact same problem, but I’m guessing it is.

I actually just posted a topic about it a couple days ago, and it appears to be known issue.

I don’t know if the library you’re using has anything built-in to handle it, but the way I’m currently getting around it in my own library is by splitting up large parsers into chunks and boxing each chunk into a trait object. It’s not a great solution because it adds unnecessary dynamic dispatch, but so far it’s the only way I’ve found to reduce the compile time without drastically changing the code.

I’m also writing a simple language, and using this method I got the compile time down from many minutes to a few seconds.


#3

Thanks for reply. I’m using parser_combinators crate.

Checked links you provided and found alternative method, using separate fn’s for each parser. I guess macros was not a good idea.


#4

My initial guess was that your macros were expanding into some monstrous source, but -Z unstable-options --pretty=expanded shows they are not. Therefore, I’d call this a compiler bug.