How to compile Rust AST at runtime?

help
1

I'm writing a small implementation of a custom domain-specific-ish language. I'd like to translate it to Rust and then produce an executable.

Now I could generate code as a string and call rustc or cargo as a subprocess, but where's the fun in that... So I want to generate an AST and compile it at runtime, which I think I can do using the rustc crate.

I found a guide that explains how to create an AST, but:

  • I will need to include a small "standard library" which I want to ship as code and parse into AST at runtime
  • in the end I want to compile the AST into a binary.

I didn't find any resource that shows a simple example of these two tasks. Does it exist?

4 Likes
2

You may want to look at compiler plugins, which after all do nothing else than generating AST nodes.

2 Likes
3

Yeah, but they don't have to care about compiling it afterwards...

1 Like
4

Just wondering: did you find an answer or workaround for this?

I was just looking into a project which could use this kind of functionality as well.

5

If you are OK with an earlier Rust version, you could try syntex, which is basically a clone of libsyntax (as of 1.0.0, IIRC).

6

I am also interested in a proper solution for this. Does rust perhaps have an internal library that takes an AST and compiles it?

7

Would be interested to see a library as well.

8

It sounds like you're trying to use rustc itself as a library.

I know C# has something similar where you can import Roselyn (their compiler) as just another library. You can do this in Rust as well by pulling in the rustc and rustc_driver crates (hidden behind the rustc_private feature flag) but it can be pretty finicky because you're trying to hook into a pretty sophisticated codebase and it wasn't designed with public consumption in mind. It's also completely unstable (the compiler is always evolving) so its not uncommon for your code to stop compiling if left alone for a couple months.

I remember contributing a page to The Rustc Guide about using rustc_driver to analyse crates (original issue). That may be of some help if you are wanting to play around with compiler internals.

2 Likes
9