How do I modify rust code in AST level and pass to compiler?

Hi all,

I'm trying to auto-modify when given a crate, with just simple command without manually touching original rust code. So for sample crate A, modification would be done with some simple command. After modification, I expect other crate B would be built with modified version of A, assuming that B has dependency with local path of A.

For now, I'd like to change visibility of modules, items and fields in entire crate. Since modifying rust source code would be error-prone, I decided to do modification on AST level.

I can produce AST on given crate with unpretty=ast-tree option. Also, I guess modifying visibility on AST would be easy, helped by syn::visit_mut. However, what I don't know is... how can I pass this modified AST to compiler? So that compiler do later step with modified AST...

Yes I know, there is proc macros for working on AST. I also tried to adjust proc macro to entire crate with #![feature(extern_absolute_paths)], but it seems not work.

Any ideas on this...?

Well, you could do what you would do when writing a proc-macro (parse the code using syn and emit the new code using quote), except that you wouldn't actually write a proc-macro. You would instead write e.g. a build script that goes through every affected file and performs the transformation explicitly. This isn't pretty, but it would be particularly easy to transform into a proper proc-macro once the bug you linked to is fixed.

Thanks for your reply.

However, AFAIK, build script runs right before the crate is built. So I'm afraid it's not possible to get AST from original source code.

Furthermore, since transformation in AST would not affect original source code, we need to pass modified AST to compiler. I'm not sure whether it's possible on build script as well...

I don't understand what you mean by this. By the time you start building your crate, the source code (that you have written) must be there, right? Otherwise, there's nothing to build. Or, to put it differently, macro expansion also runs at the beginning of the build process, it's not very different from a build script in this regard.

What I'm proposing is that you do generate new files (containing the modified code) to be compiled. It's 100% possible.

Sorry, I was ambiguous at that memoment.

So let's say we have dep crate and child crate. Those crates are not written by me, so I would not assume anything on them. This only assumption is that child has dependency of dep by local path. (well, actually, I'll modify manifest of child for this)

In this case, I'd like to do some transformation to dep crate. Of course, we can do some parsing on rust code and do some transformation on it. But this is too error-prone, so I decided to work on AST, rather than source code.

What I'd came up with was... add attribute macro on top of root of dep. This macro would gets entire AST of dep crate, and do transformation, pass modified AST to compiler. Note that I need to slightly add some lines on lib.rs or main.rs in this scenario.

Above method is does not supported yet. Rather than this, you suggested to add build script. Cool, this would be added on dep/. But my questions was - how do I get entire AST of dep, and how do I pass modified AST to compiler?

AFAIK the only way is unpretty=ast-tree, which you already mentioned. I don't think there's a programmatic way since the actual AST implementation is not guaranteed to be stable. Even unpretty=ast-tree should only be used for debug purposes.

AFAIK the compiler doesn't accept inputs different than rust source code, so you'll have to either translate the AST back to rust source code (which may not be possible since some things can't be expressed in source code, like hygiene), or modify the rust compiler to make it do what you want.