Macro with side effects

Hi all,

As far as I understand proc macros are intended to be pure. Partly because we want our builds to be deterministic, then there are plans to run the macros inside a wasm runtime, etc.

My question is that then what would be the correct way to generate non-code files during build?

The build scripts solve the issue of generating input for the compiler based on external state, such as a protobuf definition. I'm looking for the opposite, to generate extra output. Augmenting serde to generate definition files for typescript, protobuf, whatever.
I'd have imagined to generate them from a proc macro, but that is discouraged. The build script runs before the code is actually compiled, so it can't access the rust code it should generate bindings for.
I also looked into compiler plugins, no luck there either.

The ts_rs crate works around this by generating code that can generate the files during runtime, and suggests to run the code generation during test runs, which is a pretty ugly hack, and the generated content winds up in the final binary too, bloating it.

I understand the challenges that are caused by partial/incremental compilation and that generated codes should be idempotent, but I think those issues can be solved, once there is a solution to generate output artifacts from a source file.

Does this post by @matklad help?

Well not really... This is also a similar ha k to what the ts_rs create that I mentioned does, but also relies on the test to process the file rather than doing it in a macro.

I'm not sure what you mean by this – you can absolutely access the code from your build script. You can discover all *.rs files, for example, and parse them using syn just like you would in a procedural macro.

1 Like

See GitHub - 1Password/typeshare: Typeshare is the ultimate tool for synchronizing your type definitions between Rust and other languages for seamless FFI. for an example of this approach. It serves the same purpose as ts_rs but instead of a proc macro that generates the necessary code inside your project, it's a CLI that searches your project's .rs files for type definitions annotated with #[typeshare], parses them and generates the type definitions without compiling the project itself.

2 Likes

Hmm that's interesting. It would mean the code is parsed twice though. I wonder how much it would affect build times if this would be invoked from a build script. It would mean the generated bindings would be overwritten even if the code does not compile in the end though.

Negligibly. Parsing is almost completely insignificant compared to everything else the compiler does (notably, type checking and code optimization).

3 Likes

Gotcha this makes sense. This wouldn't cover types generated from macros though, would it? Or dependent crates?