How to benchmark Procedural Macros?

help
1

I want to benchmark derive macros, both in synthetic and real use cases.

By "real", I mean measuring how quickly a basic Rust program (including procedural macros) binary compile.

cargo build --timings command could be use, is there a specialized tool or method specifically designed for benchmarking procedural macros (derive macros) ?

2

Most proc-macros crates (including serde_derive) don't have any benchmark,
How do they measure any performance improvement ? or regression ?

3

I think you should destinguish between

  1. the compile time of the proc macro itself (maybe split into dependencies and self),
  2. the runtime of the proc macro for a given input
  3. and the compile time of the generated code.

The first is only relevant for fresh builds while the latter is also impacting incremental builds. I suspect that the compile time of the generated code dominates the runtime of the proc macro in most cases since most time is spend in LLVM.

4

Sure:

  1. the runtime of the proc macro for a given input

This is why synthetic benchmark is for, while synthetic benchmarks are useful, they don't always reflect the complete picture.

  1. and the compile time of the generated code

You are right, I am not interested in full compile time, But only compile time of generated code by proc-macro, Any idea or example how to do that ?

5

trybuild is specifically for testing whether small pieces of code (that use proc macros, even!) compile successfully or not. It's not meant for benchmarking, but you could time its whole run, or take inspiration from its code as to how to set up the build.

1 Like