Obtain expanded crate source from the build script

I would like to obtain an expanded version of the crate's source (i.e. the output of cargo expand) from build.rs. Are there any examples of how to do that?

(Recursively spawning another copy of cargo does not work, because it just keeps waiting on cargo-lock in the target directory).

It might not be possible in general, since cargo expand can easily use the build script output (some files written to OUT_DIR and then include!d, for example).

It looks like you aren't the only person that wants the ability to cargo expand a build script.

If I've understood the OP correctly, they want stuff the other way around: rather than expanding the build.rs script, they'd like to expand the whole crate under src/ for the build.rs to process.

In that regard, @0x387234, what you want is conceptually contradictory: you wan't a pre-processing step (the build.rs), to work off post-processed stuff (the expansion of the input code). The cargo-lock error is thus just a symptom of the problem.

There are two ways to see this, which lead to kind of the same solution:

  • either what you want to do is an actual post-processing step, and thus a build.rs script should not be used (at least not for the crate you want to cargo expand)

  • or the cargo expand input is not for the build.rs's current crate, but from an "earlier"/tangential-to-the-build.rs crate.

The difference between those two is whether the "output" of the .rs script is source code for an eventual compilation or something else as kind of a CLI tool.

  • For the latter option, this is thus kind of related to a post_build.rs wish that some people seem to want (maybe phrased / named differently), and which currently requires an external build-management tool, such as make, just, cargo xtask, etc.

From now on, I'm gonna assume that you want the former option.

The goal

For a more concrete goal, let's assume that you are sick of slapping #[derive(Debug)] on each struct / enum definition and that you'd like it to be done automagically.

Steps to achieve this

So you'd like to cargo expand -> add `#[derive(Debug)]` to each type's definition -> compile this. Since the middle layer is a preprocessing step for the last step, it can be the build.rs script of an actual crate whose input source code shall be the output of the build.rs. This means that the source code fed to cargo expand cannot be the official source code of the build.rs's crate; it would need to go on a separate helper crate.

  • Disclaimer

    This whole idea is kind of hacky, I wouldn't really recommend it for production.

So let's go with the following setup, to start with:

├── Cargo.toml
├── build.rs
├── pre/
│   ├── Cargo.toml -> ../Cargo.toml  # symlink (or ephemeral copy done by the `build.rs`)
│   └── src/
│       ├── lib.rs
│       └── ….rs
└── src/

The build pipeline would then be:

  1. the build.rs would go and basically, capture the output of (cd pre && cargo expand);

  2. it would then process that output (probably using syn (and proc-macro2 and quote))…

    • Back to concrete goal of slapping #[derive(Debug)]s on struct and enum definitions,

      we'd ::syn::parse_file() the output emitted by the previous command, and from there use the visitory pattern to append the #[derive(Debug)] to enums and structs:

      fn append_debugs (code: &'_ str)
        -> Result<impl ::core::fmt::Display>
      {
          use ::syn::{*, visit_mut::VisitMut};
      
          let mut file = ::syn::parse_file(code)?;
          struct DebugAppender;
          impl VisitMut for DebugAppender {
              fn visit_item_struct_mut (
                  self: &'_ mut DebugAppender,
                  struct_def: &'_ mut ItemStruct,
              )
              {
                  // subrecurse (in practice shouldn't be needed).
                  visit_mut::visit_item_struct_mut(self, struct_def);
      
                  struct_def.attrs.push(parse_quote!(
                      #[derive(::core::fmt::Debug)]
                  ));
              }
      
              fn visit_item_enum_mut (
                  self: &'_ mut DebugAppender,
                  enum_def: &'_ mut ItemEnum,
              )
              {
                  // subrecurse (in practice shouldn't be needed).
                  visit_mut::visit_item_enum_mut(self, enum_def);
      
                  enum_def.attrs.push(parse_quote!(
                      #[derive(::core::fmt::Debug)]
                  ));
              }
          }
          DebugAppender.visit_file_mut(&mut file);
          Ok(::quote::quote!( #file ))
      }
      
  3. … and emit the generated output to src/lib.rs.

Finally, a controversial extra step would be to try and hide these shenanigans a bit:

  1. make the Cargo.toml point to ./lib.rs rather than the default src/lib.rs

    [lib]
    path = "lib.rs"
    
  2. make the build.rs spit the code into ./lib.rs,

  3. and finally renaming pre/ to src/:

    .
    ├── Cargo.toml
    ├── build.rs
    └── src/
        ├── Cargo.toml -> ../Cargo.toml  # symlink (or ephemeral copy done by thr `build.rs`)
        ├── lib.rs
        └── ….rs
    
2 Likes

@Yandros yeah something like that, thanks!