What to use for code generation?

I am writing crate that will take some declaration file and generate Rust code from that during compilation.

What are the ways to do it?

So far I've been thinking about:

  1. Macro. Willwork, but not most pleasant to use.
  2. Generating string manually without using anything. Not ideal.
  3. Codegen (https://docs.rs/codegen/0.1.3/codegen/). So far this looks like best option.

Is there something better?

1 Like

I haven't done any codegen in Rust yet (but in other environments). Therefore no clear recommendation, and just some thoughts:

A benefit of 2) compared to 3) could occur if 3) doesn't implement a particular feature that you need in your generated code. Just writing strings is flexible enough to capture everything (but obviously more error-prone).

If you go for 2), maybe run rustfmt after the code generation. Then it should look rather decent and you don't have to care too much about getting formatting good while generating strings (this can be rather hard with nested indents).

Yet another option is to use a templating framework. Then define the templates for stuff you want to generate outside of code and populate them during code generation.

I've used both pure string writing and template frameworks in the past. Both worked, and every time I used one I thought "maybe the other would be better" :slight_smile:

1 Like

I have occasionally done a combination of (1) and (2) with reasonable success. In particular, I was writing a recursive-descent parser and used a custom program to generate the parse table as a macro_rules! definition. A separate hand-written macro output individual parsing functions based on the table and user-specified actions.

I'm by no means an expert, but I have used the quote and syn crates for this. The learning curve might be a bit steep, but it's really flexible.

Basically, you build up your syntax with the quote! macro. This gives you a TokenStream which can then be turned into a string.

1 Like

quote! is a great tool to generate Rust source code from within Rust itself. It will generate a ::proc_macro2::TokenStream, which can then be converted .into() a String:

//! lib.rs
trait IsArray : ::core::ops::DerefMut<Target = [Item]> {
    type Item;
    const LEN: usize;
}

include! {
    concat!(env!("OUT_DIR"), "/generated.rs")) // impl<T> IsArray for [T; 0, 1, ... ]
}

#[allow(nonstandard_style)]
fn assert_IsArray<T : IsArray> ()
{}

fn with<Item> ()
{
    let _ = assert_IsArray::<[Item; 0]>;
    let _ = assert_IsArray::<[Item; 1]>;
    let _ = assert_IsArray::<[Item; 2]>;
}
//! build.rs
use ::quote::quote;

fn main ()
{
    let mut code = quote!();
    #[allow(nonstandard_style)]
    for N in 0 .. 1024_usize {
        code.extend(quote! {
            impl<Item> IsArray for [Item; #N] {
                type Item = Item;
                const LEN: usize = #N;
            }
        });
    }
    use ::std::{env, path::PathBuf};
    let ref mut out = PathBuf::from(env::var("OUT_DIR").unwrap());
    out.push("generated.rs");
    ::std::fs::write(out, code.to_string())
        .unwrap_or_else(|err| panic!(
            "Failed to write generated code to `{}`: {}",
            out.display(), err,
        ))
    ;
}

It may not look better than basic string interpolation, but it interacts quite nicely with ::syn, a crate which lets you describe Rust syntactic constructs, such as trait(definition)s, trait impls, structs, functions, in a high-level / programatic way similar to the ::codegen crate you linked.

So you can have define a let struct_def: syn::ItemStruct = ..., and then just do #struct_def from inside quote! to interpolate it.

2 Likes

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.