Macros are Awesome (how about "extending" them, though?)

After several months of brain explosions, occuring each and every time I've tried to wrap my head around the macro_rules! - I'm beginning to see the light at the end of the tunnel. Macros might not be the most beautiful thing to write, especially when it comes to repeatable patterns (gosh, those $()* inside of other $()* can be a real headache - literally), but they're definitely one of the most powerful features of Rust.

Case in point: I'm currently experimenting with Actix (the web framework), trying to piece together a coherent structure that could be used as a solid backbone for a project I have in mind - and one of the things that are obviously needed, are templates. Askama looked promising, so I've decided to rely on it. Compile-time generation seems to be in line with the overall Rust's theme of power and efficiency. However, no matter how powerful it is, the amount of code to write, necessary to write before one is able to to use this power is also quite an important factor to consider.

And with Askama, it's not exactly "streamlined":

#[derive(Template)]
#[template(path = "page.html")]
pub struct Page {
    pub heading: String,
    pub text: String
}

If we're experimenting with "quick and dirty" ways to generate pages, it also makes sense to create an implementation for inline strings:

impl Page {
    pub fn new(heading: &str, text: &str) -> Self {
        Self {
            heading: heading.to_string(),
            text: text.to_string()
        }
    }
}

Some of them might not be &str, but regular String-s instead. Therefore, generics.

use std::string::ToString as S;

impl Page {
    pub fn new<A : S, B : S>(heading: A, text: B) -> Self {
        Self {
            heading: heading.to_string(),
            text: text.to_string()
        }
    }
}

Which is wonderful, but if we have to repeat this same thing for each and every template to use, to say it's going to be tedious is to say nothing.

Let's bring macro_rules! into the play, and suddenly all of the above becomes this:

templates![
    "page.html" => Page { heading, text }
];

Which is achieved by nothing less than a magic incantation of the sort:

macro_rules! templates {
    ( $( $path:expr => $struct_name:tt { $( $field:tt ),* } ),* ) => {

        $(
            #[derive(Template)]
            #[template(path = $path)]
            pub struct $struct_name {
                $(
                    pub $field: String
                ),*
            }

            #[allow(non_camel_case_types)]
            impl $struct_name {
                pub fn new< $($field : std::string::ToString),* >( $($field : $field),* ) -> Self {
                    Self {
                        $(
                            $field: $field.to_string()
                        ),*
                    }
                }
            }
        )*
    }
}

The fact that 1. I was able to write it - and 2. It actually works - makes me feel incredible. However, there is something I'm not completely satisfied with.

To account for several possible fields, I had to create generic parameters for all of them, using the names of the fields as types: < $($field : std::string::ToString),* >. It works just fine.

But it's not idiomatic in the slightest, and coupled with ( $($field : $field),* ) only adds to the confusion. Take the parameter, named "text", of the "text" type, and use it to generate the struct with its own "text" field filled with the value of the "text" parameter.

However, besides adding the #[allow(non_camel_case_types)] attribute macro, it doesn't appear there is much to do.

If there was a possibility, in macro definitions, to manipulate the tokens themselves - the symbols that comprise the strings which represent them - during the macro generation process, it could have been solved easily. Take the token, stringify it, modify that string (capitalize it?) and plug it back into code.

The first 2 steps are trivial: stringify!($field) does just that.

But there is no way to convert that string back into code.

Perhaps it's time to add codify! macro to do just that?

P.S. Another extremely useful thing could be "conditional" macros. The $()* allow to repeat the same process for more, than one time. But there's no way to evaluate whether something is true or false - whether inside the $()* there are no items, 1 item, or more than 1, as a trivial example.

P.P.S. Are there other ways to accomplish the things just mentioned, that I'm not aware of?

Small side note: Always use fully qualified namespaces when referencing types, i.e. ::std::string::String instead of String. It's quite simple to break macros, that don't do that. The same goes for calling methods, as well, sadly. One can implement a trait for a foreign type and the trait has a method that is already present in the type, i.e. suddenly <type>.method() syntax doesn't work anymore and you must use Type::method(<type>) to work around the naming conflict. Depending on the macro, you might be able to use use somewhere to get around the fully qualified namespace issue, but method calls are always prone to naming conflicts in macros.

Of course, if you do not intend to share the macro, it doesn't really matter, but projects grow and develop in uncertain ways, that may lead to breaking code, later on, even in a private project. No one wants to deal with a macro, that was written and last touched a year ago to fix bugs. :sweat_smile:

2 Likes
            #[allow(non_camel_case_types)]
            impl $struct_name {
-               pub fn new< $($field : std::string::ToString),* >( $($field : $field),* ) -> Self {
+               pub fn new( $($field : impl ::std::string::ToString),* ) -> Self {
                    Self {
                        $(
                            $field: $field.to_string()
                        ),*
                    }
                }
            }

Although such a macro (string literal -> tokens) could be implemented, its usability would be far below what one would expect, given how macro expansion work (e.g., codify!(stringify!(...)) wouldn't work, unless such case was hard-coded into the codify! macro (i.e. "input may be a string literal, or a the stringify identifier, followed by a ! punctuation, followed by a parenthesized group with an arbitrary stream of tokens).

That being said, a codify! working with the callback pattern could be useful:

  • expected usage:

    macro_rules! with_tokens { ... }
    codify!("..." => with_tokens!);
    

There are some tricks to achieve that:

const REPETITION_NON_EMPTY: bool =
    false $(|| { stringify!($thing); true })*
;
/// or even
const REPETITION_COUNT: usize =
    0 $(+ { stringify!($thing); 1 })*
;
  • (the stringify!(...); thing is to be able to "mention" the tokens our repetition refers to, while discarding in practice their usage).

Addendum

Regarding:

The ::paste::paste! (procedural) macro enables this:

// wrap the _whole_ expansion of your macro in an invocation to the `paste!` macro:
macro_rules foo {(
    ...
) => (::paste::paste!({
    // within the `paste` invocation, we can use the `[< ... >]` magic modifiers
    ...
    [< $field:camel >] // would expand to `Field`, if `$field` captured `field`
    ...
}))}
2 Likes

It sounds like you want to write a procedural macro. Macros-by-example are powerful, but sometimes not enough, especially if you need to match declarations or want to traverse/generate recursive structures. In these cases, having the entirety of Rust available is a blessing.

1 Like

Awesome, thank you. The last time I've tried to use impl-s as parametres I've received an error, saying that dynamic traits aren't supported the stable toolchain.

Agreed.

My goodness, now that's a brain twist. If this is ever going to end up in production environment without any comments, whoever is going to be debugging this is going to have quite a fun time.

It's perfectly valid and makes total sense, though. I'm stealing it - thank you.

I'm definitely looking further into it - this seems to be a very powerful tool, which doesn't go down all the way to ASTs and tokens directly.

1 Like

I've taken a look at them - and it's close, but the fact that I have to define a separate crate just to hold it turns me off. The whole point of this would be to engineer "quick and dirty" (more quick, less dirty, ideally) solutions right into the modules, and going through the hassle of separate declaration in a separate crate, which has to be marked for proc-macro only, is not something I'm willing to do at this point.

Well, we (and the language) can't really help with the fact that you are "not willing to" do one thing or another.

Automatic generation of trait implementations is done using proc-macros more often than not (that is the #[derive(…)] mechanism), exactly for the aforementioned reasons. A simple template- or substitution-based macro system inherently has its theoretical as well as practical limitations, and you have to accept that you are not going to be able to solve certain kinds of problems with it.

And I'm not asking anyone to help me with it - there are things that are practical, and there are things that are out of practical realm for the project I have in mind - in my personal, subjective opinion.

I accept it - and I've got no issue with it. I was looking for a simple, "plug and play" solution to what I had in mind - and I thank you personally, as well as the rest of the guys in this thread, who have helped me broaden my horizons, introducing a whole range of tools, built in directly in the language.

The paste! macro from the respective create seems to be perfect for what I have in mind.

Procedurals are powerful, but require too much additional hassle - once again, this is my personal, subjective opinion, based on what I have in mind and the amount of complexity I'm willing to deal with.

1 Like

if you are looking for quick & dirty inlined procedural macros, there is:

::inline_proc_macros

Afterwards, if you are pleased with the result you can upgrade to a full-featured proc-macro external crate.

2 Likes

Nice, thank you - that looks much more practical for my case.

Yeah, I've been there. I made a small utility crate cfg_aliases! and I really didn't want to incur or make other people incur the extra "guilt" of adding the large syn crate my dependencies for a little convenience. It was very confusing and took some ( fun! ) effort, but I only needed to do it once and with help on the forum I finally got a macro_rules! macro that did what I wanted and the end result was beautiful.

1 Like

I also think macros should be designed to be simpler, and keeping it simple is a precursor to a rapidly growing language

macro_rules seems to me about as simple as you can get while still being viable. It’s match for syntax: there’s a list of patterns, and the first one to match is selected. The macro output is ordinary rust syntax with some substitutions taken from the matching pattern.

The only complicated part is the big list of variable types, but I can’t think of another solution that makes things simpler overall. This system is surely better than having every macro duplicate the expression grammar, for instance.

Do you have any suggestions of changes that might make it simpler?

1 Like