Is macro_attribute execution order stable?

#1

I have some attributes like this:

#[parent]
#[name(value="Mike")]
#[age(value=30)]
fn main(){
   #[child]
   #[name(value="Nick")]
   fn test(){
   }
}

In my tests the #[parent] always can access the data of the inner attributes but I'm not sure if the execution order will always be the same.

#2

Hi @Neo-Ciber94 :wave: and welcome :slightly_smiling_face:

The order between parent, name, and age on main() is not guaranteed, but it is guaranteed that child and the inner name on test will only be evaluated if they still exist after the outer attributes' passes.

This is indeed a useful feature to generate "fake" attributes, and thus get ways to modify code in a way that normal attributes can't:

  • you can "keep state" between "multiple inner #[child] invocations", since in practice #[child] would just be an innert attribute for, for instance, #[parent] to parse, process, and strip.

  • you can apply the attribute on stable Rust on places that are currently forbidden, such as statement or expression position.

  • you can perform bigger transformations than the usual attributes can.

As an example of the latter, see the ::with_locals crate, which uses the following syntax, on stable Rust:

  • #[with]
fn main ()
{
    let nested = &RefCell::new(RefCell::new(42));
    #[with] // fake / innert attr (would be forbidden since on stmt)
    let x: &i32 = borrow_nested_refcell(nested);
    #[with] // fake / innert attr (would be forbidden since on stmt)
    let y: &i32 = borrow_nested_refcell(nested);
    assert_eq!(x, y);
}
// where
#[with('local)]
fn borrow_nested_refcell (it: &'_ RefCell<RefCell<i32>>)
  -> &'local i32
{
    &*it.borrow().borrow()
}
    Expansion of main 
    fn main ()
{
    let nested = &RefCell::new(RefCell::new(42));
    with_borrow_nested_refcell(nested, |x: &i32| {
        with_borrow_nested_refcell(nested, |y: &i32| {
            assert_eq!(x, y);
        })
    })
}

    As you can see, such expansion would not have been possible with local macros on statements.

    I like to call this pattern the pre-processor pattern, and, is also what allows calling macros where you technically can't, such as in this example that uses concat_idents! to define a new function. This is also how the ::paste macro works (they have just chosen to use a special syntax ([< … >]) rather than a fake / innert macro.

1 Like
#3

I think this is a misleading way to put it. When you have multiple attributes on the same item, it is in fact guaranteed that they're evaluated outside-in.

The reason this happens is that as far as the process is concerned, #[parent] is passed the entire token stream of the item it decorates to do with as it pleases. It can read, strip, and/or add any code and/or attributes it wants, then emit the transformed code. Then, that transformed code is processed; if #[name] is the outermost attribute, it is then processed.

However! The fact that these are run even in the same process is not guaranteed. Or that #[name]is run _at all_ isn't guaranteed; if#[parent]` emits an identical tokenstream to one it emitted before, the compiler could reuse its knowledge of what the previous transform was rather than rerunning any further attributes.

So no state can be remembered between the implementation of individual attributes. (Every attribute's implementation has to be "pure".) But the logical ordering of the attributes' application is guaranteed, and outer attributes are guaranteed to see attributes closer to the item (and have the opportunity to remove them).

2 Likes
#4

Oh, I thought otherwise: I found some Github issues where the lack of guarantee was phrased in the same way I have, and I thus interpreted it as outside-in not even being guaranteed (which was worrisome, tbh, so I am glad this is not the case). Do you happen to have a "source" backing your claim?

I think the docs, for instance, the Rust reference, w.r.t. evaluation order or proc_macro_attributes (let's not talk about built-in attributes such as docstrings), ought to be updated stating the following:

  • "Re-evaluation" of a procedural macro is not guaranteed: some caching strategies may apply for incremental recompilation.

  • when applied to the same item as outer attributes, they are evaluated outside-in (source?);

    • this gives the chance to the outermost attribute to observe and modify the inner attributes, as I have explained in my previous post.

  • since a whole item fed to a proc-macro may not be emitted by that macro, this necessarily implies that any attribute inside the code fed to the macro (such as #[child] in the OP's example) must also be evaluated afterwards.

  • Otherwise, the order of evaluation of attributes that apply to different / unrelated items is not guaranteed.

#5

From that same issue,

This isn't really about the execution order of proc-macro-attributes, but rather the application order .

https://github.com/rust-lang/reference/issues/692 more directly asks about documenting application order of proc macros. https://github.com/rust-lang/rust/issues/67839#issuecomment-570652165 is I think the most recent intent about the issue.

The one caveat is that the order of "inert" attributes (i.e. derives, comments, and other attributes that aren't proc_macro_attribute) aren't actually preserved by the compiler currently. For noncomment attributes, I believe it's currently requires for all inert attributes to be after all active attributes, so this is mostly a non-issue. For doc comments, though, this isn't required, so it is possible to observe this reordering.

The intent definitely is to have the obvious application order in all cases, however.

1 Like