Having helper macros call each other in generated code

help
1

So I've been trying forever to get this to work:

$OUT_DIR/generated_macros.rs

#[macro_export]
macro_rules! quote {
    () => { $crate::_quote_impl!() };
}

#[macro_export]    
macro_rules! _quote_impl {
    () => ( $crate::quote::foo() );
}

src/lib.rs

#[macro_use]
pub mod quote {
    #[macro_use]
    mod generated {
        include!(concat!(env!("OUT_DIR"), "/generated_macros.rs"));
    }

    pub fn foo() {}
}

fn main() {
    quote!();
}

  • The code above fails because $crate::_quote_impl!() becomes ::_quote_impl!() in the current crate, which searches the root namespace and fails.

  • If I try to move the macros out into an extern crate crate_macros;, then $crate::quote::foo() fails to resolve

    • foo needs to be defined in the main crate. The two places where the macros need to be used are that same main crate, and its integration tests. Therefore, the macros need to be in this crate so I can use $crate::quote::foo().

  • If I try deleting the "$crate::" on $crate::_quote_impl!() and change the attributes on the macros to #[macro_export(local_inner_macros)], then I run into this crazy bug?!

  • I tried working around that bug by changing it to #[path = ...] mod generated; using the macro early expansion trick that works on #[doc]. Turns out it doesn't work on #[path]. (you get a "no such file" error searching for a generated.rs as if the annotation wasn't even there!)

  • it can't be this difficult

  • halp

2

It looks like just changing $crate::_quote_impl!() to _quote_impl!() works within the same crate, but as soon as it's used outside the crate, it predictably doesn't work. I think the only way to fix this is to fix issue #81066 (linked in OP). (I though that include was just straight up copying another file into the current one, maybe there's more going on than I thought)

edit: digging a bit deeper, this looks like a problem with macros expanding to macro_rules macros, because the following doesn't work either

macro_rules! expand_to_macro_rules_macro {
    () => {
        #[macro_export]
        macro_rules! quote {
            () => {
                $crate::_quote_impl!()
            };
        }

        #[macro_export]
        macro_rules! _quote_impl {
            () => {
                $crate::quote::foo()
            };
        }
    };
}

#[macro_use]
pub mod quote {
    #[macro_use]
    mod generated {
        expand_to_macro_rules_macro! {}
    }

    pub fn foo() {}
}

fn main() {
    quote!();
}
3

Thanks for the sleuthing!

In the meantime, a couple more alternatives occurred to me:

The old way

#[macro_export]
macro_rules! quote {
    () => { _quote_impl!() };
}

#[macro_export]    
macro_rules! _quote_impl {
    () => ( $crate::quote::foo() );
}

and then in integration tests and other downstream code, one can do

#[macro_use]
extern crate mycrate;   // instead of `use`-ing the macros they want

The unfortunate part is that the user will end up importing more macros than they bargained for, but :man_shrugging:

The somewhat-more-manual way

The thing that's forcing me to use import!() is that my files are in $OUT_DIR, as required by best practices for build scripts.

So, instead of having a build script generate my macros, I can have another binary crate (which I only run manually) generate them at a fixed path relative to the src/ dir of this crate, and check them into version control. This would let me import them without needing import!, and so I'd be able to turn local_inner_macros back on.

Maybe I can change to this strategy once the dust settles more.

4

Can you use some combination of concat! and env! to construct a #[path = ...] directive that points into $OUT_DIR?

EDIT: Nevermind, that's what the "macro early expansion trick" you're referring to is about.

5

That doesn't work, even with #![feature(extended_key_value_attributes)] to enable using macros in #![path = ...]. Rust seems to just ignore the #[path = ...] attribute :pensive:.

6

Indeed. To be clear, the trick is to do this, since writing #[path = concat!(...)] directly won't parse:

macro_rules! _path_hack {
    ($path:expr) => {
        #[macro_use]
        #[path = $path]
        mod generated;
    }
}
_path_hack!(concat!(env!("OUT_DIR"), "generated_macros.rs"));

which actually works just fine for #[doc = $doc], but not for #[path]. I'm certainly glad to hear that #![feature(extended_key_value_attributes)] is a thing because I've always hated having to resort to this trick!