How to import procedural macros that is not in lib.rs?

help
1

The below screenshot is my project layout, the macro_def is the project which has proc_macro=true in its Cargo.toml, but I can not use any macros which are defined in rs files other than lib.rs from main.rs which resides in outter project macro_demo.
2021-04-13_16-46-53

2

Can you post your macro_def/src/lib.rs file?

3

Wow, that's too long, nothing special, just starting with extern crate proc_macro;, and followed by importing syn, quote stuff then 150+ lines of codes.

4

I'm just interested in which mod and use statements you have.

5

In main.rs, the procedural macros sitting in lib.rs are imported like this use macro_def::{make_answer, trace_vars};, but this doesn't work for those sitting in other files in macro_def.

6

As usual, if I'm trying to expose proc_macro_def module in lib.rs with pub mod proc_macro_def;, then VSCode says this: proc-macro crate types currently cannot export any items other than functions tagged with#[proc_macro], #[proc_macro_derive], or #[proc_macro_attribute]

7

Here it is:

extern crate proc_macro;

use proc_macro::{TokenStream};

use quote::{quote, ToTokens};

use syn::parse::{Parse, ParseStream};

use syn::punctuated::Punctuated;

use syn::{parenthesized, Type, token, parse_quote, fold, fold::Fold, Pat, Local, Stmt, Expr, Result, Token, LitStr, parse_macro_input, ItemFn, Ident};

use std::collections::HashSet;
8

So you don't have any mod statements for the extra files at all? You probably need something like this:

mod proc_macro_defs;
pub use proc_macro_defs::the_macro;
9

@Davidliudonghao you cannot define #[proc_macro…] functions outside the root file of the proc-macro = true crate (usually lib.rs).

You have two options:

  • the most common and cleanest one, is to define the functions at the top-level file (lib.rs) by delegating to a call to the function with the meat of the logic:

    //! `src/lib.rs`
use ::proc_macro::TokenStream;

#[proc_macro_attribute] pub
fn foo (attrs: TokenStream, input: TokenStream)
  -> TokenStream
{
    my_module::foo(attrs.into(), input.into())
        .unwrap_or_else(|err| err.to_compile_error())
        .into()
}

…

mod my_module;

    //! `src/my_module.rs` or `src/my_module/mod.rs`

#![allow(unused_imports)]
use ::proc_macro2::TokenStream /* as TokenStream2 */;
use ::quote::{format_ident, quote, quote_spanned, ToTokens};
use ::syn::{*,
    parse::{self, Parse, ParseStream, Parser},
    punctuated::Punctuated,
    Result, // explicitly shadow Result
};

pub
fn foo (attrs: TokenStream, input: TokenStream)
  -> Result<TokenStream>
{
    let _: parse::Nothing = parse2(attrs)?; // notice how we get to use `?` in the function's body 👌
    let fun: ItemFn = parse2(input)?;
    …
    Ok(quote!(
        …
    ))
}

  • The other option is to use include!("my_module.rs") instead of mod my_module;, although this has the drawback of flattening all your crate namespacing, possibly leading to clashes. This allows you to write #[proc_macro…] functions inside the include!d files.

Regarding the first option, if you have many such proc-macros, you can define a helper macro_rules! to wrap the stuff for you:

macro_rules! reexport_proc_macro_attribute {
  (
    pub use $module:ident :: {    
        $(
            $( #[doc = $doc:expr] )*
            $fname:ident
        ),+ $(,)?
    };
  ) => (
    $(
        $( #[doc = $doc] )*
        #[proc_macro_attribute] pub
        fn $fname (
            attrs: ::proc_macro::TokenStream,
            input: ::proc_macro::TokenStream,
        ) -> ::proc_macro::TokenStream
        {
            $module::$fname(attrs.into(), input.into())
                .unwrap_or_else(|err| err.to_compile_error())
                .into()
        }
    )*
  );

  (
    $( #[doc = $doc:expr] )*
    pub use $module_name:ident :: $fname:ident;
  ) => (
    reexport_proc_macro_attribute! {
        pub use $module_name::{ $( #[doc = $doc] )* $fname };
    }
  );
}
  • (and so on for proc_macro_derive and proc_macro)

That way you can write:

mod my_module;

reexport_proc_macro_attribute! {
    pub use my_module::{
        /// This is a very nice proc-macro that does …
        foo,
    };
}
10

@ Yandros Cool! Thank you very much. I realized that the definition of procedural macros must be sitting at the root of current crate(lib.rs). Your suggestions are very constructive, I tried, but as you mentioned, naming conflicts come up immediately. In the end, I decided to create an independent project for each type of procedural macro so as to keep it simple and with zero worries about naming collisions.