Proc macros - using third party crate

Hello! I am trying to use a macro from an external crate (izip from itertools). After some trial and error, I found that I need to use external crate itertools inside the quote! {} macro to satisfy the cargo expand command (running the command on the program that uses the macro I have created). I can even see the macro is used in the output.
But when I try to run the program that uses the macro I have created, I get the error that says that the itertools crate is missing.

can't find crate for itertools

I am using quote and syn crates.
Any help?

itertools must be in the dependencies of the crate using your macro. Is it present there?

No, I had not added it there. I expected it to work without adding it. I don't remember adding any such extra dependency when I use third party macros.
Thanks! It worked when I added it

The Rust solution to this, more general than just to procedural macros, are public dependencies through public reexports:

//! Your `lib.rs`

// 2018 edition
pub use ::itertools;
// old style
pub extern crate itertools;

and then within a macro_rules! macro, for instance, you expand to a path referring to your crate first, which must contain the itertools crate as the second element of the path / as if it were a top-level module:

macro_rules! my_izip {(
    $($input:tt)*
) => (
    $crate // self crate
    ::
    itertools
    ::
    izip! { $($input) * }
)}

This way users of your crate will not need to depend on itertools on their own.


This is all well and fine, but with procedural macros things get a bit more complicated. Indeed:

  • there is no $crate magic metavar for procedural macros.

    • You'll have to hardcode the name of your crate instead of $crate:

      ::name_of_my_crate::itertools::izip! { ... }
      
  • a proc-macro = true crate currently cannot export any items other than functions tagged with #[proc_macro], #[proc_macro_derive], or #[proc_macro_attribute]:

    Screenshot 2020-05-12 at 11.42.29

The solution to the second problem is easy, but a bit more cumbersome: a "normal" Rust crate that wants to define, among other things, procedural macros, needs, in practice, to be split in two crates:

  • the proc-macro / derive / internals / helper crate, which defines the procedural macros;

  • and the "frontend" crate, which reexports the proc-macro crate, while defining all the other stuff.

    • (The name of the crate to hardcode in the expanded paths is that of the frontend crate)

This means you need to:

Create your helper proc-macro = true crate

Run
cargo new --lib src/proc_macros --name your_crate_name-proc_macros
(cd src/proc_macros && mv src/lib.rs mod.rs && rmdir src)
Add to src/proc_macros/Cargo.toml
[lib]
proc-macro = true
path = "mod.rs"

[dependencies]
# proc-macro-hack = "0.5.15"  # if function-like proc-macros expanding to exprs
proc-macro2 = "1.0.*"
quote = "1.0.*"
syn = { version = "1.0.*", features = [] }  # you may need "full" features
Edit src/proc_macros/mod.rs to your liking

Set up your frontend crate

Add to your Cargo.toml
[dependencies]
# proc-macro-hack = "0.5.15"  # if function-like proc-macros expanding to exprs

[dependencies.proc_macros]
package = "your_crate_name-proc_macros"
path = "src/proc_macros"
version = "..."  # Same as main version

[workspace]
members = ["src/proc_macros"]
Add to your src/lib.rs
extern crate proc_macros; /* to avoid a cargo bug when cross-compiling (e.g. wasm) */

// #[::proc_macro_hack::proc_macro_hack] /* if function-like proc-macros expanding to exprs */
pub use proc_macros::{
    /* your procedural macros here */
};

#[doc(hidden)] pub /* hide from doc since it is just a tool for your macros */
use ::itertools;
// or
#[doc(hidden)] pub /* hide from doc since it is just a tool for your macros */
extern crate itertools;

What your procedural macro should expand to to refer to itertools' izip!:

src/proc_macros/mod.rs
extern crate proc_macro;
use ::proc_macro::TokenStream;
use ::proc_macro2::{
    Span,
    TokenStream as TokenStream2,
};
use ::quote::{
    quote,
    quote_spanned,
    ToTokens,
};
use ::syn::{*,
    parse::{Parse, Parser, ParseStream},
    punctuated::Punctuated,
    Result, /* explicitly shadow `Result` */
};

#[proc_macro...] pub
fn your_proc_macro (/* attrs: TokenStream,*/ input: TokenStream)
  -> TokenStream
{
    let input = parse_macro_input!(input as /* ... */);
    // ...
    let izip = quote! {
        ::your_crate_name::itertools::izip
    };
    TokenStream::from(quote! {
        /* your expansion here */
        #izip! { /* ... */ }
    })
}
  • You could even reexport izip directly (pub use ::itertools::izip) and have the proc-macro expand to ::your_crate_name::izip.
8 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.