What is the common way to use procedural macros internally?

In my case, I want to use procedural macros to do some internal job for my lib, which means it's totally useless for a user. But for now, procedural macros must be written in a separated crate and marked it as proc-macro = true.

I wondered what other famous projects handle this problem, I have two (or maybe three) chooses:

  1. Publish the macro crate as well. I think it's kind of waste of the resources in crate.io, since it's useless outside the crate.

  2. Fetch the macro crate from GitHub. But I know some companies or organizations uses their own crate registry, and fetch from a website may not acceptable.

  3. I wondered how is the progress of putting procedural macros into a normal crate, maybe I can do this in nightly now? (I did some search, but didn't find anything, maybe my searching keyword is not right)

If you want to publish your library to crates.io, then the proc-macro dependency must be published there as well. There's currently no way around this.

If you "publish" your library only as a git repo, then the proc-macro can be in the same repo as a relative path dependency.

I vaguely recall discussions about allowing crates.io packages to bundle proc macros with a library crate, but my searching isn't turning that up right now either.

7 Likes

About the suggestion 2, even if it's a git repo, a user still need to use xxx = {git = "https://xxx"}, in this case, only the crate of xxx is fetched, the crate xxx-derive will not, why the relative path dependency works? Do I miss something?

When using a git dependency the whole git repo is fetched and checked out. Within the git repo crates can depend on each other using path dependencies.

1 Like

This is probably what I was remembering:
https://github.com/rust-lang/lang-team/issues/139

See the discussion on zulip. It initially mentioned proc-macros, but that was dropped, and the whole proposal was eventually closed anyway.

3 Likes

One interesting thing to keep in mind is that proc-macro logic does not necessarily need to be invoked as a proc-macro.

If you are doing something like:

generate_stuff!("some input…");

for instance, kind of code, then know that you could be exporting generate_stuff as a function over ::proc_macro2 types rather than ::proc_macro ones, thence yielding a function rather than a macro, and then have some helper Rust script that uses that function to generate some generated.rs kind of source file that would be tracked by the crate (even if it's generated code).

  • The main code would then either include! or mod generated; to use it.

This, effectively, means you'd be easily caching the output of this "procedural logic", thence eliminating the need for each and every dependent to run such logic, leading not only to faster compile-times, but also to avoiding the "I have to publish that procedural logic to some registry" problem altogether. Indeed, such helper Rust script would be a complementary dev-siding tool which would not need to be officially published (even if it could be nice to still expose it somewhere, such as in the repo, so that people can still opt into using it themselves).


If you are instead doing something like #[some_annotation] to get some helper stuff, you could sometimes switch to a macro_rules! implementation that you would apply using


Other than those two options, yeah, the simplest thing would be to publish the internal crate over crates.io even if it wouldn't be intended for direct usage. This happens already quite often, so you shouldn't worry about "cluttering". Just "mangle" the name enough to deter people from accidentally directly depending on it

5 Likes

May I ask do you experts rely on IDEs a lot? I'm always trying to avoid of using normal macros because the IDE will not detect the code inside it, leads to a lot of functioning tools broken (code index, function auto complete, grammar checking and so on, and I really enjoy those features). While using procedure macros is a little better, since the code which be annotated is normal codes and can be detected by the IDE, only generated code will be undetected.

I wouldn't avoid macros just for its own sake. You'd be surprised how well IDEs handle them these days.

3 Likes

That is a declarative macro. Those are much easier to deal with than procedural macros as both the grammar and definition are known to the ide for declarative macros. Procedural macros can do whatever they want and often don't handle partial code and instead opt to panic.

https://veykril.github.io/posts/ide-proc-macros/

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.