Is it possible to access and run procedural macros as conventional functions (so I can do things like read in files and transform the tokenstreams I get from that) without having a crate boundary between the declaration of the proc_macro entry points and and the functions themselves? I already have two crates because I have conventional code that links to the macros, so I don't want a third if I can help it since I want to put this on crates.io
I tried using the macro crate as a dependency and overriding the lib.proc-macro configuration, and then also tried it the other way around by hiding the proc macro declarations behind a feature flag, but neither of these worked correctly. There seems to be a lot of hardcoded expectations around how proc macro crates work that go beyond the documented requirements - e.g. the docs say exported items need proc_macro_attribute, but this doesn't allow that attribute to be defined conditionally, it means it has have #[proc_macro_attribute]exactly
All this is coming from wanting a middle ground between the zero context I get from an error in generated code during actual use and the overload of detail I get from cargo expand, since the code I'm generating tends to invoke more macros and I don't have an option to leave those unexpanded.
Probably not. When compiling Rust code, there are generally two different targets:
The machine being used to compile Rust code on.
The machine that the Rust code should run on.
Sure, most of the time, these are the same, but there is no such requirement. You can compile for arm on an intel machine. Binaries and libraries are compiled for the second target, but proc-macros are compiled for the first target. Therefore, binaries cannot use things defined in a proc-macro crate.
That makes sense, but does it rule out being able to link the macro crate in a way that it doesn't have the proc-macro configuration set? (And so builds for the target)
The best solution I've come up with so far is to have a build script clone the entire crate source code into a local module, tweak it to work in a non-macro context, and then use it from there. This is fantastically cursed and likely quite fragile, but it does work.
If your macro is complex and does a lot of unrelated processing, then what you should do instead is respect the single responsibility principle. Break your macro's code into the following parts at the minimum:
a parser written against a generic interface (e.g. proc-macro2) that doesn't depend on being in a proc-macro crate and a similar code generator
another layer that merely translates between the generic representation and actual proc-macro TokenStreams (if you are using proc-macro2, then this is a pair of trivial From/Into calls)
the bulk of the logic in a separate, decoupled crate that works with the generic parsing/codegen interface and doesn't care about the internals of proc-macros.
Then, whenever you want to use your macro's essence outside an actual macro, use layers 1 and 3 only. Then export a trivial proc-macro that includes layer 2 as well.
Internally it is divided up like that, but the problem is that being able to use 3 outside a macro context requires it being in a separate crate than 2, and since I want to publish a crate that depends on 2, I'd have to have a third package to upload, and I didn't want to do that if I could avoid it.