I've a macro that does complex processing steps to achieve hierarchical structures with dynamic dispatch, which is very useful when implementing a compiler's semantic model.
Unfortunately, every struct must appear inside the macro. E.g. the macro results in a closed/final/sealed set of structs.
It looks like this:
use hydroperx_sem::sem;
sem! {
// arena type.
type Arena = Arena;
/// Basemost struct representing a semantic symbol.
/// For instance, a var slot, a method or a class.
pub struct Symbol {
/// Declaration name.
pub fn name(&self) -> String {
"".into()
}
}
/// Represents a method.
pub struct Method: Symbol {
let ref _name: String = "".into();
pub fn Method(name: &str) {
super();
self.set__name(name.to_owned());
}
pub override fn name(&self) -> String {
self._name()
}
}
}
I ask again if there's now a way to split that into multiple files (something like include!(...)).
You control how your macro handles its input TokenStream. You could define your own include "xy.rs"; statements or resolve include!("xy.rs"); inside of your macro.
Normally, the IDE expands the procedural macro to get code awareness (it's also very handy if you want to check the output of the macro, but it won't work if your output stream is too messed up). At least, it's the case with RustRover, but when I did some test with the VSCode editor and rust-analyzer, it seemed to work the same way.
For the included file, however, it has only itself as reference. The IDE doesn't know it's included in your macro, so chances are it'll even constantly look for a use reference to it in all the potential modules and ask you to fix that.
So it will be fine from the source file where the macro is, but you'll have limited code awareness in the included file, plus some potential hassle from the IDE (that you might be able to avoid by putting the included file out of the source tree, or by using another extension, or if you're lucky, if the IDE has a list of files to exclude).
To add to what @jofas said, one way to do that is to read the file and use the parse method on a string or string reference. I think it's available for both proc_macro::TokenStream and proc_macro2::TokenStream.
If text: &str (or String, ...)
let included_stream: proc_macro2::TokenStream = text.parse()?;
Can't you make the included parts independent so that they're legit types on their own? I must admit the purpose escapes me somewhat.
I mean, the base type is independent, so it's fine if it's in another file. The subtype could be expressed with your macro, so it's fine. And so on if there are other subtypes or sub-subtypes.
EDIT: You might need to keep the information about the structure, which is likely gone in the output code, but you could store that information in a comment or an attribute (in the generated code), since it's not visible anyway. The advantage of that approach is that it's modular.
Subtypes may override a method in the super-type, and even do super.method() to invoke an instance method of the super-type. The super-type, to be converted into a struct, needs to know what methods in subtypes override it.
The architecture is also based on an arena using an exhaustive enum that identifies data of one of the many types in the semantic model.
Forget what I said about storing the information anyway; the proc macro of the subtype wouldn't be able to recover it since it doesn't have access to any context (e.g. can't find the doc comment associated to an item), and I think that sharing information between proc macros isn't really possible since they could be asynchronously expanded.
