I’m contributing in a library which uses proc macro to generate JSON and some structs based on trait signature.
The problem is that in this case this library adds
serde_json as a runtime dependency while it’s only used when building the library. It makes sense to move it to the
build.rs and make serde dev-dependency. But here is a catch: I didn’t find any way to convert source file into token stream.
syn crate only works if you have TokenStream, but in my case I don’t have.
How could it be done? Am I missing something or there is no way to do it with today architecture? It looks quite common that your attribute may use some dependencies that aren’t actually runtime ones.
If you wonder why I’m worried: there is a known bug (feature?) in the cargo that it cannot resolve one library linked as std and no_std (I have seen 6 or 7 issues about it). And this is exactly what I have. So I want to move one into dev-deps to resolve the conflict.
TokenStream implements the standard library’s
let tokens: TokenStream = string.parse()?;
It also implements (edit: this is irrelevant because syn::parse requires you to already have a TokenStream)
Thank you, gonna try it tomorrow.
P.S. Is it known practice? Because I have never seen it before in other crates… It seems a ubiquitous task (in most cases codegen require much more deps than resulting code), but everybody just puts everything in
Anyway, is there the better way to do it? Because currently I only see following:
- find all occurences of
custom_attribute in the code
- extract somehow the whole method decorated by it
- use existing code that was creating runtime dependencies.
Isn’t it too complicated? May I be missing something?
It’s something more like this:
Read the entire file to a string.
Parse the entire string into a TokenStream.
syn::parse on the entire TokenStream to obtain a
File. This gives you the full AST.
Items that have the attribute and do something with them. This is most easily done by creating one of the following:
Visit is basically a glorified
VisitMut is basically a glorified
Fold is basically a glorified
FnMut(T) -> T
Whichever one you implement, you should only need to override a small number of functions on the trait. (based on your post, you probably want to override either
foo_item_fn. (But don’t override both, as that would be redundant!))
New AST nodes can be generated with
parse_quote!. You can also use
quote! directly to turn an arbitrary AST node back into a TokenStream if necessary.
Yep, I see that there is a visitor. Thank you.
Maybe better alternative is write custom
cargo command to build this JSON?.. I’m not quite sure. Generating it in attribute is awful, but the JSON shouldn’t be generated if build failed. But if I write
build.rs which runs before build it JSON could be generated while build failed. Need some investigation here too.