Is there a way to "preview" macro-derived TokenStreams?

I generally use VS Code with the rust-analyzer to write code. I am happy with it in general. But debugging procedural macros that create a TokenStream (typically to automate the implementation of some functions) has been very difficult. VS Code tells me what the underlying error in the output TokenStream is, but there is no way for me to "see" the token stream to identify what otherwise might be an obvious syntactic error.

Does anybody know of a way to "preview" the token stream resulting from #[derive(Whatever)]?

IIRC Tokenstreams are Displayable, so you can just do something like println!("{}", tokenstream);.

Though if you want nicer formatting, you might want to consider using cargo expand, which is installable via cargo install cargo-expand.

4 Likes

Oh that seems simple enough. Thank you.

I added this line in the derive crate:

#[proc_macro_derive(DerivedStuff)]
pub fn my_trait(input: TokenStream) -> TokenStream {
    let ast: syn::DeriveInput = syn::parse(input).unwrap();
    // ... lots of stuff ...
    println!("{}", &tokenstream); // ADDED THIS LINE
    tokenstream.into()
}

Back in the "main" library, I made a test on a struct that has #[derive(DerivedStuff)] fail so it would show the output. Not sure if this is more complicated than what you imagined, but I was flying blind and now I can see.

This is pretty much what I had in mind.

BTW, you can also run cargo test -- <TEST_NAME> --nocapture to show test println output without altering the test.

2 Likes

Thx. I forgot about --nocapture

I recommend using cargo expand for the general case, and for the case of a proc-macro of your own, print can be very useful but, alas, doesn't yield such a nice looking output on its own (not formatted nor colored).

In that case, you can write your own helper pretty_print_tokenstream to palliate that, and it will yield cargo expand-looking code:


Regarding debugging macros more generally, see this summary of mine.

2 Likes

Wow- thanks for the super detailed and helpful answer. I will try to work through it- cargo expand looks pretty useful.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.