Procedural macro to help testing with temporary folders

help
1

Mostly to learn how to write procedural macro, I tried to tackle a very simple use case which is: provide a temporary folder to a test function that will be wiped out once the test is done.

For example, I would write something like

#[test_with_tempdir]
fn my_test(path: &Path) {
  // write tests using path as an existing temporary folder
}

So far, I've been able to write this macro which basically take the input TokenStream, and wrap it inside another function with no argument that creates a temporary folder, then call the inside function with the provided &Path (see here). And this works fine... for a happy scenario that will pass. As soon as I try to test scenarios that I expect to fail.

#[test_with_tempdir]
#[should_panic]
fn my_test(path: &Path) {
  // write tests
}

the compiler complains that my #[should_panic] is a

warning: unused attribute
  --> tests/tempdir.rs:47:1
   |
47 | #[should_panic]                                                                                                                                             | ^^^^^^^^^^^^^^^
   |
   = note: #[warn(unused_attributes)] on by default

I realized that maybe the #[should_panic] was behaving weirdly because it didn't find a corresponding #[test] so I experimented a change in my design with something like.

#[test]
#[with_tempdir]
fn my_test(path: &Path) {
  // write tests
}

but then the compiler would tell me this

error: functions used as tests can not have any arguments
  --> tests/tempdir.rs:12:1
   |
12 | / fn path_exists(path: &Path) {
13 | |     assert_eq!(path.exists(), true);
14 | | }
   | |_^

A few questions comes to mind:

  • Is it possible to make my macro be executed before #[test] so I can replace the function with a no-argument function before #[test] start to execute?
  • When there is multiple macros on a function, is the order important?
  • Can one macro know there is another one, parse it and interpret it (for example, does #[test] know there is #[should_panic] and behave accordingly)?
  • How can I make the #[should_panic] be considered? I could obviously change the design to accept something like #[test_with_tempdir(should_panic)] for example, but it'd be nice to be able to reuse existing macros #[test] and #[should_panic] instead of rewriting everything

There is a git repository temp_testdir_proc_macro which is quite experimental for now but I believe the code is readable. Don't hesitate to ask questions if things are not clear.

And in advance, thank you for those who will take the time to read, maybe understand and even more answer to my ask for help :slight_smile:

UPDATE: I've moved the repository to the following link GitHub - woshilapin/with_tempdir_procmacro so some of the above link will likely be outdated.

2

One thing to keep in mind here is that the vast majority of attributes are not implemented as macros. #[test] does indeed appear to be implemented as what the compiler calls a "builtin macro attribute"; but #[should_panic] certainly is not. You can see the code in the implementation of #[test] that searches for a #[should_panic] attribute here.

I know that custom derives can declare attributes that they manage, so that the compiler does not throw an error on an unrecognized attribute. This is what makes it possible for structs with #[derive(Serialize, Deserialize)] to have #[serde] tags on their fields. I don't think you can do the same with proc_macro_attribute.

I have not personally written a proc_macro_attribute so I can't help too much here. If I had to guess, user macro attributes should always expand before builtin macro attributes, because builtin macro attributes are not supposed to feel like macros (last I recall, there's a similar rule for derives). But there's no point guessing. I think maybe the best thing you can do right now is to write a simple proc_macro_attribute whose implementation is just:

panic!("{}", input)

and see what the input looks like when you try it on items with attributes defined in various orders. (e.g. one with #[test] #[myattr], one with #[myattr] #[test], etc.).

Assuming that you can see stuff like #[should_panic] and #[test] in the output, they should appear in the attrs field after you parse the input into a syn::ItemFn.

3

I tried to expand your macro using cargo expand which did not work until I removed the #[test] attribute, probably related to it being a builtin macro. I then added one back
resulting in this

fn path_exists() {
    use tempfile::Builder;
    #[new_test_attr] // wrong spot right?
    fn wrapped_path_exists(path: &Path) { ... }}

It seems there are 2 options, adding more attributes arguments to test_with_tempdir, like with ignore, or as

let mut fn = parse_macro_input!(input as ItemFn);

let original_attrs = fn.attrs.clone();
// replace with empty
fn.attrs = vec![]; 

let wrapped = quote! {
        #test_macro
        #(#original_attrs)* // this must expand for each attr 
        fn #test_function_ident() { ... }

had to use the repeat syntax because ToTokens is not implemented for a Vec. I hope this was helpful good luck! :+1:

1 Like
4

Let's not forget to copy / move the metadata of the wrapped function into the wrapping function:

The main thing here is the attrs field: I suggest you scan the Vec for a potential #[should_panic] attribute, and swap_remove() it out of test_fn, so that you can add the attribute to the outer / wrapping function, with the other #test_macro attributes.

Other remarks

  • you don't need to rename the wrapped function, since, in Rust, it is perfectly fine to have:

    fn foo ()
{
    fn foo (_: i32) {}

    foo(42);
}
2 Likes
5

Thanks very much all of you. I read all your answers. I'm not yet where I want to be but the hints helped me move forward. I am now able to do something like

#[test_with_tempdir] // I should probably rename it to `with_tempdir` now
#[test]
#[should_panic]
fn some_func(path: &Path) {
}

Details

When the function is parsed as syn::ItemFn, the other macros (#[test] and #[should_panic]) are actually parsed into the attrs field (some of the metadata as mentioned by @Yandros). So what I did is exactly what @DevinR528 mentioned: reapply these syn::Attribute to the wrapping function and remove them from the wrapped function. And this works... but!

The problem comes from the following code.

#[test]
#[should_panic]
#[test_with_tempdir]
fn some_func(path: &Path) {
}

Note how both #[test] and #[should_panic] are above #[test_with_tempdir]. In this case, once the function is parsed as syn::ItemFn, the field attrs contains a vector of only one element which is the #[should_panic]. The #[test] is not present...

I also tried

#[should_panic]
#[test]
#[test_with_tempdir]
fn some_func(path: &Path) {
}

with the same result, only #[should_panic] is present.

The fact that #[test] is a builtin macro attribute (as mentioned by @ExpHP) is probably the root cause but I'm not even sure how to solve that.

I tried to cargo expand 2 identical tests with only the order of #[test] and #[test_with_tempdir] different. The one with #[test] as first is not working as a test but #[test_with_tempdir] is expanded correctly. The one with #[test] as last just work fine (as seen above). My next step is to try to bring trybuild crate to improve my tests.

The updated repository is now called with_tempdir_procmacro.

6

You won't be able to circumvent this limitation; on the other hand, having both #[test] and #[test_with_tempdir] seems redundant: what about making your proc_macro_attribute add #[test] on its own?

This way, you would have two ordering possibilities:

#[should_panic]
#[test_with_tempdir]
fn some_func (path: &'_ Path) { ... }

and

#[test_with_tempdir]
#[should_panic]
fn some_func (path: &'_ Path) { ... }

which should both work fine.

7

It was actually what I had in the first iteration of this project (that’s why it’s called test_with_tempdir, it was injecting the #[test] itself). In the last iteration, I thought it would be nice to reuse existing macros like #[test] and #[ignore] or #[should_panic] (and others) so I changed the API of my macro and renamed it only #[with_tempdir]…

Do you have some more precise details about why #[test] will cause me problems in this case? I’m having a hard time understanding this limitation.

8

The one you showed: if #[test] is written before #[test_with_tempdir], then #[test] won't appear in the attrs field. If you are fine with that then there is no issue :slight_smile:

The other thing I said was not an issue but just an aesthetic preference: in the same vein that there is no #[test_ignore] nor #[test_should_fail], the #[test_with_tempdir] could be renamed to just #[with_tempdir] if it is to be used with #[test]. But that's just a personal preference :wink:

9

So we agree… but seems like my previous message was not well phrased :stuck_out_tongue: You can look at the current status of the tests to understand what I mean.

2 Likes
10

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.