Any suggestion for ignoring #[should_panic] tests at runtime?


#1

Take this test in vulkano:

#[test]
#[should_panic(expected = "Tried to submit a present command without any swapchain")]
fn no_swapchain_added() {
    let (_, queue) = gfx_dev_and_queue!();
    let _ = SubmitPresentBuilder::new().submit(&queue);
}

In this code we start by initializing a Vulkan device with the gfx_dev_and_queue!() macro.
Then we try to submit a present command without any swapchain. Since doing so in invalid, the submit function panics.
The test has a #[should_panic] attribute that catches this situation and makes the test succeed.

This works on my local machine. Unfortunately when running this test on Travis, there is no Vulkan implementation available on the machine and the gfx_dev_and_queue!() macro uses return as a way to ignore the test (here, here or here).
While using return to ignore a test works for regular tests, it doesn’t work for tests with #[should_panic] as it makes the test fail.

I know that there’s no way to solve this problem in a clean way. However I’m wondering if people had suggestions for hacks to solve this without having to call panic!("Tried to submit a present command without any swapchain") and more generally without having to copy-paste the panic message.

Thank you


#2

This is going to sound ugly, but could you rely on some of the environment variables travis provides and make a macro to return early/panic?

macro_rules! ci_check {
    ($expected_msg:expr) => {
        if option_env!("TRAVIS").is_some() || option_env!("APPVEYOR").is_some() {
            panic!($expected_msg);
        }
    };
    (PANIC) => {
        panic!("Running on CI");
    };
    () => { return };
}

#[test]
#[should_panic(expected = "some panic message")]
fn test_something() {
  ci_check!();
  ...
}

Then if you run that at the start of any test which depends on the Vulkan implementation it can return or panic with either the expected panic message or a generic one. You still have to copy the expected panic if you are using #[should_panic("some panic msg")], but it’s still better than before.


#3

Your build script could output cargo:rustc-cfg=travis if the TRAVIS environment is set, and then you can use #[cfg_attr(travis, ignore)] on those tests.


#4

Maybe you could use catch_unwind to make your own custom assert_should_panic!()?