Hiding stacktrace from should_panic tests

#1

Hi everyone. As I get more should_panic tests this issue gets more inconvinient. I'm looking for a way to disable panic stacktraces for should_panic tests (but keeping it for those who shouldn't panic!) but I've failed to do so.

Requirements are simple:

  • allow running test in multiple threads
  • simultaneosly do not show panic stacktraces from should_panic tests.

This is my best effort code which as you can imagine doesn't work since it's blinking and its output is different whenever test_a was running before test_b and test_c or not.

pub fn run_should_panic_test(test: impl FnOnce() -> ()) {
    let hook = std::panic::take_hook();
    std::panic::set_hook(Box::new(|_| {}));
    test();
    std::panic::set_hook(hook);
}

#[test]
#[should_panic]
fn test_a() {
    run_should_panic_test(|| {
        panic!("I don't care")
    })
}

#[test]
fn test_b() {
    panic!("wwoowow")
}

#[test]
fn test_c() {
    panic!("wwoowow")
}

Desired output:

running 3 tests
test test_b ... FAILED
test test_a ... ok
test test_c ... FAILED

failures:

---- test_b stdout ----
thread 'test_b' panicked at 'wwoowow', src/lib.rs:14:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test_c stdout ----
thread 'test_c' panicked at 'wwoowow', src/lib.rs:19:5


failures:
    test_b
    test_c
#2

I ended up with

#[ctor::ctor]
fn init_disable_panic_stacktrace_hook() {
    let original_hook = std::panic::take_hook();
    std::panic::set_hook(Box::new(move |x| {
        let backtrace = Backtrace::capture();
        let backtrace = backtrace.to_string();
        let should_print_stacktrace = !backtrace.contains("::disable_panic_stacktrace");
        if should_print_stacktrace {
            original_hook(x)
        }
    }));
}

pub fn disable_panic_stacktrace(test: impl FnOnce() -> ()) {
    test();
}

Usage:

#[test]
#[should_panic]
fn my_test() {
    disable_panic_stacktrace(|| {
        // do stuff
    })
}
#3

One alternative may be to catch the panic unwinding and assert that it is an error, and then you can also assert the panic message to be safer.

#4

panic_unwind doesn't prevent panic printing. It's first prints it and then you can process it in catch_unwind.

#5

Doesn't #[should_panic] already suppress backtraces?

#6

No

#7

In playground, it does. Could you share the exact problematic case?

#8

It still show the stacktrace with cargo test -- --show-output which is default mode in cargo test command in IntelliJ-Rust. I really want some debug prints from tests but not panics form should_panic tests.

#9

If you only care about those debug prints you can remove --show-output. The test harness will print it anyway for failed tests. Otherwise you got what you asked for. You asked for the output of tests even if they didn't fail, so you got saod output, including panic messages.

#10

Consider I'm having two tests: one is one I'm currently debugging and I want to see its output. The other one is should_panic one and I don't want to see its output.

This is my situation and this is how I solve this.

#11

If you only care about that one test at the moment, why not just run that test instead of both/all of them?

#12

Because I'm using IDEA and it defaults to --show-output when running all tests. And i'm always panicking myself when I see panics in output. This way I can use all default and feel myself safe from "oh no i broke something". it's also easier rather than making all new team members changing their IDE defaults to prevent this behvaior.

#13

Maybe IDEA should instead switch it's default to the default of libtest, that is no --show-output.

1 Like
#14

Maybe, but unfortunately I have no power to tell IDEA what to do by default.

#15

You could open an issue at Issues · intellij-rust/intellij-rust · GitHub and point to this discussion maybe?