Weird test behaviour with release mode, ASAN, panic=abort and LTO disabled

help
1

I just would like to know what is going on and if this could maybe be a compiler bug. The problem is actually very easy to circumvent, as it is quite specific...

Just in case this is an OSI layer 8 problem - apologies upfront! :stuck_out_tongue:

My Cargo.toml looks like this:

[package]
name = "asan_lto_reproducer"
version = "0.1.0"
edition = "2021"

[profile.release]
panic = "abort"

[dependencies]

lib.rs:

#[cfg(test)]
mod tests {
    fn foo(_s: &str) {}

    #[test]
    #[should_panic]
    fn weird() {
        let some_string = String::from("some_string");
        // std::hint::black_box(foo(&some_string)); // comment this in for great success!
        panic!("blarb");
    }
}

Command:

export RUSTFLAGS="\
    -Zsanitizer=address \
    -Clto=no \
    "

cargo \
    +nightly \
    test \
    -Zpanic-abort-tests \
    --release

With this setup and the given command, the test fails with:

running 1 test
test tests::weird - should panic ... FAILED

failures:

---- tests::weird stdout ----
---- tests::weird stderr ----
thread 'main' panicked at src/lib.rs:10:9:
blarb
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

=================================================================
==71672==ERROR: LeakSanitizer: detected memory leaks
<ASAN output>

If I change any of these settings, the test succeeds:

  • comment-toggle the black_box line
  • remove ASAN flag
  • remove disable LTO flag
  • remove release flag
  • remove the panic-abort-tests flag
  • remove the panic=abort setting in Cargo.toml

As a sidenote, I also tried with additional flags for ASAN, but it did not change anything, so I left them out:

    -Zbuild-std=std,panic_abort \
    --target=x86_64-unknown-linux-gnu \

My expectation would be exactly the opposite of what is actually happening with the black_box line:

  • When the String is not used, the compiler is optimizing it out, nothing leaks, and no error is reported.
  • If it is used in the black box, it won't be optimized out, it leaks, causing the test to fail.
  • Also, the test succeeds in debug mode, I would expect it to do the same in release.

BTW, I found a bug that might have something to do with it. It seems to be not deterministic, but has a lot of things in common (panic abort, LTO disabled, should_panic):

So I don't know what is happening, but my assumption is that somehting is going on with optimization. I played around a bit, but can't make any sense of it. Maybe somebody else has some insight? Should I file a bug?

Thanks in advance!

2

I investigated a little more on this. This seems to not appear anymore with the latest nightly (2024-03-18). I just don't know if there was a fix or if some optimisation changed slightly.

But my very uneducated guess:

The -Zpanic-abort-tests (which is still an unstable feature, see panic=abort testing / subprocess testing · Issue #67650 · rust-lang/rust · GitHub) inserts some code to spawn processes for each test, which then is optimised in some uncanny way together with the actual test code, resulting in the described behaviour that the should_panic does not properly catch the actual panic anymore.

Changing settings that affect optimisation "fix" the issue because it is done differently in some kind of way so that the final produced code works again.

That being said, I still don't understand why the absence of -Zpanic-abort-tests does not fail the tests. I always thought that this is needed in case of panic=abort, because the process just aborts instead of unwinding and all tests fail?

3

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.