Why the `unreachable!()` can't be optimized away?

help
1 
type Arg = Vec<u8>;
#[unsafe(no_mangle)]
fn test(this: &mut Option<Arg>, arg: Arg) {
    *this = Some(arg);
    let Some(arg) = this else {
       unreachable!()
    };
    black_box(arg);
}

With rust v1.98.0, its assembly:

test:
	pushq	%r14
	pushq	%rbx
	pushq	%rax
	movq	%rsi, %r14
	movq	%rdi, %rbx
	movq	(%rdi), %rsi
	testq	%rsi, %rsi
	jle	.LBB5_2
	movq	8(%rbx), %rdi
	movl	$1, %edx
	callq	*__rustc::__rust_dealloc@GOTPCREL(%rip)

.LBB5_2:
	movups	(%r14), %xmm0
	movups	%xmm0, (%rbx)
	movq	16(%r14), %rax
	movq	%rax, 16(%rbx)
	cmpq	$-1, (%rbx)
	je	.LBB5_4
	movq	%rbx, (%rsp)
	movq	%rsp, %rax
	#APP
	#NO_APP
	addq	$8, %rsp
	popq	%rbx
	popq	%r14
	retq

.LBB5_4:
	leaq	.Lanon.6f8eaf9bed218e625010b376854c49ff.1(%rip), %rdi
	leaq	.Lanon.6f8eaf9bed218e625010b376854c49ff.3(%rip), %rdx
	movl	$40, %esi
	callq	*core::panicking::panic@GOTPCREL(%rip)

.Lanon.6f8eaf9bed218e625010b376854c49ff.1:
	.ascii	"internal error: entered unreachable code"

.Lanon.6f8eaf9bed218e625010b376854c49ff.2:
	.asciz	"src/main.rs"

.Lanon.6f8eaf9bed218e625010b376854c49ff.3:
	.quad	.Lanon.6f8eaf9bed218e625010b376854c49ff.2
	.asciz	"\013\000\000\000\000\000\000\000\b\000\000\000\b\000\000"

But when I change Arg to Box<[u8]>, the unreachable!() can be optimized away:

type Arg = Box<[u8]>;
#[unsafe(no_mangle)]
fn test(this: &mut Option<Arg>, arg: Arg) {
    *this = Some(arg);
    let Some(arg) = this else {
       unreachable!()
    };
    black_box(arg);
}

test:
	pushq	%r15
	pushq	%r14
	pushq	%rbx
	subq	$16, %rsp
	movq	%rdx, %rbx
	movq	%rsi, %r15
	movq	%rdi, %r14
	movq	(%rdi), %rdi
	testq	%rdi, %rdi
	je	.LBB5_3
	movq	8(%r14), %rsi
	testq	%rsi, %rsi
	je	.LBB5_3
	movl	$1, %edx
	callq	*__rustc::__rust_dealloc@GOTPCREL(%rip)

.LBB5_3:
	movq	%r15, (%r14)
	movq	%rbx, 8(%r14)
	movq	%r14, 8(%rsp)
	leaq	8(%rsp), %rax
	#APP
	#NO_APP
	addq	$16, %rsp
	popq	%rbx
	popq	%r14
	popq	%r15
	retq

It seems only Option<Vec<T>> has this problem, but I'm not sure.

2

TBH, as soon as there's a black_box it being bad doesn't surprise me at all. It exists only to make optimizations worse.

3

Do you feel similarly unsurprised by this version, with no black_box, which also does not remove the unreachable!?

fn test(this: &mut Option<Arg>, arg: Arg) -> &mut Arg {
    *this = Some(arg);
    let Some(arg) = this else {
       unreachable!()
    };
    arg
}
4

The example works without black_box:

So it's definitively something "interesting" happening here. If I am to look for a culprit, it's likely that some magic in Drop impl of Vec is responsible for this.

5

The practical answer to eliminating the unnecessary panic branch is to use Option::insert() instead of assignment:

fn test(this: &mut Option<Arg>, arg: Arg) {
    let arg = this.insert(arg);
    black_box(arg);
}

(Internally, Option::insert() just uses .unwrap_unchecked().)

6

In my codebase i'm using another enum instead of Option. It seems i have to use unsafe to get around this problem now.

7

It looks like niche optimizations are responsible for this somehow. Using struct MyVec { p: NonNull<u8>, len: usize, cap: usize } prevents branch eliminations, while it happens properly with struct MyVec { p: *mut u8, len: usize, cap: usize }. Removing cap from the NonNull variant also enables the optimization.

8

I agree.

9

Looks like rustc translates arg: Box<[u8]> into two separate parameters (pointer and length) in the LLVM IR, with the pointer marked nonnull:

define void @test(ptr noalias nofree noundef align 8 dereferenceable(16) %this, ptr noalias noundef nonnull %arg.0, i64 noundef %arg.1) unnamed_addr #0 personality ptr @rust_eh_personality {

Whereas arg: Vec<u8> is passed by reference, presumably because it's larger, 3 fields rather than 2 fields:

define void @test(ptr noalias nofree noundef align 8 dereferenceable(24) initializes((16, 24)) %this, ptr dead_on_return noalias nofree noundef readonly align 8 captures(none) dereferenceable(24) %arg) unnamed_addr #0 personality ptr @rust_eh_personality {

The information that the Vec pointer is non-null is lost here, so LLVM doesn't know about it. rustc has dropped the information that the pointer inside Vec is not null, and so LLVM optimizations can't use that information.

None: Option<Vec<u8>> is encoded as a null pointer inside Vec. LLVM doesn't know that Some(arg) is not the same thing as None, since it would be encoded the same way if the pointer inside Vec was null. So it has to do a runtime check for a null pointer.

10

Maybe not. I think dereferenceable(..) implies nonnull, but I'm not an LLVM expert.

Nope. It is encoded as Vec { cap: -1, ptr: undefined, len: undefined }.

Nope. The two variants can be distinguished by just checking the cap field:

cmpq	$-1, (%rdi)

This problem is actually more weird than I think previously.

11

It implies the pointer to the Vec is non-null, but it doesn't imply the inner pointer inside Vec is non-null.

Interesting. This is correct. But either way, that information is not passed to LLVM: LLVM doesn't know that cap is not -1 in a Vec passed by reference.

12

Oh, I kind of understand. This problem may need earlier optimization in mir.

13

Actually the logic of my codebase is more close to this:

type Arg = String;
#[unsafe(no_mangle)]
fn test(this: &mut Option<Arg>, arg: Option<Arg>) {
    let Some(arg) = arg else {
        return;
    };
    if !arg.starts_with("--") {
        return;
    }
    *this = Some(arg);
    let Some(arg) = this else {
       unreachable!()
    };
    black_box(arg);
}

In this case, the unreachable!() can be optimized away because LLVM has more infomation now.
Thanks for everybody.

14

I think Rust just stabilized Never type, you can specify as '!'. So you may want to retest with it.

15

The stabilization of the never type is completely unrelated to this.

16

I know.

17

So I don't understand what you meant.