Why I get NaN values when multiply by 0?

Hello !

I have the following problem: I have 3 variables (a, b and c), using fast-floats. When I multiply them I get NaN.

println!("{:.32} {:.32} {:.32} {:.32}", a, b, c, a * b * c);

output:

23786734966226792158959790718976.00000000000000000000000000000000 0.00000000000000000000000000000000 1000000015047466219876688855040.00000000000000000000000000000000 NaN

if I switch to f64 using fast-floats it works fine. And also it works fine if I remove fast-floats and use the primitive f32. What could be happening ? If I'm multiplying by 0, why doesn't it give 0 instead of NaN ?

Thank you !

2 Likes

Presumably the fast-floats is reordering your multiplication and you're getting overflow. Infinity times zero is NaN.

It's worth keeping in mind that fast math isn't supposed to give you the correct answer.

4 Likes

Enable fast-math mode. This option lets the compiler make aggressive, potentially-lossy assumptions about floating-point math. These include:

  • Floating-point math obeys regular algebraic rules for real numbers (e.g. + and * are associative, x/y == x * (1/y) , and (a + b) * c == a * c + b * c ),
  • Operands to floating-point operations are not equal to NaN and Inf , and
  • +0 and -0 are interchangeable.

(emphasis mine)

If a * b * c is reordered to a * c * b (allowed by -ffast-math) then you get Inf * c. LLVM assumes that no operands are not Inf (because of -ffast-math) and as such Inf * c is allowed to return anything including NaN.

2 Likes

Perfect. I don't understand why fast math operations work well in C but badly in Rust ? and why if I use f64 (fast) it works well, but not with f32 (fast)

With f64 it just so happens that there is no overflow to Inf. There should be no difference between clang compiled C and rustc compiled Rust. Both use LLVM as backend. Maybe you were comparing it to gcc compiled C?

I'm using ICC for C with this options:

-lm -xCORE-AVX512 -fp-model fast=2

What happens if you use clang with -ffast-math?

I cant install clang on the server, but I tried with an online compiler and I got 0

If it could overflow to Inf, then it is undefined behavior, thus any result is allowed, including NaN and 0. Additionally with UB there is no guarantee that it is consistent.

2 Likes

The problem is that if I don't use fast-floats. my program becames 3 times slower. In that cases what could I do ? Of course I could use fast-floats f64 (obviously slower than f32), but I find it strange that in ICC I can calculate it well with f32 and it works.

You cannot expect consistent behavior between spec-compilant compilers if you're touching UB. Undefined Behavior means the behavior is undefined so anything can happen.

Modern compilers use this undefined-ness for optimization invariant. If you don't touch the UB the behavior must not be changed after optimization. If you touch the UB invariants are broken and the code become garbage, but it's ok since there's no defined behavior here.

It seems the ICC doesn't have some optimizations the LLVM have. Sounds reasonable, nowadays ICC isn't the strongest compiler in the world.

2 Likes

A standard trick in floating point computations is to do things the fast way and check for NaN. If you don't get a NaN, you're good to go. If you do, you repeat the calculation the slower but safe way.

Standard trick cannot be applied on the fast-math. As bjorn3 said in the fast-math mode compiler assumes there's no NaN/Inf/overflow/signed zero, and since it's originated from the C compiler if assumption becomes wrong it's UB.

2 Likes

You can always redo a failed calculation with f64 if the problem only occurs with f32.

If the failed calculation involves UB, there no reliable way to detect so. If it just gave you a bogus answer, maybe there's some check you can apply that the answer makes sense for your problem. But if that UB led to memory corruption or nasal demons, there's no redoing it...

1 Like

Agreed, but the reported symptom is NaN, which gives some hope.

The problem is that you can't check for NaN after fast math, because LLVM knows that it can't return NaN -- after all, you said it doesn't when you picked fast math.

1 Like

That's the comment I am responding to.

I know what you're replying to. My point still stands -- it might have looked like it was NaN that time, but that just means it's UB, and you can't check for that.

A simple example that you can't check for NaN after doing "fast" math:

(Spoiler: The function gets optimized to return "ok".)


EDIT: Hmm, this crate is just fundamentally unsound. It exposes a + to safe code that can create poison in LLVM, and thus can trigger UB.

Though with new LLVM getting freeze it might soon be possible to expose this to safe code soundly...

6 Likes

This discussion about UB and fast math is super educational for me! As far as solving the original problem, perhaps there's an algebraic approach that could help?

If the original application allows for it, could you do the entire computation in log-space? I.e. instead of a * b * c, instead compute log(a) + log(b) + log(c)? Obviously it depends how you need to use the output. This is a common trick in machine learning because it's good for minimizing a function that is a product of many variables.

Alternatively, if you're expecting that one of the numbers may be exactly 0 (or almost 0) you might be able to use an if statement to detect that case and return a 0.

If neither of these tricks work, maybe sharing some more details of your application could help in determining a solution.

1 Like