Formatting floats and rounding

1

Guess what this prints:

fn main() {
    println!("{:.2}", 0.015f64);
    println!("{:.2}", 1.015f64);
    println!("{:.2}", 2.015f64);
    println!("{:.2}", 3.015f64);
}

Okay, I'll tell you:

0.01
1.01
2.02
3.02

Why the strange rounding behaviour? Can I somehow enforce the familiar semantics (round up)?

2

Try again with {:.20} and you'll see why... it's the old problem of difference between input (decimal literals) and representation (binary floats).

3 Likes
3

They have different binary exponents, which affects the level of precision represented in the mantissa.

4

Birkenfeld is right. I was being naive thinking 1.015 would get rounded up; it is first rounded to the nearest binary representation (either slightly less or slightly more), and of course 1.014999999.. is correctly rounded to 1.01 not 1.02. Printing with {:.3} rounds back to .015 so the final rounding step is correct.

I guess for correct rounding I need to use a decimal format (integers or something like decimate.

1 Like
5

I didn't say birkenfield was wrong, but I stand by my statement. The different binary exponents explain why they have different nearest binary representations. A lower exponent means the mantissa is representing smaller pieces.

Python makes it easier to see decomposed floats:

>>> for f in (0.015, 1.015, 2.015, 3.015):
...     print(f.hex())
...
0x1.eb851eb851eb8p-7
0x1.03d70a3d70a3dp+0
0x1.01eb851eb851fp+1
0x1.81eb851eb851fp+1

Printing the mantissas in binary:

>>> for f in (0.015, 1.015, 2.015, 3.015):
...     print(format(int(f.hex()[4:-3], 16), "052b"))
... 
1110101110000101000111101011100001010001111010111000
0000001111010111000010100011110101110000101000111101
0000000111101011100001010001111010111000010100011111
1000000111101011100001010001111010111000010100011111

Manually aligned to compensate for the different exponents:

        1110101110000101000111101011100001010001111010111000
 0000001111010111000010100011110101110000101000111101
0000000111101011100001010001111010111000010100011111
1000000111101011100001010001111010111000010100011111

So I hope you can see how the effective precision of the mantissa made the difference between .0149999... and .015000...

4 Likes