I think it would be nice to have variant 2 ("explicit error handling on DirtyF64 -> F64 conversion") always on, and to be able to optionally turn on variant 1 ("check every operation and blow up when one fails") when a floating-point issue has been found by variant 2 in order to quickly isolate the faulty float operation that caused it. Curious about other opinions!
I agree with HadrienG. Variant 2 by default with an ability to turn on variant 1 for testing/debugging. I would lean towards a Result instead of Option. We should have ? on Option soon, but it could also be nice to know why it failed (ie Nan or infinite or maybe some would care about denormals).
Thanks for looking in to this. I am excited to see what you come up with.
Although this will not be an issue for Rust anytime soon, as rustc support for GPU backends is still at a highly experimental stage of development, I just wanted to mention that I checked my earlier tentative statement that IEEE 754 exception flags (or optional trap-based exception handling for that matter) cannot be assumed to be available on GPUs, by looking at what a couple APIs for programming them have to say on this matter.
OpenCL 1.2 reference manual:
Floating-point exceptions are disabled in OpenCL. The result of a floating-point exception must match the IEEE 754 spec for the exceptions not enabled case. Whether and when the implementation sets floating-point flags or raises floating-point exceptions is implementation-defined. This standard provides no method for querying, clearing or setting floating-point flags or rapping raised exceptions. Due to non-performance, non-portability of trap mechanisms and the impracticality of servicing precise exceptions in a vector context (especially on heterogeneous hardware), such features are discouraged.
GLSL 4.6 specification:
NaNs are not required to be generated. Support for signaling NaNs is not required and exceptions are never raised. Operations and built-in functions that operate on a NaN are not required to return a NaN as the result.
CUDA Programming Guide, "Floating Point and IEEE 754" white paper:
Trap handlers for floating point exceptions are not supported. On the GPU there is no status flag to indicate when calculations have overflowed, underflowed, or have involved inexact arithmetic.
So it does seem that in addition to being a pain to program generally speaking, the ability to discriminate "inf-as-an-extended-real" and "inf-as-an-error-code" via IEEE status flags may also be hardware-specific.
I don't know if it is worth the extra complication, but I was just thinking that maybe this could be set up both for cases where you care about Inf and those where you do not.
Maybe instead of just the F64 type you were talking about there could be two types depending on what guarantees currently need to be made. Maybe something like a NN_F64 that is never nan and NNI_F64 that is never nan or infinite.
This is the approach which noisy_float has went for. When I mentioned it previously, @madmalik pointed out that it feels like a library design cop-out:
Personally, what I would be most worried about is to keep the lines blurred between the two meanings of Inf in IEEE 754 (inf-as-an-extended-real and inf-as-an-overflow-error). I feel like this ambiguity makes it unnecessarily dangerous to use float-infinity as a kind of real-infinity.
It would have been nice if the IEEE 754 opted to use a variant of NaN as an error code for overflow, instead of infinity. But again, the javascript "must keep running" mentality struck...
Thats my feeling too. Imo it's more "rusty" to prohibit a valid use-case than to allow an invalid one.
Of course, a lot of textbooks algorithms use infinities. But my working theory is, that infinite values are used as (a) "not yet set" or (b) very big starting value that is reduced in following iterations.
For (a), Option would be clearer and in the case of (b) -- start with an infinite DirtyF64 and unwrap it at the end. If it's still infinity then, it's most likely a logic error anyway.
Please call me out if I overlook other uses of infinite values.
Sorry I guess I didn't read previous comments well enough. After reading through the comments thoroughly, I agree. It would be an unnecessary complication with dubious benefits.
One place where I've found them useful is computing a geometric mean: values.map(|x| x.ln()).mean().exp(). If one of the values is a zero, the geometric mean correctly comes out as a zero, using an intermediate value of −∞.
...which reminds me of another peculiar thing about Inf in IEEE 754: unlike NaN, it will not necessarily remain an Inf over the course of a computation. So on its own, the F64/DirtyF64 approach may not succeed at detecting overflow if it is applied to too large a computational chunk.
I've gone through all operations and tried to identify if they are safe link to playground
tldr: I think floor, ceil, round, trunc, fract, abs, signum, to_radians, cbrt, hypot, sin, cos, tan, atan, atan2, sin_cos are safe methods, also the operation neg. The methods min, max are kind of special, since they can eat NaNs.
I've examples for all the unsafe ones, so that should be settled. But of course i could have made mistakes in identifying the "safe" ones.
min and max appear to be special since they imply the use of < or > operators to select one of the arguments while many may expect a NaN poisoning type of result. Here are some links I found interesting from other languages discussing their reasoning for the behavior they chose:
https://github.com/JuliaLang/julia/issues/7866
https://ghc.haskell.org/trac/ghc/ticket/9530
https://caml.inria.fr/mantis/print_bug_page.php?bug_id=5781
https://stackoverflow.com/questions/25375294/max-and-min-with-nan-in-haskell
And a chart showing how some different languages handle min/max:
https://github.com/danluu/dump/tree/master/equals-transitive
I think that the debate for what f64 should do has no clear answer. I do think that the crate being discussed here should follow the NaN poisoning behavior since it is intended to help catch errors. I think it would be good to have min/max for F64 and DirtyF64 override the f64 behavior and take the NaN poisoning approach.
I have not found any other unsafe operations other than min/max either, but am no expert and easily could have missed cases as well.
I think the reason I'm fairly ok with Inf-on-overflow is that Zero has the same behaviour, just in the opposite direction. Multiply enough values with |x|>1 and you get Inf; multiply enough values with |x|<1 and you get Zero. And I'm not sure that I'd expect a hypothetical library to error for things like (0.5_f32).powi(200).
A possible design direction would then be to catch invalid operations (as exposed by NaN), but leave overflow and underflow alone. Perhaps it would be a better fit for the session type approach, leaving exhaustive error checking to the more expensive "check every operation" approach used by noisy_float. @madmalik, what would you think about it?
(Speaking personally, I would love to have a library which checks for overflow and underflow errors, as they are often an indicator that something goes very wrong in your code. But I understand that it may not be implementable in a cheap and portable way.)
Lazy me says: Maaan, I just implemented everything for the non-Inf version. xD
But seriously, that seems like the most sensible solution. Maybe optional panicing overflow checking in debug mode as a compile time configuration?
A little update: I've worked a bit on better errors in debug mode.
For example
fn main() {
let a = F64::try_new(0.0).unwrap();
let c = a / 0.0; // <- invalid operation
let d = c + 2.0; // <- consequential error
println!("{}", d.sanitize().err().unwrap());
}
will print FloatError at src/main.rs:3: Division 0 by 0 resulted in NaN .
So, when unwrapping a DirtyF64, the error will point to the line, file and operation that was responsible for the first NaN in the chain of calculations.
This works through following mechanism: In debug mode, all operations are checked. When a new error occurs, an error message is stored in a global lookup table and the NaN gets the index as payload. I use the backtrace crate to get the position of the caller of said operation.
What i like about that is that we can habe precise errors without altering the control flow in debug mode (which would happen for panicing operations).
This would in theory also allow to log overflows for debugging purposes.
The interface looks very nice! On the implementation side, I am a bit worried about the global lookup table part. Does this mean that float operations in multiple threads need to synchronize with each other? Wouldn't it be better to dynamically allocate a storage block for the error and store it alongside the NaN?
The global table is of type Mutex<Vec<FloatError>>, so yes, when an operation logs an error, it has to acquire a mutex lock.
But only the first operation in a chain of invalid operations logs the error, so i figured it'd not be excessive.
Storing the error beside the NaN would mean doubling the size of a f64 and (after padding) quadrupling the size of f32 in debug mode. Also, the memory layout would differ from debug to release mode. To be honest, I don't know if that is problematic in the real world.
But you do need to increase the size of a float as well when you store an index into the global table in it, right? I do not know your DirtyF64 layout, but I figured it had to look like this:
// Uses 64 bits + enum discriminant
enum EDirtyF64 {
Ok(f64),
NaN(usize), // Index into the global error table
}
...in which case a dynamically allocated error would take up exactly as much space while being easier to use and requiring no global table:
enum BDirtyF64 {
Ok(f64),
NaN(Box<FloatError>),
}
You are right, though, that with either of these layouts, the global table would only be touched at the time where the Ok -> NanError transition error occurs, so the performance impact of synchronization or dynamic memory allocation would be limited.
I store the index in the exponent of the NaN value (NaN tagging). So my layout is still just an simple wrapper around the float.
Sorry, I didn‘t explain that.