Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

>As a side note, Excel also uses floats for currency

It's still problematic, but the defaults and handling there avoid some issues. So, for example:

Excel: =1.03-.42 produces 0.61, by default, even if you expand out the digits very far.

Python: 1.03-.42 produces 0.6100000000000001, by default.



Excel rounds doubles to 15 digits for display and comparison. The exact precision of doubles is something like 15.6 digits, those remaining 0.6 digits causing some of those examples floating (heh) around.



A lot of these edge cases are about theoretical concerns like "how many digits we need in decimal to represent an exact IEEE binary float".

In practice a double is 15.6 digits precise, which Excel rounds to 15 to eliminate some weirdness.

In their documentation they do cite their number type as 15 digit precision type. Ergo that's the semantic they've settled on.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: