Slashdot Mirror
What Every Programmer Should Know About Floating-Point Arithmetic
-brazil- writes "Every programmer forum gets a steady stream of novice questions about numbers not 'adding up.' Apart from repetitive explanations, SOP is to link to a paper by David Goldberg which, while very thorough, is not very accessible for novices. To alleviate this, I wrote The Floating-Point Guide, as a floating-point equivalent to Joel Spolsky's excellent introduction to Unicode. In doing so, I learned quite a few things about the intricacies of the IEEE 754 standard, and just how difficult it is to compare floating-point numbers using an epsilon. If you find any errors or omissions, you can suggest corrections."
I seems to me that this problem would pop up any time you worked with an irrational number.
Back in the early days the analog computer was used for things like ballistic calculations. I would think that they would be less prone to this type of problem.
Linearity may still be an issue (analog systems have their own set of problems).
Tisha Hayes
This article should mention strictfp in the section on Java.
If you think that every language except Java implements IEEE-754 to the letter, you are sadly mistakenly. That fact is Java can be used just fine for floating point work in most applications.
You really need to talk about associativity (and the lack of it). ie a+b+c != c+b+a
This is actually non-commutativity. Non-associativity means (a+b)+c != a+(b+c).
Actually, the linked article says exactly the opposite, and up above I posted a link to the JVM definition that verifies it. So you are 100% incorrect.
to floating point, please, everyone should've read Everything you ever wanted to know about C types and part 2 (which explains fp too).
this will save a lot of time & questions to most beginning (and maybe mediocre) programmers.
>>> (1.0/3)*3
1.0
Anyone who loves or hates any language, platform, or manufacturer, doesn't know what they're talking about.
PLEASE stop this thread - interval arithmetic and all that has been around as a research topic for decades now.
If you did not know: there are search engines providing lots of links...
Java have a strictfp keyword for strict IEEE-754 arithmetic.
The non-trivial problems with floating-point really only turn up in the kind of calculations where *any* format would have the same or worse problems (most scientific computing simply *cannot* be done in integers, as they overflow too easily).
Floating-point is an excellent tool, you just have to know what it can and cannot do.
The illegal we do immediately. The unconstitutional takes a little longer.
--Henry Kissinger
People interested into floating point math will find some very interesting materials and horror stories in the documents collected at the home page of professor William Kahan, the man behind IEEE754 standard.
According to my personal experience the paper by David Goldberg cited in the post isn't that difficult after all. Plenty of interesting materials can also be found in the Oppenheim & Shafer textbook about digital signal processing.
And wrong. I don't know how to use Github and if he won't bother to post an email address, I won't bother to learn about Github just for this.
The comparison page is wrong. Take nearlyEqual(0.0000001, 0) for example. As the author said, using Epsilon can be bad if you don't know what you are doing. The correct form of the function is:
epsilon = 0.00001;
function nearlyEqual(a,b)
{
return (Math.abs(b) < epsilon) ? (Math.abs(a) < epsilon) : (Math.abs((a-b)/b) < epsilon);
}
Also, parentheses don't hurt you and they help the reader.
Yes, it's sarcasm. Deal with it!
Continued fractions are a straightforward way to implement exact computable real arithmetic. So yes, it's been used. And it's slow. But it is exact.
After all, I am strangely colored.
I think the original poster was referring to this piece by the father of floating point, William Kahan, and Joe Darcy
"How Java's Floating-Point Hurts Everyone Everywhere"
http://www.eecs.berkeley.edu/~wkahan/JAVAhurt.pdf
In case you weren't just being funny, that == is correct, as it's meant to prevent NaN or Infinity results from the division, which can only happen with the actual "zero" values.
The illegal we do immediately. The unconstitutional takes a little longer.
--Henry Kissinger
Well, so much for my smug mathy reply! Amusingly, the reason it worked so well is that my "Evaluate" statement asked Mathematica to symbolically evaluate the function prior to graphing. So, the graph looked nice because Mathematica was just graphing the "Abs[x]" function!
Without the symbolic evaluation or requesting a particular precision level, the graph you actually get is this:
http://www.untruth.org/~josh/real-rounding-oops1.png
You can get a more reasonable looking answer by messing with the calculation precision and accuracy...
Internal arithmetic always includes the exact solution, but only the rarest circumstances does it actually give the exact solution. For example, an acceptable interval answer for 1/3 would be [0.33,0.34]. That interval includes the exact answer, but does not express it.
It didn't, however, as you imply, have BCD hardware. In fact, it had no hardware at all for arithmetic. At the start of the core memory, you had two lookup tables, one for addition and one for multiplication. To add two values together, you iterated over each pair of (decimal) digits, looking up their values in the table and then storing the result somewhere. If one of the digits had the carry bit set, then you did the same thing with the result of adding the next two digits together to add the extra one.
The 1620 was nicknamed CADET by its designers for this reason - Can't Add, Doesn't Even Try. It was one of IBM's first attempts to make a cheap computer (you could buy a minimal unit for around $100,000 (paper tape instead of punch cards, only 10,000 characters of memory). It was the first computer my university ever bought, and the subject of an incredibly scathing review by Dijkstra.
I am TheRaven on Soylent News
Interval Arithmetic does *not* produce exact solutions, it provides upper and lower error bounds on the calculation. Most IA implementations introduce a bias on every bounds calculation to insure that the "exact" result is proven to be bounded by the interval. I think some SPARC processors had IA support.
Correction: COBOL, PL/I and Ada. Ada has both fixed and floating point BCD arithmetic. And yes I too wonder why it is not in wider use. Perhaps it has to do with the ill conceived notion of "light weight languages" - most of which are not light weight at all any more once they are on the market for decade or so.
Martin