Slashdot Mirror
Floating Point Programming, Today?
An anonymous reader asks: "I'm rather new with programming and stumbled across these twe articles: The Perils of Floating Point from 1996 and What Every Computer Scientist Should Know About Floating-Point Arithmetic from 1991. I tried some of the examples in these articles with Intel's Fortran Compiler and g77 and noted that some of those issue reported no longer seem valid whereas quite a few still very much are around. Could someone, please, give me a pointer to some newer thoughts and/or new facts surrounding floating point programming. What has been improved since those articles were written? What is still the same? How is the future, especially with the new platforms IA64 and AMD64? I am most interested in the x86 and x86-64 architectures. Thank you for your kind help."
I've worked on a couple of projects where this is very important. One was writing control software for metrology equipment, industrial strength QA kit that measured manufactured parts down to fractions of a micron or even nanometres to make sure they were in spec. Another was a geometric modelling tool used in CAD applications and the like.
In neither case am I aware of any physical real world failure caused by a problem with the floating point calculations. You do have to be really careful with manipulating the numbers, though.
For example, the loss of significance when you subtract can be horrible if you've got two position vectors close together, and you're trying to calculate a translation vector from one to the other. The error in that translation vector can be enormous if the points you started with were very close: you might get only one or two significant figures, when the rest of your values have 15 or more. If you're interested in the direction of the vector, that can give you errors of +/- several degrees!
Inevitably, there are always going to be bugs in complex mathematical software, and I've seen plenty of wrong answers from programs like the above. Fortunately, it's normally possible to have checks and balances that at least identify and highlight inconsistencies so, in the worst case, at least nobody relies on them. You can also use ruthless automated testing procedures, which run zillions of calculations every night and flag the smallest changes in the results, so no-one accidentally breaks a verified algorithm with a change later. The combination makes it reasonably unlikely that any algorithm would fail catastrophically with the sort of consequences you're talking about.
The possibility is always there, of course, because programming is subject to human error. However, FWIW, I've worked on software that's used to design cars, and software that controls the QA machinery to make sure they're put together right, and I still drive one. :-)
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
One interesting approach I have seen is the use of strings to store almost arbitrary decimal positions. You can set a maximum length, at which point it rounds. But the nice thing is that the rounding is done in the decimal number system instead of binary, so it is closer to how business managers expect it to be rounded (like you would do it on paper). This approach obviously is not ideal for scientific computing, but is geared toward business uses where rounding accuracy is more important than speed. PHP used to include the "BC" library that did this kind of thing. I don't know what happened to it.
Table-ized A.I.
Yes, this is pretty typical in most lisp or scheme implementations (it should have been in Python too, but for some reason isn't). Testing for overflow on e.g. x86 can be done by simply testing the overflow flag. Some 20 years ago, that might have been conceived of as fast.
But in order to be able to switch to larger representation, there needs to be an if-test somewhere. And that means there is a branch instruction behind every addition. Not fast. Especially on todays pipelined processors. In other words, the documentation/propaganda you've seen is lying.
However, what you loose in speed might not be that important, because, if you are lucky, you can arrange for that test to be needed anyways. This is because, when you have dynamic typing, an if-test would be needed anyways, and if you can arrange for your integers to be in some other range than pointers, then you can specualatively add (or whatever arithmetic operation you want) two fixed size integers before having tested that they really are fixed size integers, and then do something slow only if the result isn't what you should expect.
Not sure how well the same approach would work for floating point, but if you can do it reasonably efficiently for integral types, I guess why not?
Sure, you can do something similar. But it isn't necessarily faster or better because of that.
First, we can't check for just underflow or overflow or NaN's, if what we really are interested in is precision. So if we want to test precision, we need interval arithmetic (or something similar). This is already slowing down stuff by at least a factor of two.
Second, we need no just maintain this calculated precision value. We also need to monitor it all the time. This adds a lot of if-tests, slowing down the calculation even more.
Finally, if precision is too bad, we need to be able to rollback the current calculation. Because, if we do a calculation, and find that precision is lost beyond what is acceptable, then we need to redo the whole calculation, not just the last step. I have no idea what this will cost, but it will most likely be very expensive, and certainly complex.
My guess is that these three factors combine to make the proposed scheme rather unattractive combined with simpler solutions such as just using more bits in the first place.
FWIW, but the Intel Numeric Coprocessors have always done their math in 80-bit floats since their introduction, what was it about 20 years ago ?
in lisp 1/3 is stored as 1/3. Maybe the rest of the computer languages will catch up some day.
Do you even lift?
These aren't the 'roids you're looking for.
Float point is a well-defined and easy to understand representation. Of course, that doesn't mean it's easy to use--mathematically, it can be pretty complicated to deal with at times. Perhaps the biggest sin is to think of floating point numbers as "real numbers"--they aren't.
Unfortunately, IEEE 754, the most widely used floating point standard, fixes none of the complexities of using floating point but creates many completely unnecessary complexities of its own. Many CPUs just give up and throw any kind of specialized IEEE features into software, making them nominally compliant but unusable. And many programming languages refuse to implement the inane and broken semantics specified for IEEE comparison operators.
The only good thing that can be said about IEEE 754 is that even a lousy standard is better than nothing at all. And, on the bright side, you can usually put CPUs and compilers into modes where they behave somewhat sanely (no denormalized numbers, sane comparisons, no NaNs).