Slashdot Mirror
Floating Point Programming, Today?
An anonymous reader asks: "I'm rather new with programming and stumbled across these twe articles: The Perils of Floating Point from 1996 and What Every Computer Scientist Should Know About Floating-Point Arithmetic from 1991. I tried some of the examples in these articles with Intel's Fortran Compiler and g77 and noted that some of those issue reported no longer seem valid whereas quite a few still very much are around. Could someone, please, give me a pointer to some newer thoughts and/or new facts surrounding floating point programming. What has been improved since those articles were written? What is still the same? How is the future, especially with the new platforms IA64 and AMD64? I am most interested in the x86 and x86-64 architectures. Thank you for your kind help."
extend mantissa so there is enough overlap - usually involves some kind of multiple precision libraries like mentioned in other post GNU MP and many others. I've implemented one for my own use, too. Generally means lots of overhead since there will be less than 5% of operations actually benefitting from greater precision.
postpone such operations until there is overlap - store such numbers together and do operations on them together, too. Sometimes additions in loops will add up small parts so actually there will be overlap with big part and additions can be done with enough precision.
On a side, interesting thing is that in computers multiplications and divisions are better (that is more accurate) than additions and subtractions because of logarithmic format.
I know that Sun was working on a variable precision floating-point CPU. I'm not sure how that project is going and what the end effect is, but I remember it being an interesting idea.
Multiple precision libraries are usually decent with only one problem, they are always slower by a couple orders of magnitude than regular CPU operations, so using them is just such a pain.
iThink iHate iMod
PDA level mobile FPUs are very rare indeed. In practice, devices using the ARM family processors have no hardware float support. It's thus very important for developers to understand floating point intimately, so that they won't be left at the mercy of awful compiler-emulated floating point code. Of course, in those cases most code tends to orient itself for fixed point arithmetic. Fixed point calculations are much better suited for the integer crunching power of, say, the Intel XScale.
There are also good tradeoffs developers can make between floats and fixed point, for example by using block floating point (BFP) formats, where a whole block of values shares the same common exponent.
Now that 3D is really coming to mobile devices, plenty of people will get first-hand experience of emulating floating point for the first time since the 80's. :-)
Jouni
Jouni Mannonen | Game Designer, Consultant
But how is that achieved, if?
I guess one would go and hunt for some arbitrary precision library for integers or some intervals lib for exact error bounds.
Think for a moment that compilers came just with integer data types and you had a to get a floating point arithmetics library every time you want to use floating point arithmetics! (I can only remember old Apple ][ integer basic, where something like that might have been really happened :-)
Wouldn't you say that is too uncomfortable?
So why not make arbitrary precision integer calculations and interval arithmetics part of the compilers? (A compiler switch?)
My guess is that people would start to use these features more, if they were easy to add to existing software.
To some extend functional languages already offer certain integer operations with arbitrary precision. But I believe one could do much better.
Let us hope that future languages will have such extended support right built in.
Regards,
Marc
It only truncates when saving to memory, that is why you can get different results when optimizing than not optimizing with gcc (you can force gcc to truncate all the time by using -ffloat-store).
With gcc you can force the floating point calculations in the sse registers by -mfpmath=sse.
Any college level engineering numerical methods course will teach you all the pitfalls involved with using floating point calculations on modern processors and how to minimize the impact of rounding errors (cumulative and otherwise) on your calculations.
Hell, any decent numerical methods book should cover stuff like that as well.
Would you mind tell us what those "issues" where. Because the articles hardly deal with "issues" at all. What they deal with is the theoretic limitations that must exist in floating point, due to the fact that we have finite hardware, while real analysis assumes infinite precision. This should not have changed between 1991 and now (especially, since we have all standardized on IEEE floating point formats, but even if the article was from 1960, you should easily be able to "translate" it to your favourite floating point format (which is probably IEEE)).
Could someone, please, give me a pointer to some newer thoughts and/or new facts surrounding floating point programming.
There are very few new thoughts with regards to floating point programming, just as there are very few new thoughts on the use of "if-then-else"-branches or "while"-loops. Floating point programming is basically a solved problem. The only problem with it is that it sometimes flies in the face of intuition, and most programmers are ignorant about it. This has not changed since 1991 either.
The articles you mentioned are very good articles for understanding issues surrounding floating point. Just make sure you read them with your brain, instead of just feeding your favourite compiler with any examples you see.
What has been improved since those articles were written?
Speed. Computers have become faster. (It's possible that there also have been some minor software improvements such as an ISO C addendum clarifying tricky areas with rounding modes, or something like that.)
What is still the same?
Essentially, nothing have changed.
How is the future, especially with the new platforms IA64 and AMD64?
Very predictable. Nothing will change there either. Non-IEEE floating point vector instructions, or "multimedia" instruction sets will probably continue to be unstandardized and platform-dependent.
I am most interested in the x86 and x86-64 architectures
There is nothing special about those architectures with respect to floating point (well, the x86 reuses its floating point registers for MMX instructions, but you shouldn't need to know that unless you use assembler).
Basic rule of thumb, if you want it to be accurate don't use floating point.
Basic rule of thumb, determine what accuracy you need, then pick your number representation.
Those articles are still quite valid - and will remain so.
So long as a float is still 32 bits and a double 64, you'll get about that degree of precision. It's not that the hardware is inaccurate - they all do pretty much the best they can with the information provided.
Roundoff errors and other evils of floating point representations are here to stay.
However, you can't just automatically decide to punt and use fixed point arithmetic. There is a 'tension' between dynamic range and precision. If you want reliable precision, you can't have large dynamic ranges for your numbers and vice-versa.
The biggest and best improvement we've seen since the early '90s is that doing your work in double precision is much less of a penalty than it used to be (when compared to working in single precision or integers).
With 64 bit machines, we should expect that penalty to become yet smaller.
So if speed is an issue, modern machines can be more precise - but if speed was not an issue, machines of the early '90s were every bit as precise as the latest wizz-bang 64 bit CPU. IEEE math hasn't changed much (at all?) in that time.
www.sjbaker.org