The Trouble With Rounding Floats

lukfil writes "We all know of floating point numbers, so much so that we reach for them each time we write code that does math. But do we ever stop to think what goes on inside that floating point unit and whether we can really trust it?"

Comments

Decimal Arithmetic

[(1 -sqrt(5))*(2**-1)]

From TFA:

Example 1: showing approximation error.

// some code to print a floating point number to a lot of
// decimal places
int main()
{
float f = .37;
printf("%.20f\n", f);
}

The main problem with that example, I take it, is that single-precision datatypes are only guaranteed for roughly seven decimal places; using double, of course, only defers the problem.

What about encoding floats as a pair of ints or longs: one to express the numerical value, and the other its tenth power; id est, decimal arithmetic?

Re:Decimal Arithmetic

[Anonymous Coward]

This is not newsworthy. This is computer science 101.

Re:Decimal Arithmetic

[SageMusings]

Okay,

Show of hands: Who did not already understand that floats are approximations? Anyone? I didn't think so. I've gotta wonder why this story ever made it into Slashdot. This is more worthy of Time magazine where it can be spun as a startling new revelation into the dirtier corners of computer science and foisting a lie on the public.

Re:Decimal Arithmetic

[tomstdenis]

My college taught "numerical analysis" in the software comp.eng side. You learn the format of IEEE types, the range, accuracy, precision issues, etc.

We had assignments to not only perform matrix ops but also give the expected error, etc.

Maybe the author of the article should either go to a better school or pay more attention to the classes.

Tom

Re:Decimal Arithmetic

[Bender0x7D1]

Exactly. Unfortunately, there are too many people out there who are programmers, even good ones, who don't know, or understand, the basics. While I'm not claiming that formal education is the only way to get the knowledge you need, it is a good way to avoid gaps in your knowledge. I hated some of the computer science classes I had to take, but I did learn something important in each and every one of them.

Another advantage in the formal classes is you get the theory that allows you to make decisions on what data types to use and when. Sometimes you need the precision of BigNum systems, (crypto for example), and sometimes the accuracy of float is enough. For example, in a lot of financial applications, float would be good enough since 2 decimal places is enough. If you need performance, float will beat any BigNum system hands down. However, if you are dealing with decimals on top of decimals, (such as calculating someone's dividend from a mutual fund where they own partial shares), you might need BigNum. Either way, with the proper theory and good understanding of the formats, you can make these decisions.

These situations are why I am a big supporter of actual software engineering instead of programming. Sure, standard programming is great for a lot of situations, but serious applications need to use software engineering practices. You wouldn't build a bridge without an engineer, so why build an application that handles billions of dollars without applying the same rules and principles?

Re:Decimal Arithmetic

[gweihir]

These situations are why I am a big supporter of actual software engineering instead of programming. Sure, standard programming is great for a lot of situations, but serious applications need to use software engineering practices. You wouldn't build a bridge without an engineer, so why build an application that handles billions of dollars without applying the same rules and principles?

I could not agree more. The issue is not to get it done fast or cheap. The issue is that the person designing the solution does understand what the limitations of the tools used are. Anybody that builds mission critical stuff without good engineers as designers and supervisors gets what they deserve. Same is true anywhere. Trouble with programming is that bridges collapse far more newsworthy than software.

Re:Decimal Arithmetic

[modeless]

Floats where [sic] good enough. [...] Had to do a bunch of

decimal.parse(value.ToString())

to get things to sum up correctly.

Oh god. Please tell us which financial institution you worked for so we can all avoid it like the plague.

Re:Decimal Arithmetic

[Eivind]

It's not more "accurate". It's just that the inputs and outputs are often decimal, in which case using something other than decimal can give "unexpected" results.

For example for finance, floating-point is useless, people generally do something like use a single int to store number-of-cents.

The issue ain't accuracy per se, it's accuracy with *certain* numbers (thous representable in base10).

In a financial program people expect $0.40 * 1000000000 to come out as *precisely* 400000000 and not 399999999.99

There's lots of numbers that base10 can't accurately represent, such as 1/3. The thing is, you'll never have those as inputs if the inputs are stuff like, for example, prices.

Re:Decimal Arithmetic

[Nutria]

The RPG language also had an implementation of BCD,

Argh, "forgot" about RPG.

and probably any compiler for the IBM S/370 line would at least have a library for it,

Having a library isn't the issue. Early versions of TurboPascal also had BCD libraries.

That you used via function calls. Very not useful.

To be practical, a datatype needs to be usable by the 5 base arithmetic operators.

as I believe the IBM mini/mainframe architectures had implemented it in hardware.

The System 3x0 "CPU" is extremely CISC.

Re:Decimal Arithmetic

[Anonymous Coward]

The only error would be in our method of using division for conversion

Of course, the simple thing to do would be to always use cents, and just remember to print a . in the right place. "Fixed point" math has been around for a long time, and was really popular in games before we had stuff like 3d accelerators and the like ;)

Doesn't help with dealing with stuff like interest though. What's 1.27% of 24592 cents?

The author is seriously confused

[mlyle]

Apparently the author of the article didn't read the stories in RISKS that he cited. In particular, the 'pensioners being shortchanged' one talks about them not being paid interest on 'float'-- cash flow on transactions in progress. This has little to do with floating point numbers.

Similarly, the spacecraft problem mentioned is one of an errant cast, not because of dilution of precision in floating point calculations.

The author could really pick his examples better-- as mistakes in numerical programming happen often and are often of great import.

Not news.

[SJasperson]

This is not a new problem. Or an unsolved one. Is there any modern programming language that does not supply a data type or library with exact decimal arithmetic support? Using a float to represent monetary amounts and expecting them to be free of rounding errors is as stupid as using integers to store zip codes and wondering where the leading zeros went from all the addresses in New England. If you can't be arsed to choose the right data type get out of the business.

science; business

[bcrowell]

He talks about scientific applications, but actually very few scientific calculations are sensitive to rounding error. Remember, they sent astronauts to the moon using slide rules. Generally for scientific applications, you just don't want to roll your own crappy subroutines for stuff like matrix inversion; use routines written by people who know what they're doing. (And know the limitations of the algorithm you're using. For example, there are certain goofy matrices that will make a lot of matrix inversion algorithms blow chunks.)

For business apps, the classic solution was to use BCD arithmetic. But today, is it more practical (and simple) just to use a language like Ruby, that has arbitrary-precision integers, so you can just store everything in units of cents? A lot of machines used to have special BCD instructions; do those exist on modern CPUs?

Numbers and bases

[Todd Knarr]

We have the same problem in everyday numbers. Try representing 1/3 in any finite number of digits. You can't. The big thing about floating-point numbers that trips people up is that we're used to thinking in base 10. Floating-point numbers in computers typically aren't in base 10, they're in base 2. The rounding problem he describes is simply us getting confused and wondering why a fraction with an exact representation in base 10 doesn't have an exact representation in base 2. The obvious solution is the one he alludes to at the end: don't use base 2. Computers have had base-10 arithmetic in them for decades, in fact the x86 family has base-10 arithmetic instructions built in (the packed-BCD instructions). COBOL has used packed-BCD since it's beginning, which is why you don't find this sort of calculation error in ancient COBOL financial packages running on mainframes.

Re:Numbers and bases

[flibbajobber]

Try representing 1/3 in any finite number of digits. You can't.

"1/3"

You can. I just did. So did you. In base-10, even. In fact, the answer is the same for base-4 or higher. Using only two digits, "1" and "3". Any rational number can be represented using a finite number of digits, using... (wait for it) a RATIO.

(Represent one-third in Base 2? why that would be "1/11". One-third in Base 3 would be "1/10".)

Bah. Author doesn't understand arithmetic.

[swillden]

The author goes on and on about how floating point numbers are inaccurate, and unable to precisely represent represent real values, like this is something new, or even something different from the number approximations we normally use.

The reason the examples the author cites can't be represented precisely is that floating point numbers are ultimately represented as base-2 fractions, and there are a bunch of finite-length base-10 fractions that don't have a non-repeating base-2 representation. Guess what? We have *exactly* the same problem with the base-10 fractions that everyone uses all the time. Show me how you write 1/3 as a decimal!

The problem isn't that floating point numbers are inherently problematic, the problem is that we typically use them by converting base-10 numbers to them, doing a bunch of calculations and then converting them back to base 10. Floating point rounding isn't an unsolved problem -- floating point rounding works perfectly, and always has. It's just that the approximations you get when you round in base 2 don't match the approximations you get when you round in base 10.

Bottom line: If you care about getting the same results you'd get in base 10, do your work in base 10. This is why financial applications should not use floating point numbers.

Re:BCD isn't the answer

[Nutria]

I mean, do you need a cash register than can tally sums > $1000000?

You do realize that there's more to business than cash registers, right?

A 32-bit fixed point number maxes out at 21,474,836.47 which is severely limiting for all but small-sized businesses and tiny governments.

64-bit fixed point number (max 92,233,720,368,547,758.07) are obviously better, but are only efficient on 64-bit machines, which are still a minority of installed machines.