What Every Programmer Should Know About Floating-Point Arithmetic

-brazil- writes "Every programmer forum gets a steady stream of novice questions about numbers not 'adding up.' Apart from repetitive explanations, SOP is to link to a paper by David Goldberg which, while very thorough, is not very accessible for novices. To alleviate this, I wrote The Floating-Point Guide, as a floating-point equivalent to Joel Spolsky's excellent introduction to Unicode. In doing so, I learned quite a few things about the intricacies of the IEEE 754 standard, and just how difficult it is to compare floating-point numbers using an epsilon. If you find any errors or omissions, you can suggest corrections."

Comments

Simple, effective and useful

[Jorl17]

Excellent. The article may not be *that* specific or lengthy, but it helps out most people that have a simple problem. People with more than simple problems should know the standard word-per-word already ;)

Re:Simple, effective and useful

[Anonymous Coward]

If you do not mind, I would suggest the opposite: the author should return to square one and do his homework instead: this "new" topic has been around for decades - with the internet full of references, links etc. - maybe he never heard of search engines ?

Re:Simple, effective and useful

[Dumnezeu]

And wrong. I don't know how to use Github and if he won't bother to post an email address, I won't bother to learn about Github just for this.
The comparison page is wrong. Take nearlyEqual(0.0000001, 0) for example. As the author said, using Epsilon can be bad if you don't know what you are doing. The correct form of the function is:

epsilon = 0.00001;
function nearlyEqual(a,b)
{
        return (Math.abs(b) < epsilon) ? (Math.abs(a) < epsilon) : (Math.abs((a-b)/b) < epsilon);
}

Also, parentheses don't hurt you and they help the reader.

Re:Simple, effective and useful

[-brazil-]

I would suggest you heed your own advice: do some research before mouthing off. Yes, there's a lot of stuff about FP math. But there didn't seem to be anything that is both novice-friendly and comprehensive.

Re:Simple, effective and useful

[EvanED]

...the author should return to square one and do his homework instead: this "new" topic has been around for decades - with the internet full of references, links etc. - maybe he never heard of search engines ?

And yet how many sources say "compare floating point numbers by saying "x - y epsilon" instead of the better solution here? I know I've seen the former method many, many times; I'm not sure I've ever seen anyone articulate the problems with it and suggest something like the latter ones.

Re:Simple, effective and useful

[-brazil-]

On second glance, your code seems to be wrong too, because it uses epsilon as an *absolute* error measure.

Re:Simple, effective and useful

[Anonymous Coward]

It can't be "novice friendly" and comprehensive. That's why the standard is a pain in the butt, and why "What Every Computer Scientist Should Know About Floating Point Arithmetic" is a good reference.

If you want to use a better approximation to real arithmetic, use exact computable real numbers. It's much slower than floating point arithmetic, though.

Re:Simple, effective and useful

[tolkienfan]

How about:

return ! (Math.abs((a-b)/b) > epsilon);

Discuss.

Re:Simple, effective and useful

[-brazil-]

Sure it can be: by starting with simple explanations fit for novices (who usually aren't actually doing serious numerical math and simply wonder how come 0.1 + 02 != 0.3) and getting into more details progressively.

And I mention the alternatives to floating-point formats and when to use what.

Re:Simple, effective and useful

[poopdeville]

Take a class in real analysis.

Re:Simple, effective and useful

[gumbi west]

Your criticism is correct.

While your algorithm would say that plank's constant (6.6E-34) and the inverse speed of light in a vacuum (3.33E-9) are the same. You just underscore the fact that it isn't easy to write a good nearlyEqual. But I would expect the author of an article on the topic to write a good one. After all, it is what people are most likely to go to look for.

Re:Simple, effective and useful

[-brazil-]

Fails on a=epsilon/2 and b=0

Re:Simple, effective and useful

[SanityInAnarchy]

Github is a useful thing to "learn" anyway. It's trivially easy if you already know Git, which is also a useful thing to learn. Even without knowing Git, it's not difficult to send a message.

Also, I'd argue the use of an actual if/else in this situation is much more helpful than your use of parentheses.

Also, presumably this is JavaScript. If so, I find your lack of var disturbing -- does 'epsilon' really need to be a global variable? Even if it does, shouldn't it at least be named such that this is obvious?

Re:Simple, effective and useful

[-brazil-]

I have changed the comparison page, hopefully the algorithm is correct now.

Re:Simple, effective and useful

[safetyinnumbers]

The code on the site for

function nearlyEqual(a,b)

contains the line

if (a==0.0){

Clearly, this should be

if (nearlyEqual(a, 0.0)){

There. What could possibly go wrong now? :)

Re:Simple, effective and useful

[-brazil-]

In case you weren't just being funny, that == is correct, as it's meant to prevent NaN or Infinity results from the division, which can only happen with the actual "zero" values.

Re:Simple, effective and useful

[gumbi west]

No, it is still wrong. Part of the problem is that it is impossible for it to be right without an estimate for epsilon for a and b. But lets set that aside, because any general algorithm is going to have to.

What you are doing is saying that a and b are nearly equal when abs(log(a) - log(b)) -5 and then you try to deal with log(0). There are other problems. In addition, you didn't deal with log(0) corretly. Notice first that nearlyEqual(a,b) need not equal nearlyEqual(b,a), this is probably a property you are going to want.

In addition, you didn't deal with numbers on opposite sides of zero well. If a and b are equal to the positive float closest to zero and the negative float closest to zero you will call these not nearly equal. At the same time, epsilon and the float nearest to zero you would have called nearly equal. But I'd say there is a much better argument for the first pair being nearly equal.

Re:Simple, effective and useful

[tolkienfan]

Whoosh...

Re:Simple, effective and useful

[eugene ts wong]

Hi.

Thank you for sharing your information. I found it fascinating.

It was sad for me, in that I'll probably never use it, because I never program.

Re:Simple, effective and useful

[tolkienfan]

Commutativity is preserved by using Math.abs((a-b)/(a+b)) or test both Math.abs((a-b)/a) and Math.abs((b-a)/b).

I don't think you are correct about two numbers not being "nearly equal" when they are both close to zero, but with opposite signs. The function returns "true" in this case, no? Are you suggesting this is undesirable? I could see for some use cases that property might be undesirable, but if that's what you meant it wasn't clear. Certainly that property is desirable for some applications.

Re:Simple, effective and useful

[petermgreen]

I don't think you are correct about two numbers not being "nearly equal" when they are both close to zero, but with opposite signs. The function returns "true" in this case, no? Are you suggesting this is undesirable? I could see for some use cases that property might be undesirable, but if that's what you meant it wasn't clear. Certainly that property is desirable for some applications.
IMO this sort of thing is a good reason NOT to write a nearlyequals(a,b) function. That will just lull you into a false sense of security that the same rules are appropriate in every case.

You need to consider each case on it's own merits to decide what is meant by "nearly equals" in context.

In some cases that may be best defined in terms of absolute error, in some cases that may be best defined in terms of error relative to the value and in yet other cases it may be best defined in terms of the error relative to the current precision which is related to the value for larger numbers but becomes fixed for smaller (subnormal) numbers.

Re:Simple, effective and useful

[gumbi west]

Okay, you are (mostly) right, but what about when a = -b? I think you might want

|a-b| / [|a| + |b|]

now the only special case is when a=b=0.

Re:Simple, effective and useful

[Thinboy00]

What if epsilon is one ulp? Then epsilon/2==0.0.

Re:Simple, effective and useful

[Dog-Cow]

That would be because 0.1 + 02 is 2.1. :-)

Re:Simple, effective and useful

[-brazil-]

Still returns false for a=ulp and b=-ulp, I think

Perhaps the best solution is to stop thinking about a relative error margin between real numbers and instead consider the error margin as a maximum number of discrete ulp steps - I've added that suggestion to the page.

Re:Simple, effective and useful

[gumbi west]

The fix is to do

|a-b| / [|a| + |b| + epsilon]

But this has its own problems. I think going over why each one is wrong is the best idea. Because sometimes the objection doesn't apply, and it gives you and idea why nearlyEqual is never globally a good idea.

Consider a and b=a*(1+30*me), where me is the machine epsilon, then if you have c = b-a, it has about 2 digits of accuracy and should be treated as such. But you have to know the history of c to know that.

Re:Simple, effective and useful

[Slur]

Okay, maybe I oversimplify, but this seems fine to me:

function NearlyEqual(a,b) {
    epsilon = 0.00001;
    return (Math.abs(a-b) epsilon); // what could go wrong?
}

Re:Simple, effective and useful

[JWSmythe]

    That's what I was thinking too. But hey, what do I know, I just work computers, I'm not a mathematician. :)

    The way some folks do it,

0.1 + 02 = 0 + 2
0 + 2 = 2

    There was a thread on here a few weeks ago, where I explained it in the calculation of payroll. If you're calculating fractional hours, then those decimals come in handy.

    1 minute = 0.0166666666666667 hours.
    Depending on how many decimal points you make it, it can really mess with your pay.

    0.01 * 60 = 0.6
    0.02 * 60 = 1.2
    0.0166 * 60 = 0.996
    0.0167 * 60 = 1.002

    For hourly folks, check your paychecks. I'd bet the company is using the most advantageous rounding for their profit rather than for accuracy.

    I was recently told on a something that one interval = 0.0083333 (1/120), that it should always be simply cut off (not rounded) at 1 decimal point. I tried to explain, that would make the numbers totally wrong.

1 = 0.0
10 = 0.0

    10 instances of 10 would then be 0.0, rather than 0.8. They wanted "absolute" accuracy over thousands of instances, but still insisted chopping it off to one decimal place is the way they wanted it. *sigh*

    I do understand why floating point numbers can induce errors, but is it necessary to make it worse by adding in sloppy math?

Re:Simple, effective and useful

[Slur]

The Less-than is missing, but pretend it's there...

Re:Simple, effective and useful

[gumbi west]

This is an absolute comparison. Imagine comparing numbers such that a * me (machine epsilon) GT 0.00001, then b = a * (1+me) will not be nearlyEqual to a, but it is the closest number to a, so only a exactly will be nearlyEqual to a for any a of the type described. (my greater/less operators got dropped too despite my selected "text" entry format, so I used GT).

Re:Simple, effective and useful

[Anonymous Coward]

For a lot of purposes, 6.6e-34 and 3.33e-9 *are* the same.

The point is that it doesn't make sense to define what "almost equal" means without some knowledge about the problem you're trying to solve.

Re:Simple, effective and useful

[gnasher719]

Okay, maybe I oversimplify, but this seems fine to me: function NearlyEqual(a,b) { ... }

You'll have to add a few other functions. You'll want a function "PossiblyGreaterEqual" which returns true if a >= b or NearlyEqual (a, b) and "DefinitelyGreater" which returns true if a > b and not NearlyEqual (a, b); same for LessEqual. Then you need a new subtract operator that returns 0 for a-b if NearlyEqual (a, b) is true. And replacements for floor () and ceil (). It would be wrong if NearlyEqual (a,10) is true but floor (a) = 9.

And suddenly the thing gets very complicated.

Re:Simple, effective and useful

[tolkienfan]

Good point

#1 Floating Point Rule

[Anonymous Coward]

#1. Don't use Java, it's not IEEE-754 compliant for floating point arithmetic.

Re:#1 Floating Point Rule

[abigor]

"The floating-point types are float and double, which are conceptually associated with the 32-bit single-precision and 64-bit double-precision format IEEE 754 values and operations as specified in IEEE Standard for Binary Floating-Point Arithmetic , ANSI/IEEE Std. 754-1985 (IEEE, New York)."

http://java.sun.com/docs/books/jvms/second_edition/html/Overview.doc.html

Re:#1 Floating Point Rule

[lenmaster]

If you think that every language except Java implements IEEE-754 to the letter, you are sadly mistakenly. That fact is Java can be used just fine for floating point work in most applications.

Re:#1 Floating Point Rule

[kestasjk]

I'm not sure whether that is factually true, but IEEE-754 isn't exactly perfect or without alternatives so I wouldn't base my language choice on it..

That'd be like not using Java because it doesn't represent ints using ones complement; if your code relies on the specific internal implementation of data primitives you're probably doing something wrong.
(Before I get replies: Of course sometimes these things really do matter, but not often enough to dismiss a multi-purpose langauge.)

Re:#1 Floating Point Rule

[sdiz]

Java have a strictfp keyword for strict IEEE-754 arithmetic.

Re:#1 Floating Point Rule

[owlstead]

True, but in many cases (those that don't require too much performance) it is probably better to use BigDecimal anyway. I've never used that keyword, IMHO it's for inner libs only. It's certainly not something I would learn starting programmers (just as you should not learn new programmers the use of wait() or sleep(), rather CountDownLatch and the different Queue's).

Re:#1 Floating Point Rule

[Eharley]

I think the original poster was referring to this piece by the father of floating point, William Kahan, and Joe Darcy

"How Java's Floating-Point Hurts Everyone Everywhere"

http://www.eecs.berkeley.edu/~wkahan/JAVAhurt.pdf

Re:#1 Floating Point Rule

[Nimey]

It's probably not perfect, but IMO the reason for sticking with 754 unless there's a real need for something else is:

Repeatability. If your code and language are standard-compliant, then you'll get the same floating-point math results as someone using another compliant language on any other platform. Not crucial for some tasks, but it certainly is for others, such as scientific work.

Re:#1 Floating Point Rule

[gnasher719]

Repeatability. If your code and language are standard-compliant, then you'll get the same floating-point math results as someone using another compliant language on any other platform. Not crucial for some tasks, but it certainly is for others, such as scientific work.

Wouldn't it be great if you could change a switch in your computer to change all double precision fp from 53 bit mantissa to 52 bit, and if your results are suddenly radically different then you know your first set of results couldn't be trusted?

Repeatability is highly overrated. It's no good if you get the wrong results, and a different computer system gets you identical wrong results.

Re:#1 Floating Point Rule

[thoughtspace]

The standard is essential as it allows compatibility across CPUs.

For example, we write Windows (hence Intel/AMD) applications that communicate to Analog Devices DSPs.

Re:#1 Floating Point Rule

[Trepidity]

There are some decent points there, but a lot of them aren't really related to IEEE 754 compatibility. For example, bullet point #5 on their first-page list of five "gratuitous mistakes" is that Java doesn't support operator overloading. But by that standard, C sucks too, and yet is somehow used in lots of floating-point libraries.

Re:#1 Floating Point Rule

[Anonymous Coward]

Actually it's pretty important for multiplayer games to get deterministic results, because this is how many RTS and sports games are networked: starting with the same initial state they lockstep the player inputs forward in time, implicitly synchronizing the game without sending the state. For example most RTS games use this technique (eg. “Starcraft”, “Age of Empires”) also some physics simulation games like “Little Big Planet” do too, plus most sports games you play online as well.

And yes in this case, there is really no right or wrong. All you care about is the same result. *Exactly* the same result. Down to the bits. Unfortunately IEEE-754 makes no guarantee of this, and you'll find that in practice you'll get slightly different results across different architectures, compilers, hell - even between release and debug build.

http://gafferongames.com/networking-for-game-programmers/floating-point-determinism/
http://www.yosefk.com/blog/consistency-how-to-defeat-the-purpose-of-ieee-floating-point.html

Re:#1 Floating Point Rule

[spuk]

No... the #1 floating point rule is: do not compare for equality

Re:#1 Floating Point Rule

[petermgreen]

As I understand it there are two sides to the IEEE 754 issue.

The first is the data format. Most modern systems use the IEEE 754 formats so you can pass data arround pretty freely (there are some potential endian issues but those aren't too bad to deal with).

The second is rounding rules and the size of intermediate values etc. Afaict very few systems get these exactly as the standard specifies. For a well known example C compiled for the x87 will store values in floating point registers in much higher precision than the same value stored in memory in IEEE format.

Compilers will typically store variables both in memory (in the declared format) and in floating point registers (in the FPU internal format) depending on current register allocation pressure.

This is only a problem if you consider repeatability more important than accuracy.

Java is actually pretty unusual in giving you the option (though it doesn't do it by default) to force all intermediate values to be rounded to the variable's declared size.

Re:#1 Floating Point Rule

[Hurricane78]

Protip: Search for “strictfp” on http://floating-point-gui.de/languages/java/
Oh, and to the OP: EPIC FAIL!

Analog Computers

[Tisha_AH]

I seems to me that this problem would pop up any time you worked with an irrational number.

Back in the early days the analog computer was used for things like ballistic calculations. I would think that they would be less prone to this type of problem.

Linearity may still be an issue (analog systems have their own set of problems).

Re:Analog Computers

[maxume]

Precision isn't that big a deal (we aren't so good at making physical things that 7 decimal digits become problematic, even on something the scale of an aircraft carrier, 6 digits is enough to place things within ~ 1 millimeter).

The bigger issue is how the errors combine when doing calculations, especially iterative calculations.

Re:Analog Computers

[Anonymous Coward]

No, irrationality has nothing to do with it. It's a matter of numeric systems, i.e. binary vs. decimal. For example, 0.2 is a rational number. Express it in binary floating point and you'll see the problem: 2/10 is 1/5 is 1/101 in binary. Let's calculate the mantissa: 1/101=110011001100... (long division: 1/5->2/5->4/5->8/5=1,r3->6/5=1,r1->2/5->4/5->8/5...)

All numeric systems have this problem. It keeps tripping up programmers because of the conversion between them. Nobody would expect someone to write down 1/3 as a decimal number, but because people keep forgetting that computers use binary floating point numbers, they do expect them not to make rounding errors with numbers like 0.2.

Re:Analog Computers

[-brazil-]

True, but irrational numbers are those that cannot be written down exactly in *any* base - not even if you use recurring digits.

Re:Analog Computers

[Anonymous Coward]

That's why they never cause problems for novice programmers: Irrational numbers can't be written down in source code, so everybody knows there's going to be some rounding error. But when people enter an innocent looking decimal number, do some trivial math and get the "wrong" result back, then the pitchforks come out and programmer heads start rolling.

Re:Analog Computers

[Anonymous Coward]

2E-1
Oh my god! How did I do that?

Re:Analog Computers

[cupantae]

Irrational numbers are not such a problem as rational numbers which can't be represented in the base used.

Lets say our computer has 6-digit-decimal precision. If you add two irrational numbers, say pi and e, you'll get 5.85987. It's imprecise, but imprecision is necessary, since it can't be represented in any base.

But if you add 3/7 and 2/3 you get 1.90524 which is imprecise even though a precise answer does exist.

Re:Analog Computers

[moonbender]

True, but irrational numbers are those that cannot be written down exactly in *any* base

... except the irrational number's own base. ;)

Re:Analog Computers

[Chris Mattern]

The problem is, if you're doing a long string of calculations--say a loop that repeats calculations thousands of times with the outcome of the last calculation becoming the input for the next (approximating integrals often does this) then the rounding errors can accumulate if you're not paying attention to how the floating point works.

Re:Analog Computers

[pentalive]

Don't CAD systems represent everything in very large integers?

Re:Analog Computers

[Anonymous Coward]

Wrong is how you did that. 2E-1 is a decimal representation. We're looking for a binary representation. That means we need a number of the form m*2^n, where m and n are integers. 0.2 can not be expressed like that.

Re:Analog Computers

[Chris Newton]

Don't CAD systems represent everything in very large integers?

Not necessarily: the libraries I used to work on used floating point values for the APIs. Typically, there would also be explicit resolution/tolerance values provided, which would be chosen according to the scale of the model.

Re:Analog Computers

[Anonymous Coward]

The problem is, if you're doing a long string of calculations--say a loop that repeats calculations thousands of times with the outcome of the last calculation becoming the input for the next

Did you plan to verbosely describe iteration? It was already mentioned in the GP.

Re:Analog Computers

[adonoman]

Nobody would expect someone to write down 1/3

I use base 3, so 0.1 is a perfectly easy number to express in floating point.

Re:Analog Computers

[Anonymous Coward]

How about 000010E-0001?

Re:Analog Computers

[Anonymous Coward]

What's that supposed to mean? 2*2^-1? That's not 0.2. If you mean 2*10^-1, then no, that's not binary. Binary is base 2. You need to solve 0.2=m/2^n (m, n integers). 0.2=m/2^n <=> 2^n=5*m. Look at the prime factors: Left side: only twos, right side: at least one five. There can't be an integer solution.

Re:Analog Computers

[zippthorne]

Integration is supposed to reduce the error....

Re:Analog Computers

[RAMMS EIN]

``Nobody would expect someone to write down 1/3 as a decimal number, but because people keep forgetting that computers use binary floating point numbers, they do expect them not to make rounding errors with numbers like 0.2.''

A problem which is exacerbated by the fact that many popular programming languages use (base 10) decimal syntax for (base 2) floating point literals. Which, first of all, puts people on the wrong foot (you would think that if "0.2" is a valid float literal, it could be represented accurately as a float), and, secondly, makes it impossible to write literals for certain values that _could_ actually be represented exactly as a float.

Re:Analog Computers

[Anonymous Coward]

say pi and e, you'll get 5.85987. It's imprecise, but imprecision is necessary, since it can't be represented in any base.

How about in base pi+e, where it would be 10. You can always represent any number exactly if you get to specify the base. Conversion to other bases is a bitch, though

Re:Analog Computers

[Anonymous Coward]

Hogwash! I have no problem representing irrational numbers on one of my infinitely long tapes.

Re:Analog Computers

[Anonymous Coward]

True, but irrational numbers are those that cannot be written down exactly in *any* base - not even if you use recurring digits.

Just so not true that it is unbelieveable anyone would say that. 1/3 isn't irrational, and can be written with recurring digits. How about e? We use it for the base in Natural Logarithms, where it would be 10.

Re:Analog Computers

[-brazil-]

> 1/3 isn't irrational, and can be written with recurring digits

So where does that contradict my statement?

> How about e? We use it for the base in Natural Logarithms, where it would be 10.

I don't think anyone uses non-integer bases to write down fractions in positional notation.

Re:Analog Computers

[petermgreen]

Back in the early days the analog computer was used for things like ballistic calculations. I would think that they would be less prone to this type of problem.
With analog voltages/currents you have noise, with fixed size digital values* you have rounding errors. Both mean that you won't get an exact answer in most cases.

The big advantage of digital over analog is you can make the errors arbitrarily small. FAR smaller than you could reasonally acheive in an analog system.

The big disadvantage of digital is that rounding errors are less independent of each other than noise. Noise tends to cancel out in integration but rounding errors don't tend to cancel out anywhere near as well (consider the simple case of numeric integration of a straight line in fixed point for example: ALL the rounding errors would be in the same direction).

*This applies whether decimal or binary and whether fixed or floating point, it doesn't apply to things like bignums but there you have the problem that your numeric representation can end up arbiterally big.

Re:Analog Computers

[Carewolf]

and, secondly, makes it impossible to write literals for certain values that _could_ actually be represented exactly as a float.

Except any binary floating point number can be accurately converted to a decimal floating point number.

The ability of representation can be expressed as the prime-numbers in the base. base-2 can only represent divisions on the form 2^a accurately, base-10 can represent divisions by 2^a * 5^b accurately, which includes all 2^a forms. Similary a base-30 system could do division by 2, 3 and 5 accurately.

Re:Analog Computers

[mikael]

I get this problem when doing task parallel addition on a multi-threaded application. A single task that sums the results of a function gets the same result as N threads doing 1 function result each, but different results from N threads doing M function results. Each thread does get an unique set of tasks.

Re:Analog Computers

[petermgreen]

All depends on the *type* of error.

Integration over a time period rather than taking a single reading reduces noise. Rounding errors are NOT noise. Sometimes they are modelled as noise but under certain circumstances that model breaks down.

In particular if the numerical integration is implemented as a running sum and the input to the integration is relatively constant all the rounding is likely to be in the same direction and hence builds up in the integration rather than cancelling. In the extreme degenerate case the output may stop changing AT ALL.

One way around this would be to use a probabilistic rounding algorithm but I'm not aware of any system that does this. Other possibilities include increasing the precision and doing the integration as an addition tree [ ((a+b)+(c+d))+((e+f)+(g+h)) ]rather than a running sum [ (((((((a+b)+c)+d)+e)+f)+g)+h) ].

Re:Analog Computers

[Anonymous Coward]

You have a 2 in binary (10) and you move it one digit over (E-1) like scientific notation but scaled to integers and in binary. Notice how 2E-1 is an integer-based representation of a fraction, using 2 and -1, similar to 2E1 = 20.0.

In binary, one set of bits represents the significand and the rest represent the decimal position (including a sign for each). Assuming one byte for each with the exponent first and not using two's complement: 1000000100000010. Same with 0.5: 1000000000000101 (not 0.1 even though that would be half because it's scientific notation) or 1.5: 1000000000001111 (15E-1)

Re:Analog Computers

[maxume]

I don't know. I just made up an example where the inexact representation of a constant would not be a practical problem, and I tried to pick something that is pretty huge (I'm sure there are lots of things on an aircraft carrier where 1 mm matters, but they are they sort of things the designers try to get rid of, they are expensive to build).

(and sure, there are lots of tolerances in a car engine that are much less than 1 mm, but it is about 1 meter across, not 1/3 of a km)

Re:Analog Computers

[Anonymous Coward]

Really? What if the base is irrational?

I can represent pi exactly, in base pi.

Yes, it's well defined if you use the exponential function to represent powers of the base.

Re:Analog Computers

[Volguus Zildrohar]

But if you add 3/7 and 2/3 you get 1.90524 which is imprecise

That is really imprecise.

Re:Analog Computers

[JWSmythe]

    Well, it would depend on what you're doing the calculations for, and how you're doing them.

    Say it used diesel fired engines, and you were instructed to calculate the fuel consumption per engine revolution, and then apply that to a trip. I don't know the specifics on an aircraft carrier, so I'll just make up some numbers.

    At full speed, the ship travels at 12 nautical miles per hour (knots). The engines spin at 300rpm. It burns 1275 gallons of fuel per hour.

  That's 18,000 engine revolutions per hour, or 0.0708334 gallons per revolution.

    1,000 miles at 12 knots = 84.3333334 hours.

    If you are to travel 1,000 nautical miles, 18,000 * 83.3333334 = 1,500,000.0012 revolutino. At 0.0707334 gallons per revolution, that would be 106,100.100085 gallons.

    But knowing that it burns 1,275 gallons per hour at 12 knots, and you will be traveling for 83.3333334 hours, you will require 106,250.000085 gallons. Using the measure of gallons per revolution to try to come up with a very precise number to work with, you've actually fallen short by 150 gallons for the trip. I can imagine a slight embarrassment by having your aircraft carrier run out of fuel just 7 minutes from its destination.

    Using 7 decimal points of precision, when it's multiplied so many times, it can easily cause errors.

    I'd be pretty sure they aren't counting gallons per revolution, I only used that as an example of where errors could happen. If you're considering the full length of the ship, 0.1 inches is more than enough to believe you have a good number. :) I believe due to expansion of the metals, the total length of the ship may change more than that depending on if it's a hot or cold day. :)

Re:Analog Computers

[maxume]

Sure, you can make it a problem, but it isn't particularly insidious.

And the part where I say "The bigger issue is how the errors combine when doing calculations" is a pretty compact version of what you said.

Re:Analog Computers

[Anonymous Coward]

Good spot. Sorry about that!

Cupán Tae

Re:Analog Computers

[LordVader717]

It's not even that our data isn't "precise" enough, it's simply that the various methods we use for machine computation are non-associative. Smart algorithms can deal with typical numerical errors and only need crude precision. Dumb algorithms work much faster, but are prone to errors, despite the ludicrous precision of double-float numbers.

Re:Analog Computers

[cupantae]

I'm saying the answer isn't precise.

Re:Analog Computers

[Anonymous Coward]

Those are not binary floating point numbers. They're binary coded decimal floating point numbers, if you will.

Re:Analog Computers

[Anonymous Coward]

I was thinking of something equivalent to (in C):
struct float {int exponent; int significand;};

Re:Analog Computers

[Anonymous Coward]

I know what you were thinking of, but those are not binary floating point numbers, if you interpret them as significand*10^exponent. The radix point position is given as a power of 10, i.e. it is a decimal point. If however you interpret the data structure as significand*2^exponent, then they are binary floating point numbers and then they can't express 0.2 exactly.

Re:Analog Computers

[jesset77]

I don't think anyone uses non-integer bases to write down fractions in positional notation.

Oh, I've seen it done. It's a marvelous, and frightening world out there, friend! :D

Speaking of which, soon it will be my 1120.0012002.. th birthday! And, bonus points to whoever can decode that back out of base e ;3

Re:Analog Computers

[dzfoo]

In order words,
the bigger issue is how the errors combine when doing calculations, especially iterative calculations..

      -dZ.

Re:Analog Computers

[Anonymous Coward]

Actually, your example shows precisely why 6 digits is more than sufficient. As the GP stated, we can't build real-life machinery to 7-digit specs (100 ppb). For instance, the engines won't spin at 300.0000 rpm. It won't burn 1275.000 gallons of fuel per hour. In fact, with ships that size the acceleration and deceleration phases will be several miles, in which the fuel consumption is significantly higher - an effect 3 orders of magnitude more than your numerical precision.

So, in reality your example should be calculated with 3 or 4 significant digits, not 6 or 7.

Besides, the real problem comes into place when digits start to cancel. E.g. when you represent your aircraft carriers position using 6 digit coordinates, you've got a resolution of about 40 meters. Now, if you take GPS positions once per second, you'll have a precision of centimetres (there are very few civilian aircraft carriers ;) ). If you calculate the difference between 2 measurements, you can estimate your speed with an accuracy of centimeters per second. But if you had rounded both positions to 6 digits, they'd like be the same: Your typical aircraft carrier moves at 15 meters per second. Hence, your carrier's speed would be reported as "0 knots. 0 knots. 0 knots. 80 knots - whoa!"

For those who consider this funny: replace "carrier" with "Patriot missile". This very bug (diffference of big, rounded numbers) caused multiple Patriot missiles to fail in the first Gulf War.

Re:Analog Computers

[LordVader717]

But that isn't the issue. Double-float numbers are as precise as you'll likely ever need. Incidentally, when you ever come across 6-digit decimal precision?
And how often do you input perfectly rational numbers into a problem you will be using floating-point arithmetic for?

Re:Analog Computers

[Anonymous Coward]

So you're going to be 33 soon, kiddo. However, representations with a non-integer basis (which are called beta-expansions) are not unique. For example, there are different ways to write 3 as a fractional number in base e: You can start with 10.**** (1*e^1+0*e^0+....) and just as well with 2.**** (2*e^0+....). This is an undesirable property for a positional number system, because in a comparison of numbers, digits of higher significance can be smaller even if the number is larger.

Re:Analog Computers

[Anonymous Coward]

I understand that but does it really matter? It's storing a decimal float value using bits. It's not 'pure' binary but is that even used for floats?

Re:Analog Computers

[JWSmythe]

    Ya, I as actually assuming steady cruise for the ship, but still, you're absolutely right.

    On the GPS thing, it's pretty funny that you mentioned it. I've been writing my own GPS software here and taking into account errors GPS readings. In a static location (like, sitting on my porch with the receiver having a clear view of the sky) I plotted my location over a period on Google Earth. The dilution of precision is a bitch. I started plotting the "squiggles" (everywhere it said I was), along with the mean and median of the readings as points. The reported coordinates went all over the place, including a few extreme points over a mile away. I thought I could get accuracy down to inches with some method, but the best it's done has been about 1 meter. I was hoping to get the resolution "perfect" down to the point where the receiver is sitting to make a better for geocaching. Then it dawned on me. Even if I made my precision correct down to an inch, that still can't take into account the error when the person who created the cache location.

    I was hoping to do a little weather balloon exercise, sending a smart gilder up to 100k feet on a weather balloon, and having it fly home (i.e., landing on a runway for RC flyers). The best I could hope for without setting up my own ILS beacons would be getting it somewhere really close to home. Walking a few meters to recover it isn't as bad as driving for miles to recover it. :) Without good altimeter readings (like, echoing off the ground below it), it could flare and stall too high or too low pretty easily.

Re:Analog Computers

[petermgreen]

and sure, there are lots of tolerances in a car engine that are much less than 1 mm, but it is about 1 meter across, not 1/3 of a km)
Now lets assume there is a car engine on that aircraft carrier. Lets further assume that someone copies the "car engine" model into the "aircraft carrier" model and then later copies and pastes out the car engine and the machine it drives out of the aircraft carrier model to simulate it.

If the system is using global floating point coordinates and the engine wasn't pasted near coordinate zero then the model of the car engine copied out will be much less accurate than the model of the car engine copied in!

Re:Analog Computers

[Anonymous Coward]

It changes the mathematical operations completely. For example, to add two numbers with different exponents, you have to multiply or divide one number by powers of 10. With binary floating point numbers, you multiply or divide by powers of 2, which in binary is a simple bit-shift (just like multiplying by powers of 10 is a "digit-shift" in decimal). Computers use binary floating point numbers (there are software libraries for other number formats, of course, but those are comparatively slow).

Re:Analog Computers

[cupantae]

What on Earth are you talking about? The article is about errors that are much smaller than 10e-6. So that certainly is the issue. It's not about usefulness, it's about delivering answers that are as accurate as they can be.

Re:Analog Computers

[LordVader717]

I'm talking about numerical analysis. The problems don't only apply to numbers "that are much smaller than 10e-6", and it isn't about "about delivering answers that are as accurate as they can be". It's about numerical operations delivering vastly different results from what you intended. You imply that computers aren't precise enough for a given task, but that's really missing the point.

Re:Analog Computers

[cupantae]

Don't tell me what I'm implying when you clearly can't do basic reading comprehension. I just hope English isn't your first language.

Re:Analog Computers

[maxume]

I think I would phrase that as "You could try to paste a model of the car engine in, but it wouldn't be one anymore", or something like that.

I guess it is a good thing I don't write CAD software without thinking about it very much.

(The better example would have been to point out that 6 digits of precision is probably enough to choose which of your enemies eyeballs to aim the artillery at, if you can see well enough. Of course, the gun probably isn't that accurate.)

Re:Analog Computers

[Anonymous Coward]

I think understand now. I guess I just assumed the required logic was in the hardware. Thanks for the enlightenment.

Re:Analog Computers

[LordVader717]

Because your comments lack any meaningful elaboration, we're left to interpret your original post as best we can. So maybe you would like to explain why the decimal cutoff of rational numbers is a much bigger problem than irrational numbers and the problems we model with them.

Re:Analog Computers

[Anonymous Coward]

makes it impossible to write literals for certain values that _could_ actually be represented exactly as a float.

Wrong: Every number that has a terminating representation in base 2 also has a terminating representation in base 10.

Interval arithmetic

[SolusSD]

Floating point math should be properly verified using interval arithmetic: http://en.wikipedia.org/wiki/Interval_arithmetic

Re:Interval arithmetic

[harshaw]

Gah. Yet another unintelligible wikipedia mathematics article. For once I did like to see an article that does a great job *teaching* about a subject. Perhaps wikipedia isn't the right home for this sort of content, but my general feeling whenever reading something is wikipedia is that the content was drafted by a bunch of overly precise wankers focusing on the absolute right terminology without focusing on helping the reader understand the content.

Re:Interval arithmetic

[JoshuaZ]

Interval arithmetic is not a magic bullet. It has multiple problems.

First, some functions just don't work very well when you try to describe them using interval arithmetic and you get bounds that are much larger than what you want. Consider for example x^2+x on the interval [-1,.1]. You'll get a much larger interval for the final answer than what you can actually obtain in practice (you get a resulting interval of [-1,2] when in fact the function ranges from [-1/4,2]. There are ways of handling this with quadratic functions so you in fact get the correct interval (essentially by completing the square). However, for higher degree polynomials this doesn't work. Thus for example, x^4+x^3+x^2+x has the same problem around 0. And when you try more complicated functions like say x^2 + sin kx for large k, you get similar problems.

Second, in order to do interval arithmetic you need to implement the intervals upper and lower bounds as floating point numbers if you don't want to incredibly computationally intensive. If you use such an implementation then things can break down in certain extreme cases if you aren't careful. There are ways around this (essentially you have your lower interval always round down and your upper interval always round up) but this leads to more problems of the type in the previous paragraph.

Both these problems are (generally) solvable if one is careful but the upshot is that relying on interval arithmetic is not a guarantee of correctness.

Re:Interval arithmetic

[Anonymous Coward]

Everything should be explained clearly. If we bring everything to the retard level then you end up with a useless, nonsensical qualitative explanation which doesn't teach anything to anyone. So, instead of complaining that "math is hard" you should wonder why you aren't able to understand it to begin with.

Re:Interval arithmetic

[Anonymous Coward]

You don't have to read the whole Wikipedia article to understand interval arithmetic. This holds for other "unintelligible" Wikipedia articles. What you're looking for is a page that shows only a part of a wikipedia article so you don't feel bad about the rest. That's just lame. Learn to read only the part you need and move on.

Re:Interval arithmetic

[Anonymous Coward]

Well, I'm a math guy, so I like the detail, and while I don't read and understand every part of a wikipedia article on such a subject, I'm glad it's there in case I want to focus on a topic. I thought the intro, history and implementations sections are easy to understand, and give most people more information than they'd ever need. Is skipping the math stuff you don't care to spend the time understanding so hard? I do it all the time. It's nice though, that if I want to spend the time with it, that it's still there.

Re:Interval arithmetic

[Raffaello]

presumably because it was written by "overly precise wankers." I'm guessing that the "overly precise" bit is more of a barrier to effective communication than the "wanker" bit. Though maybe a wanker's ability to type is impaired because one his hands is busy...

Re:Interval arithmetic

[Anonymous Coward]

Gah. Yet another unintelligible wikipedia mathematics article.

It's not unintelligible, it's just a frolsnaz hibbowits of the unbersnot variety. Simplified, it takes the form

B@HHOYBIY#T$R(^FG(WEHF(&#@HF@NF@HFJHOYB#@(^(GFGBIOY9dfads)))))/2

This is perfectly clear to any postdoctoral student.

Re:Interval arithmetic

[Anonymous Coward]

Skimming the article now, it seems that I've read much more difficult to understand articles on wikipedia before. In fact, I'd argue that this is pretty easy as far as technical explanations go... and I only have mathematical knowledge up to university intro courses...

Am I missing something?

Re:Interval arithmetic

[EdIII]

Though maybe a wanker's ability to type is impaired because one his hands is busy...

For some of us it's both hands......

Re:Interval arithmetic

[Anonymous Coward]

Uh, that article just looks like normal maths to me. Not even overly precise maths. (Granted its not an overly technical subject from a mathematicians POV...).

But it is maths, getting the terminology right is crucial.

What's your objection? That a wikipedia article about maths is written using maths?

Re:Interval arithmetic

[Anonymous Coward]

Wikipedia is not and never has been a resource for your education and tutelage.

Re:Interval arithmetic

[Anonymous Coward]

Gah. Yet another unintelligible wikipedia mathematics article. For once I did like to see an article that does a great job *teaching* about a subject. Perhaps wikipedia isn't the right home for this sort of content, but my general feeling whenever reading something is wikipedia is that the content was drafted by a bunch of overly precise wankers focusing on the absolute right terminology without focusing on helping the reader understand the content.

They are not wankers, they are people doing exactly what you want. Writing an article that would help them. The problem is that the article that would help you and the article that would help them is not nearly the same.

Re:Interval arithmetic

[steelfood]

Well, duh. Wikipedia is a place to store and retrieve knowledge, not a place to transfer or teach it. Wikipedia can't replace a real human teacher or professor, or even a specialized book on a particular subject. The best it can do is either be an introduction to new knowledge, or a refresher to existing knowledge, whereas what you're looking for is the wide body of stuff in the middle. Knowledge that is very general will fit into the introduction category, while knowledge that is very specific will fit into the refresher category. This math article and other specific math articles fall into the latter category.

There's a reason why specialized fields have developed their own jargon, and math in particular has its own lexicon. It's not to keep the uneducated masses out; it's to keep the ambiguity of human spoken and written language from creeping in. If you want to learn the mathematics and you don't have the base to understand the lexicon, you're starting from 0 so take a class or otherwise find someone to teach you. If you can understand the lexicon, or you can figure it out because you already have taken a class or you intuitively understand the concepts involved, Wikipedia will give you tons of knowledge with a little effort.

If you're too cheap or too unmotivated to take a class or put the effort into learning about the lexicon first, then you're just lazy and not in any position to complain.

Re:Interval arithmetic

[harshaw]

Everyone's comment in this thread is correct per se, "you need precise language to describe math", "you didn't work hard enough to read the article", "you are lazy", "wikipedia is not for teaching", etc.

You are all correct.

However, I would like a resource that helps me understand the math *without* becoming an expert. Or at least better tools. Look - it's been 13 years since College. I don't remember all the terminology. However, there are many times when I would like to dive deep into a field of mathematics, or at least refresh my knowledge (stats is a good example). I find wikipedia less than helpful in this regard -or least that grokking the page takes a ton of time and effort.

Re:Interval arithmetic

[Anonymous Coward]

It's impossible to write an article that will make you dive deep into a field of mathematics effortlessly, so stop asking for one.

Re:Interval arithmetic

[Anonymous Coward]

http://simple.wikipedia.org/wiki/Main_Page

Re:Interval arithmetic

[Anonymous Coward]

I couldn't agree more. Every article about mathematics puts me off immediately, and makes be wish I never looked it up.

As a kid, I never understood why so many others had maths as their worst subject. Reading stuff like:

Whereas classical arithmetic defines operations on individual numbers, interval arithmetic defines a set of operations on intervals:

        T S = { x | there is some y in T, and some z in S, such that x = y z }.

when I just wanted to find out what this meant, really makes me hate maths.

Re:Interval arithmetic

[Anonymous Coward]

Grokking a new mathematical concept takes a ton of time and effort. I should know, I do it for a living.

Re:Interval arithmetic

[Anonymous Coward]

You'd rather read something that was simple to understand but factually incorrect?

Einstein had a quote for this.

Re:Interval arithmetic

[Frohboy]

Personally, I find MathWorld pretty good for that purpose. Most articles are pretty encyclopedic in nature -- providing first a simple, general explanation, then diving deeper into the more complicated stuff (where the math gets more advanced by necessity).

Unfortunately, the article on floating-point arithmetic is just a one-liner (likely because it's more computer science than math).

Re:Interval arithmetic

[Hurricane78]

I think Lockhart explained the problem very very well: The thing is, that a mathematical concept was once invented in need to solve a problem. Then it grew with its history. Until finally it becomes perfectly abstract.

But what people then do, to show other people that concept, is throw the final formula or abstract concept at you. Including inconceivable symbols, semantics and terms that you never heard of. Naturally, your brain shuts completely down, because it’s stuck an the first thing it does not understand, and hence never gets to the point where it’s explained. And even if it wouldn’t, the whole thing of course makes no sense and has no point at all.

Luckily there’s a workaround: Walk the same path. But with compacted optimized information. Or in other words: First and foremost read the history. Especially the starting point(s). And from that, step by step, get to the current state.
If you can do it right, you will be rewarded with a deep and natural understanding of the matter, and will be able to come up with novel ways to think about it or use it, that people who learned it the “brainfuck” way, or learned it by heart, will never be able to come up with. :)

.9999999984 Post

[tonywestonuk]

Damn...Missed it! lol

Re:.9999999984 Post

[Yvan256]

I see you're still using that Pentium CPU.

Re:.9999999984 Post

[Anonymous Coward]

This is what happens when you use intel cpus, if you want a ifsrt spto use an athlon and nothing can go rwnog.

Re:.9999999984 Post

[CrazyJim1]

The XKCD article says how to screw with people who expect floating point errors. What about teenagers who were just learning how to program when the Pentium came out? Without the web, everyone was like isolated if they knew no other programmers. I wrote handling code to deal with it, but still! It took me like 2 weeks to figure out that I wasn't doing anything wrong in my code. Everyone hammers in your mind,"THE COMPUTER IS NEVER WRONG, it is always human error", and then your computer is wrong. It was a mind trip to say the least.

Re:.9999999984 Post

[cephus440]

Nah, he has a Cyrix Cx486DX CPU.

Re:.9999999984 Post

[Splab]

Bullshit.

1. The web was very much alive when the FDIV bug was discovered.
2. I seriously doubt you as a teenager spent 2 weeks finding this bug, this guy (who happens to be the one who found it) spent some weeks digging through everything to prove it was the FPU that bit him:
http://www.trnicely.net/pentbug/pentbug.html

Oh, and even when the "computer is wrong" it's still a human error.

Only scratching the surface

[ameline]

You really need to talk about associativity (and the lack of it). ie a+b+c != c+b+a, and the problems this can cause when vectorizing or otherwise parallelizing code with fp.

And any talk about fp is incomplete without touching on catastrophic cancellation.

Re:Only scratching the surface

[Anonymous Coward]

If you're interested in that, you're better of reading the article by David Goldberg (linked in TFS and the first paragraph of TFA). The whole point of -brazil-'s page is that it sums up some essential issues for novice programmers that find those in-depth descriptions complicated and daunting. Better to keep his page simple than duplicate an already-existing article and become just as inaccessible to newbies.

Re:Only scratching the surface

[Anonymous Coward]

You really need to talk about associativity (and the lack of it). ie a+b+c != c+b+a

This is actually non-commutativity. Non-associativity means (a+b)+c != a+(b+c).

Re:Only scratching the surface

[Anonymous Coward]

a+b+c != c+b+a is about commutativity - I think you mean (a+b)+c != a+(b+c) - that is associativity (or rather, lack of it).

Re:Only scratching the surface

[TheRaven64]

The lack of associativity is a bigger problem than you might think, because the compiler can rearrange things. If you're using x87 FPU code, you get 80-bit precision in registers, but only 64-bit or 32-bit precision when you spill to the stack. Depending on the optimisations that are run, this spill happens at different times, meaning that you can get different results depending on how good your register allocator is. Even upgrading the compiler can change the results.

If you are using floating point values sensibly, then you are aware that they only guarantee a certain minimum precision and that you shouldn't rely on the low bits too heavily.

Re:Only scratching the surface

[Anonymous Coward]

Its difficult to say when he is adding 3, not 2 arguments...but yes, what he was trying to express was non-commutativity.

Re:Only scratching the surface

[Anonymous Coward]

Floating point arithmetic is both non-commutative & non-associative.

0.5 * 5000000000.78987 * 2000000000000000000 is dependant on the order you do the mutiplication (these particular numbers chosen from thin air, but just demonstate the pattern that'll get you in trouble).

Re:Only scratching the surface

[Anonymous Coward]

Wrong. FP arithmetic is commutative, so his example can't be non-commutativity. It's actually a hidden case of non-associativity:
a+b+c = (a+b)+c = (b+a)+c = c+(b+a) != (c+b)+a = c+b+a.

Re:Only scratching the surface

[Dylan16807]

Actually, since .5 only affects the exponent, not the mantissa, your example is probably fine.

Re:Only scratching the surface

[Anonymous Coward]

I don't know why you'd expect a vectorized port of a SISD code to produce identical results, nor why you would make this a particular goal. All floating point arithmetic is inaccurate -- who is to say the original result was correct and the new one was wrong? Maybe the original was wrong. Maybe your baseline was computed on a Pentium with the FDIV bug. Who knows?

If you rely on floating point calculations to come out just exactly a certain way, I wonder whether you are trying to use float results in the logical flow of algorithms. That's really the biggest no-no as far as I'm concerned. Floats are for computing more floats, not controlling flow of execution. You never have to worry about whether two floats are "close enough to be equal" if you never test them for equality in the first place. Testing equality means doing logic with floats. Purge this concept from your mind. You don't do logic with floats.

Re:Only scratching the surface

[ameline]

The brackets were implied by the ordering. Yes, a slightly non obvious case that comes up fairly often -- optimizing the computation of dot product of long vectors. This involves summing lists of numbers, and changing the order of summation (either through vectorization or threading) changes the results. Kahan summation would help if compilers did not break it.

Re:Only scratching the surface

[Lamesword]

To those saying that a+b+c != c+b+a is about noncommutativity rather than nonassociativity: if you look carefully, this is about nonassociativity. The left expression (typically) parses as (a+b)+c; the right expression parses as (c+b)+a. In the presence of commutativity, which we do have with f.p. arithmetic, this is saying (a+b)+c != a+(b+c), which is nonassociativity.

Re:Only scratching the surface

[ameline]

If you're using the x87, just give up. It is very hard to efficiently conform to IEEE on that evil beast. (even setting the control register to mung precision only affects the fraction, not the exponent, so you still have to store to memory and reload to properly set precision.)

A former colleague described it (the entire x87 unit) as "Satan incarnate in silicon". :-)

Re:Only scratching the surface

[Anonymous Coward]

Sorry, but a + b + c != c + b + a is an example of the lack of commutativity, not associativity. To demonstrate the lack of associativity, one needs (a + b) + c != a + (b + c)

Re:Only scratching the surface

[gnasher719]

If you're using the x87, just give up. It is very hard to efficiently conform to IEEE on that evil beast. (even setting the control register to mung precision only affects the fraction, not the exponent, so you still have to store to memory and reload to properly set precision.)

Modern compilers use the SSE registers for double precision, so you don't have to worry about that anymore. And for 80 bit long double the original floating point registers behave correctly.

Re:Only scratching the surface

[Khashishi]

The GP was probably using implied parentheses, ie, (a+b)+c != (c+b)+a.
In this case, the inequality could be due to either non-commutativity or non-associativity, but in this case, it is due to non-associativity.

Re:Only scratching the surface

[5plicer]

Yes, Kahan summation helps (as long as the compiler doesn't break it), but it doesn't completely eliminate the problem. Shewchuk summation, on the other hand, does: http://code.activestate.com/recipes/393090-binary-floating-point-summation-accurate-to-full-p/

Re:Only scratching the surface

[Anonymous Coward]

Yes he does, and he also glosses over epsilons. Relative and absolute errors are *both* useful in different cases (e.g. if you constrain your data into a unit hypercube then absolute error works brilliantly). And relative error fails badly enough at zero that it needs more explanation.

His rejection of integers for financial calcs is also incredibly misguided and naive. Yes, integer calculation is more difficult, and requires more attention to rounding - but rounding is part of the financial domain and you do not want to use default FP rounding just because it's easier. The main problem he states - of currency conversion - is the place where maybe you do want to go to floating-point, but then there is the same limit problem (above a certain value your delta becomes 2 cents and you're now *wrong*). If you don't understand floating-point at least as well as the author the best advice is still to use fixed-point, but maybe synthesize a uint128_t for range.

However, he does touch on catastrophic cancellation a bit (right at the end).

Re:Only scratching the surface

[Anonymous Coward]

The property you are pointing out is commutativity which does not hold for a lot of things (matrix multiplication for example). Associativity is (a + b) + c = a + (b + c).

equivalent to Joel Spolsky's excellent introductio

[Anonymous Coward]

equivalent to Joel Spolsky's excellent introduction to Unicode.

Sorry, but his introduction to unicode is shit. Nobody anywhere should read or follow the advice therein.

strictfp

[lenmaster]

This article should mention strictfp in the section on Java.

Re:strictfp

[-brazil-]

Done :)

Re:strictfp

[owlstead]

Not really. It might point to BigDecimal, but leave strictfp out of it. Remember, this is for starting programmers, not creators of advanced 3D or math libs.

Re:strictfp

[petermgreen]

Strictfp gives you more repeatability but less performance and generally less accuracy.

It has it's place (e.g. when you have a fixed set of input data and need to be able to accurately repeat the exact same results) but generally if you think you need it you probably either misunderstood what it does or have an underlying design flaw in your system.

Re:strictfp

[lenmaster]

Strictfp gives you more repeatability but less performance and generally less accuracy.

strictfp can actually improve performance, depending on the CPU architecture and Java implementation. Think about it. If variables are stored in IEEE format and a different format is used for intermediate calculations, then there is converting going on and that takes time.

The difference in accuracy is not significant in any practical sense. The IEEE formats were designed so as to divy the bits up between the exponent and the significand in a way that works for nearly every application.

Its worth it for a Java developer to be aware of strictfp. We have all been sold on the slogan, "Write Once, Run Anywhere", but it really should be "Write Once, Run Anywhere...But Get Slightly Different Floating Point Results Unless You Use strictfp".

Re:strictfp

[-brazil-]

> If variables are stored in IEEE format and a different format is used for intermediate calculations, then there is converting going on and that takes time.

No. The different format for intermediate calculations is what the CPU uses internally (usually 80 bit, using a 64 bit mantissa to reuse 64 bit int circuits), so converting to and from that happens anyway and is built into the hardware. But if intermediate results have to be converted to IEEE 754 format, *that* takes extra work.

If you want accuracy...

[Nutria]

use BCD math. With h/w support it's fast enough...

Why don't any languages except COBOL and PL/I use it?

Re:If you want accuracy...

[larry bagina]

how do you express 1/3 in BCD?

Re:If you want accuracy...

[vlm]

$x_numerator = 1;
$x_denominator = 3;

Algorithms do exist for fractional number arithmetic. If the denominator gets unwieldy, who cares, its a computer and its fast and memory is "free".

Re:If you want accuracy...

[Nutria]

how do you express 1/3 in BCD?

Just as in "paper" decimal arithmetic, you must truncate it somewhere.

Since I've long forgotten how to "punch" the sign, this is what it would look like in 8(2) imaginary "unsigned" BCD, in hex: 00000033.

Re:If you want accuracy...

[JamesP]

Maybe because BCD is the worse possible way to do 'proper' decimal arithmetic, also it would absolutely be very slow.

BCD = 2 decimal digits per 8 bits (4 bits per dd). Working 'inside' the byte sucks

Instead you can put 20 decimal digits in 64bits (3.2 bits per db) and do math much more faster

Why don't any languages except COBOL and PL/I use it?

Exactly

Re:If you want accuracy...

[larry bagina]

on paper, I'd express it as 1/3 and not truncate it.

Re:If you want accuracy...

[TheRaven64]

also it would absolutely be very slow

Depends on the architecture. IBM's most recent POWER and System-Z chips have hardware for BCD arithmetics.

Re:If you want accuracy...

[Nutria]

Maybe because BCD is the worse possible way to do 'proper' decimal arithmetic,

"0.1 + 0.2 = 0.30000000000000004" doesn't really seem all that proper to me.

But BCD *does* do "0.1 + 0.2 = 0.3".

also it would absolutely be very slow.

Without h/w support.

Instead you can put 20 decimal digits in 64bits (3.2 bits per db) and do math much more faster

I want accurate math, not estimates.

Exactly

Do you pride yourself a Rational Man, or a low down dirty bigot?

Re:If you want accuracy...

[TerranFury]

A rational number class seems like a better solution, though there are some issues with representing every number as a ratio of integers... For instance, you need extremely large numerators as your denominators get large. For another, you need to keep computing gcfs to reduce your fractions. Still, this is the approach used in some calculator programs.

I wonder if a continued fraction representation would have advantages -- and if it has ever been used as a number representation in practice? It seems like it'd be very expensive, but I'd need to give it some thought...

Re:If you want accuracy...

[cgenman]

How is that Hardware Support going? Just curious.

Re:If you want accuracy...

[Nutria]

How is that Hardware Support going?

Very well, on machines designed for business (i.e., mainframes and VAXen).

Re:If you want accuracy...

[amorsen]

Instead you can put 20 decimal digits in 64bits (3.2 bits per db) and do math much more faster

I want accurate math, not estimates.

Math with 20 decimal digits in 64 bits is proper decimal arithmetic. It acts exactly like BCD does, it just doesn't waste tons of space and CPU power.

Re:If you want accuracy...

[poopdeville]

Continued fractions are a straightforward way to implement exact computable real arithmetic. So yes, it's been used. And it's slow. But it is exact.

Re:If you want accuracy...

[russotto]

Maybe because BCD is the worse possible way to do 'proper' decimal arithmetic, also it would absolutely be very slow.

So work base 250, or base 62500. Oh, wait, IBM has a patent on that.

Re:If you want accuracy...

[hedwards]

Eh, the point is that if you're only truncating it once at the end, you're probably fine. Very few applications really need that level of precision. However, the truncation or rounding that can happen in the middle is what you're really worried about.

Admittedly you don't always have the option of leaving the truncation to the end, such as doing that CS assignment about calculating trig functions out to a large number of digits.

Re:If you want accuracy...

[rtfa-troll]

How is BCD a benefit in this case?

Re:If you want accuracy...

[TheRaven64]

As I said in a post above, it's in recent IBM chips (and older IBM mainframes). The z/10 and POWER6 both have BCD FPUs. Now that there are IEEE standards for BCD floats, I wouldn't be surprised if it appears in a future SSE revision. There doesn't seem to be any BCD support in the C1X draft, but GCC has support for decimal types as an extension.

Re:If you want accuracy...

[rtfa-troll]

That is a very traditional IBM feature, also directly supported in COBOL for the good reason that it's the way money works.

Re:If you want accuracy...

[Nutria]

Peter Gibbons wouldn't have been able to (try to) skim off jillion rounding errors if the accounting system had been written in the appropriate domain specific language (which uses BCD).

Re:If you want accuracy...

[colinrichardday]

Exact computable real arithmetic? So what is pi+e? An exact irrational number requires infinitely many integers to represent it.

Re:If you want accuracy...

[JamesP]

You completely missed my point.

I'm not comparing BCD to floating point, I'm comparing BCD with other ways of encoding decimal numbers in a computer

Re:If you want accuracy...

[TheRaven64]

In the 1620, cited in the Wikipedia article that you mention, it wasn't just a way of doing arithmetic, it was the only way. It was a variable-word-length machine with 6-bit bytes, each storing one decimal digit or some flags, or a single character from a 64 character set. One special value indicated the end of a word.

It didn't, however, as you imply, have BCD hardware. In fact, it had no hardware at all for arithmetic. At the start of the core memory, you had two lookup tables, one for addition and one for multiplication. To add two values together, you iterated over each pair of (decimal) digits, looking up their values in the table and then storing the result somewhere. If one of the digits had the carry bit set, then you did the same thing with the result of adding the next two digits together to add the extra one.

The 1620 was nicknamed CADET by its designers for this reason - Can't Add, Doesn't Even Try. It was one of IBM's first attempts to make a cheap computer (you could buy a minimal unit for around $100,000 (paper tape instead of punch cards, only 10,000 characters of memory). It was the first computer my university ever bought, and the subject of an incredibly scathing review by Dijkstra.

Re:If you want accuracy...

[sjames]

"0.1 + 0.2 = 0.30000000000000004" doesn't really seem all that proper to me.

Especially when you quite reasonably object when a = 0.1+0.2; if a == 0.3 ... doesn't perform the then clause.

Re:If you want accuracy...

[sjames]

On Intel, not so good. On other machines, just great!

Re:If you want accuracy...

[Raffaello]

Peter Gibbons: Um... the 7-11. You take a penny from the tray, right?
Joanna: From the cripple children?
Peter Gibbons: No that's the jar. I'm talking about the tray. You know the pennies that are for everybody?
Joanna: Oh for everybody. Okay.
Peter Gibbons: Well those are whole pennies, right? I'm just talking about fractions of a penny here. But we do it from a much bigger tray and we do it a couple a million times.

Re:If you want accuracy...

[fbjon]

Actually, only 19 arbitrary digits, if you mean a 64-bit integer.

Re:If you want accuracy...

[Natural Join]

Exact computable real arithmetic? So what is pi+e? An exact irrational number requires infinitely many integers to represent it.

And yet you were able to represent this value with only 4 characters.

Read up on continued fractions and computer algebra systems and get back to us. There are more ways to do things than just those in the C library.

Re:If you want accuracy...

[colinrichardday]

And yet you were able to represent this value with only 4 characters.

Then I didn't need continued fractions or computer algebra, did I?

Re:If you want accuracy...

[petermgreen]

Decimal representations (BCD and/or a binary value with a power of ten scale factor*) make sense when you need to exactly replicate on a computer rules written by humans for humans. The main problem domain in which this is an issue is finance.

They also have the advantage that a greater proportion of low denominator fractions can be represented exactly.

However in general most scientific and engineering calculations won't come out nicely in either system so you will still have to round it's just a case of how much by. With both binary and decimal formats this ammount can be arbiterally small so no real advantage to either system for science/engineering/

I suspect the main reason though was complexity. Straight binary arithmetic is much simpler in logic terms than BCD arithmetic. Particularlly when multiplication becomes involved. Binary values with power of ten scale factors make multiplication and division (for fixed point) or renormalisation (for floating point) a lot trickier.

Re:If you want accuracy...

[JamesP]

No

2**64 = 18446744073709551616

That's 20 digits, unless you mean something else

Re:If you want accuracy...

[AuMatar]

If you want accuracy, BCD is still a failure. It only does base 10 instead of base 2. A truly accurate math system would use 2 integers, one for numerator and one for denominator and thus get all rational numbers. If you need irrationals you get even more complicated. But don't pretend BCD is accurate, it fails miserably on common math problems like 1/3.

Re:If you want accuracy...

[fbjon]

That's 19 arbitrary digits. The most significant digit there can only be 0 or 1.

Re:If you want accuracy...

[b4dc0d3r]

"paper" decimal arithmetic on the other hand, which is what gp was talking about, you'd truncate it somewhere and say "remainder 1"

Re:If you want accuracy...

[Anonymous Coward]

Because good, modern languages use fixed point numbers.

Re:If you want accuracy...

[Anonymous Coward]

Um, how about 1/3? You just use true rational numbers with a numerator and a denominator, or even a fixed point whose scale is 1/3.

Re:If you want accuracy...

[Anonymous Coward]

Do you also expect

a = 1.0 / 3
b = 1.0 / 9
c = 4.0 / 9
if(a + b == c) ...

to execute the 'then' clause?

Decimal arithmetic has its place. But it is incorrect and short-sighted to elevate it as more "exact" than any other system. I don't hear an outcry over decimal's inability to represent thirds. It is simply accepted that thirds are inexact. So why is it so hard to accept that 5ths can be inexact as well?

If anything, the Babylonians were on to something: base 60 can exactly represent fractions of 2, 3, 4, 5, 6, 10, 12, 15, 20, 30, and 60 with a single digit after the sexadecimal point. And backed 6 bits to a digit, it is very nearly as space-efficient as straight-up binary. So I wonder why this hasn't been implemented more frequently as a more exact arithmetic when dealing with "nice" fractions.

Re:If you want accuracy...

[Short Circuit]

I have an irrational need for an accurate mathematical calculation model. What, precisely, can you do for me?

Re:If you want accuracy...

[sjames]

Yes, as a matter of fact I do, but I would just express it as 1/3 +1/9 == 4/9. Let the rational math unit figure out that 1/3 + 1/9 == 3/9 + 1/9 == 4/9.

Perhaps I should say no, because I know that's not how it works now, but that I believe it SHOULD be perfectly fine.

I know decimal arithmetic can't solve it. To do that, we'll need to have rational arithmetic operations in hardware. Each register contains a numerator and a denominator and the rational arithmetic operators handle it appropriately (but perhaps only reduce when absolutely necessary or when explicitly coded).

That's how we used to do calculations and it worked quite well. When we went to using decimals, it mostly worked because we rarely managed to carry the error through enough operations for it to become significant compared to real world tolerances.

Now, we do numerical modeling where we might go through thousands or millions of iterations and the error can and does become significant.

Note, if you use e in a for loop, you deserve to lose, but for a rational number, there's not much excuse anymore.

Perhaps even that limitation shouldn't be, 1e + 5e == 6e is perfectly legitimate and absolutely true. I understand if the computer with its limitations doesn't handle that in hardware and so it might be fester if I just don't do that, but simple fractions are far more common and not that hard to do in hardware these days.

Re:If you want accuracy...

[TerranFury]

Given any positive 'epsilon,' I can compute a rational number, in finite time, which approximates 'pi + e' to within 'epsilon.' That's all that's needed for computability.

See computable number.

What was surprising to me is that not all reals are computable in this sense; an example is what the Specker sequence converges to.

This shouldn't have been surprising though, since it retrospect it's obvious that computer programs are countable whereas the reals aren't...

Re:If you want accuracy...

[Anonymous Coward]

You're right that BCD isn't any better than binary in any theoretical sense. Rational numbers, however, are not intrinsically better than going for integer arithmetic using two integers. See, rational numbers are demonstrably equivalent to integers. In fact, treating a pair of integers as part of a rational quotient has the disadvantage that the numbering system is not single valued, so you're wasting a lot of resources on either always trying to factor the two integers or simply not packing, bit for bit, as much information as you could by using a single integer with the same number of bits.

Re:If you want accuracy...

[JamesP]

You're right, thanks for clarifying :) I forgot about that...

Re:If you want accuracy...

[colinrichardday]

Which is why I doubted the OP's ability to do exact, real, computable math.

Re:If you want accuracy...

[Anonymous Coward]

Easy, it's 0.1 in base 3.

Re:If you want accuracy...

[poopdeville]

No, you're wrong, because you're confusing "exact" with something else.

http://en.wikipedia.org/wiki/Computable_number

Re:If you want accuracy...

[poopdeville]

And? What's your point? That you don't know what 'exact' means?

Re:If you want accuracy...

[poopdeville]

In other words, you are choosing the wrong representation of e and pi. Remember that "regular" real numbers are defined in terms of equivalence classes of Cauchy sequences. A Cauchy sequence is a function from the Naturals into a metric space for which there is "Cauchy convergence": for every epsilon, there is an N such that if m and n are large enough, |f(m) - f(n)| IS the number to which the sequence converges, since this is an ordered field. Real numbers are functions.

Now, given a function pi that converges to the number usually called pi, and a function e that converges to the number normally called e, what is pi + e? It's a function that converges to the number regularly called "pi + e". And yes, this function is definable.

Re:If you want accuracy...

[poopdeville]

Argh. That should read:

Cauchy sequence is a function from the Naturals into a metric space for which there is "Cauchy convergence": for every epsilon, there is an N such that if m and n are large enough, |f(m) - f(n)| < epsilon. f IS the number to which the sequence converges. f is not an approximation.

Re:If you want accuracy...

[Anonymous Coward]

Decimal arithmetic is NOT free from rounding error:

>>> Decimal(1) / 3 * 3
Decimal('0.9999999999999999999999999999')
>>> Decimal(2).sqrt() ** 2
Decimal('1.999999999999999999999999999')
>>> Decimal(1).exp().ln()
Decimal('0.9999999999999999999999999999')

a bit dup'y

[smoothnorman]

Somewhat a dup from seven years ago ... Posted by Cliff on Thursday June 26 2003, @04:42PM http://developers.slashdot.org/story/03/06/26/0044223/Floating-Point-Programming-Today ...ah well

Re:a bit dup'y

[Anonymous Coward]

Just because it's about a related subject doesn't mean it's a dupe. This article is specifically about the new web site, so it can't possibly be a dupe of an article from years before the web site was created.

Another potential solution is Interval arithmetic

[renoX]

Maybe in your list of solutions you could mention interval arithmetic, it's not very much used, but it gives "exact" solution.

What Every Programmer Should Know

[trust_jmh]

Computers are about cheating.
Do it right and you won't let anyone input enough to get caught out by the bug in your codes hidden methods.

#1 Rule, Don't use Java

[Anonymous Coward]

and the #1 floating point rule, if you need any sort of accuracy, or people's lives depend [heaven forbid] on your code, Don't use java, since it's not IEEE-754 compliant.

Might as well run on an FDIV Pentium, since the java philosophy is everything needs to be just "good enough".

Re:#1 Rule, Don't use Java

[abigor]

Actually, the linked article says exactly the opposite, and up above I posted a link to the JVM definition that verifies it. So you are 100% incorrect.

Re:#1 Rule, Don't use Java

[Da Fokka]

Don't you mean: 99.99999999904% incorrect?

Re:#1 Rule, Don't use Java

[T-Bone-T]

That's what he said, it just got rounded.

Re:#1 Rule, Don't use Java

[Stevecrox]

The real question is, if your coding safety critical applications why aren't you using the appropriate language e.g. Ada?

Java is a good programming language if you know how to use it and you can write some very efficient and small code. I'm tired of people attacking it for being slow when the people who use don't have a clue how to use it properly and just iterate their way through array lists (probably the most common mistake I see).

You should always use the right language for the right job.

Re:#1 Rule, Don't use Java

[owlstead]

What is it that lets Java have such a bad name on SlashDot? Even articles that are typical flamebait like the one above get modded up. It's open source, it's relatively secure, it's got loads of libraries and IDE's and developer support. It's even pretty fast compared to most competitors. It's definitely not a closed eco system either. Of course it's got its drawbacks, but come on...

Re:#1 Rule, Don't use Java

[RAMMS EIN]

``What is it that lets Java have such a bad name on SlashDot?''

I can only speak for myself, but the beef I have with Java is that it was promoted as if it was a great step ahead in programming languages and really the right way to do things. A lot of people fell for the propaganda (including myself) and an enormous amount of time has been spent on Java since - writing software with it, improving the platform, re-implementing things from other platforms for Java, basing research on Java, etc.

In reality, Java wasn't actually all that great. None of the features that were touted with much fanfare at the time were actually new. The language had a pretty lame type system and a lame implementation. If, at the time, I had known the languages I know now, I never would have thought Java was a great language. But I didn't know many languages back then, so I swallowed the propaganda and was really enthusiastic about Java. I can only assume the same has happened to many other people. At any rate, Java took off and went on to become a dominant language (not to say _the_ dominant language) in the software business and many universities.

To be sure, Java has changed a lot since its inception, and there definitely has been improvement. The type system is much better now than it used to be, and there are now several implementations for different niches, some of which are very impressive indeed. Still, after 15 years of enormous exposure, success, and development, I still find the language and platform lacking. Many features or missing features in the language make it difficult to express certain programs in their most natural form, often requiring much more code to write in Java than in another language. The standard library, while indeed extensive, is full of rough edges, bad APIs, and long-unfixed bugs. Many features of the host platform are simply not available in Java. Very few things in Java or the popular frameworks strike me as done right.

Now, what bothers me is not that there are features of Java I don't like. Even if they could be shown to objectively make Java less good than some other platform or programming language, I wouldn't care very much. My problem is that Java has these shortcomings, has long had such shortcomings, and is _still_ being touted by many as the greatest platform out there, still receives enormous amounts of research and development effort, and is still extremely widely used _and_ taught to students. To me, that's a lot of energy, time, and money that we, as humanity, could have more effectively spent on other things. The way I see it, Java has been a colossal waste of time, and I blame the vocal advocacy for that.

Now, what any of this has to do with IEEE 754, I don't know. As far as I am concerned, fully IEEE-754-compliant is better than just IEEE-754-like, but only marginally so - in either case, your results will be inexact and there will be many gotchas. Java does have BigInteger and BigDecimal types, which I usually find better choices than floats.

Re:#1 Rule, Don't use Java

[rtfa-troll]

Java has forced objectification whilst at the same time lacking proper introversion. Since we most people here prefer lisping, especially the Pailinites that's just not acceptable.

Re:#1 Rule, Don't use Java

[Tridus]

1. Some of it is just inevitable backlash. Many moons ago Java was hyped as the greatest thing ever that would magically solve all our problems. Of course it didn't, and anytime something gets overhyped, there is pushback.

2. Bad client experience. Java on the server is one thing, but client apps? Ugh. I've got four, and each one is picky about wanting its own version of the JRE. They're slow. They tend to look like Java apps rather then normal OS apps, so they don't really fit well and often don't follow conventions.

Some of that is just programmers, but I've never used a client side Java app that wasn't slow. These people who keep saying "Java isn't slow" need to point me to their magical fast JRE. Or maybe they mean it's not slow, except when doing UI stuff, because I've NEVER seen a UI that wasn't sluggish compared to a normal app.

3. It's not the shiny new thing anymore, and it's not buzzword compliant.

Re:#1 Rule, Don't use Java

[owlstead]

``What is it that lets Java have such a bad name on SlashDot?''

I can only speak for myself, but the beef I have with Java is that it was promoted as if it was a great step ahead in programming languages and really the right way to do things. A lot of people fell for the propaganda (including myself) and an enormous amount of time has been spent on Java since - writing software with it, improving the platform, re-implementing things from other platforms for Java, basing research on Java, etc.

I've been a follower of Java since version 1, and I don't see any difference with other languages. If anything, Java has always been a C/C++ kinda language but much simplified and, indeed, objectified. There are a lot of things that have been done with this re-implementation, and in many cases, rightfully so.

In reality, Java wasn't actually all that great. None of the features that were touted with much fanfare at the time were actually new. The language had a pretty lame type system and a lame implementation. If, at the time, I had known the languages I know now, I never would have thought Java was a great language. But I didn't know many languages back then, so I swallowed the propaganda and was really enthusiastic about Java. I can only assume the same has happened to many other people. At any rate, Java took off and went on to become a dominant language (not to say _the_ dominant language) in the software business and many universities.

Actually, it was and is pretty great. Of course, looking in retrospect, it did some things wrong. Many of those things like the type system were actually thought of as "inefficient" at that time. Now most language purist would like Java to have each and every variable to be an object instance. If they had taken that route back then, they would have been scorned by most of the community.

To be sure, Java has changed a lot since its inception, and there definitely has been improvement. The type system is much better now than it used to be, and there are now several implementations for different niches, some of which are very impressive indeed. Still, after 15 years of enormous exposure, success, and development, I still find the language and platform lacking. Many features or missing features in the language make it difficult to express certain programs in their most natural form, often requiring much more code to write in Java than in another language. The standard library, while indeed extensive, is full of rough edges, bad APIs, and long-unfixed bugs. Many features of the host platform are simply not available in Java. Very few things in Java or the popular frameworks strike me as done right.

Most of the basic objects are now immutable. There are good ways of handling threads with a brilliant java.util.concurrent lib and pretty well worked out collection framework (which is a lot easier and safer than the boost libs, imho). The date and time API still is a mess, but that is being fixed. Java has a good regexp lib, good well defined string handling and many more well defined libs. Of course, new API's (of new languages) will build on this and hopefully be better than Java. But take a look and see how many fail; well designing core classes is no easy task. It's just to big to give all to James and Joshua.

Now, what bothers me is not that there are features of Java I don't like. Even if they could be shown to objectively make Java less good than some other platform or programming language, I wouldn't care very much. My problem is that Java has these shortcomings, has long had such shortcomings, and is _still_ being touted by many as the greatest platform out there, still receives enormous amounts of research and development effort, and is still extremely widely used _and_ taught to students. To me, that's a lot of energy, time, and money that we, as humanity, could have more effectively spent on other things. The way I see it, Java has been a coloss

Re:#1 Rule, Don't use Java

[nitehawk214]

Easy, java developers are apparently easy to troll. Every time there is a discussion where someone can take a shot at java, someone does in order to get a rise out of people.

Re:#1 Rule, Don't use Java

[Nazlfrag]

I believe strictfp is the keyword you were looking for to get 100% IEEE compliant floating point in Java. Welcome to 12 years ago.

Re:#1 Rule, Don't use Java

[takev]

That is the biggest problem with Java.

1. To use Java well you need to be a good programmer.
2. A good programmer doesn't want to use Java.

The problems I've seen most programmers make is initializing and configuring objects (like full blown XML parsers, which takes over a ms to initialize and only a few us to use and reset) in the performance critical part of the program. This can be easily solved by creating object on demand, but putting them on a shared stack when you are done with it for reuse by an other thread.

Re:#1 Rule, Don't use Java

[RAMMS EIN]

Thanks for your thoughtful response.

To answer a few of the points and questions you raise:

``I've been a follower of Java since version 1, and I don't see any difference with other languages.''

That is sort of the point I was trying to make. Like every other language/platform, Java has its strengths and weaknesses. And that is to be expected, really.

I wrote: "In reality, Java wasn't actually all that great."

And then, you wrote: "Actually, it was and is pretty great.''

Right. I think I may not have expressed myself as clearly as I should have over there. What I meant to say is that Java wasn't (at the time) as great as the evangelists would have you believe. Like you said, you used Java since version 1, and don't see much of a difference with other languages. Java has its strengths and weaknesses. But that wasn't the word that was going around at the time. What I heard and read basically amounted to Java being the Messiah among programming languages, that it would deliver us from all the faults found in other programming languages, that we would write once and run anywhere, and so on. That's what I was referring to when I wrote Java wasn't actually _that_ great.

``Waste of time? Good gods, what other language would YOU have chosen 10 years back to develop major systems with? Yes, in retrospect you would not use basic types that overflow, single return values, have checked exceptions above unchecked ones, an underdeveloped module system, null-able references, mutable arrays ... the list goes on.''

Truth be told, I can't give you a good answer to that question, other than "I don't know." When Java was first released, the only languages I knew how to program in were a few BASIC dialects and x86 assembly. You asked about 10 years ago, so that would have been 2000. I also knew C and PHP, then. Compared to those, Java would have been the better choice for most application development work (though I preferred PHP for web development at the time). However, I note that, by 2000, such languages as Ruby and Objective Caml were available (which I prefer over Java now), and even in 1995, Ada, Common Lisp, Delphi, Dylan, Eiffel, Erlang, Haskell, Modula-3, Obective C, Python, Smalltalk, and Standard ML were already around. Many of those contain good ideas that a lot of people think were introduced by Java, and many of them include good ideas that I feel Java lacks.

The point I am trying to make here is that the folks behind Java certainly did a good job of getting their platform and programming language widely adopted, but in terms of making it the best platform and programming language out there, they could have been a lot better. And the thing that annoys me is not so much that Java didn't adopt all the great ideas out there (I know building a platform and programming language is hard and a lot of work), but that the world seems to have pretty much adopted Java to the exclusion of everything else, so that the great ideas that Java doesn't incorporate are still not mainstream 25 years later.

``I do try many "generic" languages out there (ECMA Script, C#, LUA, Go hell even Erlang, PHP and Perl and C++ to name a few).''

That is great, and I hope you keep doing it. And if you keep coming to the conclusion that Java is best for your needs, more power to you. You should definitely go and use it.

Re:#1 Rule, Don't use Java

[Anonymous Coward]

Show me ONE language standard that requires 754 compliance? C doesn't. C++ doesn't. Hell, even Ada doesn't.

What school are these programmers going to?

[TSRX]

What kind of terrible CS program doesn't teach floating-point arithmetic as part of a required course?

Re:What school are these programmers going to?

[-brazil-]

These days, not everyone who writes programs has some sort of formal CS education. And those who do may have forgotten about it by the time they run into this kind of problem.

Re:What school are these programmers going to?

[larry bagina]

At one previous job, of about 15 programmers, I think fewer than 5 had a CS degree. At another, of 3 programmers, not one had a CS degree. There are more copy-paste programmers than ever (especially with low barriers to entry -- php, python, javascript, vb/vba, etc).

Re:What school are these programmers going to?

[abigor]

The other day I had to explain ICMP to someone who was trying to ping a specific port on some server.

He actually did have a CS degree...from China.

Re:What school are these programmers going to?

[mdwh2]

I did a Maths degree, which I also need in my programming job. Does that make me a copy-paste programmer? Would a programmer who didn't have a maths degree do better?

Anyhow, perhaps the article is geared towards people who are learning it before they go to University, or people who are picking it up later in life. And hell, I suspect the original essay was intended for people who are currently studying a degree! No one's claiming this article is intended at people who are already professional programmers. I guess the OP thinks people doing a degree should only learn what they're spoonfed by the lecturers, rather than reading material on the topic in general.

Re:What school are these programmers going to?

[Anonymous Coward]

before they go to University

More superfluous capitalization! "University" isn't a proper noun, Mr. Follows Basic English. You mean 'go to university'.

Before we get

[toxygen01]

to floating point, please, everyone should've read Everything you ever wanted to know about C types and part 2 (which explains fp too).

this will save a lot of time & questions to most beginning (and maybe mediocre) programmers.

Re:Before we get

[noidentity]

Come on, why did you title your post "Before we get". I mean, how lazy can you be? How about "Before"? or "B"?

Re:Before we get

[Anonymous Coward]

Numerical Recipes (http://www.nr.com/) is pretty much the standard for this discussion. In terms of the "The Floating-Point Guide" site, its contents roughly summarize Chapter 1 of Numeical Recipes. It turns out, many people in computing forget about the lessons that distinguish accuracy from precision and the role of significant digits. Such infomation is usually found in a "hard science" course.

I'd just avoid it

[Chemisor]

Given the great complexity of dealing with floating point numbers properly, my first instinct, and my advice to anybody not already an expert on the subject, is to avoid them at all cost. Many algorithms can be redone in integers, similarly to Bresenham, and work without rounding errors at all. It's true that with SSE, floating point can sometimes be faster, but anyone who doesn't know what he's doing is vastly better off without it. At the very least, find a more experienced coworker and have him explain it to you before you shoot your foot off.

Re:I'd just avoid it

[-brazil-]

The non-trivial problems with floating-point really only turn up in the kind of calculations where *any* format would have the same or worse problems (most scientific computing simply *cannot* be done in integers, as they overflow too easily).

Floating-point is an excellent tool, you just have to know what it can and cannot do.

Re:I'd just avoid it

[smallfries]

And once you've finished writing your algorithm in manually coded fixed point to avoid the "complexities" of float-point you can sit down and tuck into a tasty plate of quiche.

Re:I'd just avoid it

[Rogerborg]

Depends how many "integers" you use. If you need accuracy - and "scientific" computing certainly does - then don't use floats. Decide how much accuracy you need, and implement that with as many bytes of data as it takes.

Floats are for games and 3D rendering, not "science".

Re:I'd just avoid it

[Anonymous Coward]

Lots of data lend itself to float representation. Think colors or sound samples, where the precision of the float is sufficient by a wide margin, but dealing with them in fixed point is very hard unless you know the exact numeric range in advance.

Re:I'd just avoid it

[-brazil-]

You've never done any scientific computing, it seems. While it's a very broad term, and floats certainly not the best tool for *all* computing done by science, anyone with even the most basic understanding knows that IEEE 754 floats *are* the best tool most of the time and exactly the result of deciding how much accuracy you need and implementing that with as many bytes of data as it takes. Hardly anything in the natural sciences needs more accuracy than a 64 bit float can provide.

Re:I'd just avoid it

[poopdeville]

And yet you had to write an article explaining that there are gaps between floats...

Re:I'd just avoid it

[-brazil-]

Do you really believe that what someone wondering why 0.1 + 0.2 != 0.3 needs is a format with more than 52 bits of precision?

Re:I'd just avoid it

[RAMMS EIN]

``The non-trivial problems with floating-point really only turn up in the kind of calculations where *any* format would have the same or worse problems (most scientific computing simply *cannot* be done in integers, as they overflow too easily).''

Fixed-size integers, that is. Yes. But fixed-precision ints and floats are not the only choices available. Many programming languages offer arbitrary-precision numeric types, and some even perform symbolic computation.

Many bugs in programs are the result of a mismatch between what a programmer meant to express and what the program actually expresses, and treating fixed-precision ints or floats as if they were actual, mathematical integers or reals / decimals / rationals is a very common example of exactly such a mismatch. It's fine in some cases, but it is not, strictly speaking, correct, and, sometimes, the incorrectness will come and bite you.

Re:I'd just avoid it

[Anonymous Coward]

You only need 48 bits for that. Split your numbers at the period and compute the =1.

0+0 = 0
1+2 = 3
= 0.3

Re:I'd just avoid it

[gnasher719]

Given the great complexity of dealing with floating point numbers properly, my first instinct, and my advice to anybody not already an expert on the subject, is to avoid them at all cost. Many algorithms can be redone in integers, similarly to Bresenham, and work without rounding errors at all. It's true that with SSE, floating point can sometimes be faster, but anyone who doesn't know what he's doing is vastly better off without it. At the very least, find a more experienced coworker and have him explain it to you before you shoot your foot off.

Anyone who is capable of changing an algorithm _correctly_ to use integer instead of floating point arithmetic would know how to make the floating point algorithm work correctly.

Re:I'd just avoid it

[petermgreen]

However arbitrary precision has it's issues too. In particular in a long sequence of calculations your memory use can balloon out of control. Also there are still many numbers that simply cannot be represented in an arbitrary precision decimal or even an arbitrary precision vulgar fraction.

Plus it's going to be extremely slow and memory hungry compared to using the built in floating point support in your CPU.

The only right answer is to understand your algorithm and it's characteristics so you can work out what precision is sufficient precision and whether that precision should be defined in absoloute terms or as a proportion of the value being handled.

Re:I'd just avoid it

[Rogerborg]

I'm pretty sure that one of us has never done any actual scientific computing.

Re:I'd just avoid it

[Anonymous Coward]

if you haven't ever done any scientific computing, what compelled your idiot ass to attempt a post about it to slashdot?

Re:I'd just avoid it

[MTocci]

I'm pretty sure that one of us has never done any actual scientific computing.

Wait, you're saying that floating point calculations aren't used in actual scientific computing? What are you defining as scientific computing? Does CFD count for you? I've never heard of any CFD code written using only fixed-point data.

float are over

[Anonymous Coward]

float are over, we should go back to fixed point. With 64bits interger we already have more precision then 32bit float. Let use that 64bit adventage to drop the float madness and leave the 64bit float for a few niche applications.

Re:float are over

[sco08y]

Really, the best answer is to store all numbers on the cloud, and just use a 256-bit GUID to look them up when needed.

Re:float are over

[Yvan256]

Whatever floats your boat.

Re:float are over

[Dylan16807]

32bit integers have more precision than 32bit floats, too. You still can't represent 1/3.

Re:float are over

[Frenchman113]

Sure you can. Integers can mean whatever you want them to. 1 = 1/3. 2 = 2/3. 3 = 3/3.

Re:float are over

[ClickOnThis]

float are over, we should go back to fixed point. With 64bits interger we already have more precision then 32bit float. Let use that 64bit adventage to drop the float madness and leave the 64bit float for a few niche applications.

32bit integers have more precision than 32bit floats, too. You still can't represent 1/3.

Sure you can. Integers can mean whatever you want them to. 1 = 1/3. 2 = 2/3. 3 = 3/3.

Even a 64-bit unsigned integer (roughly 0 to 1.8E+19) does not have the dynamic range of a 32-bit float (roughly 1E-30 to 1E+30.)

Three one-thirds is 1, riiiiight?

[Anonymous Coward]

I'm completely baffled by this (in Python):
>>> print (1/3)*3
0

I expected 1, and my FPU is from AMD, not Intel ;)

Re:Three one-thirds is 1, riiiiight?

[jjohnson]

>>> (1.0/3)*3
1.0

Re:Three one-thirds is 1, riiiiight?

[larry bagina]

it's using integer division.

>>> print (1.0 / 3.0) * 3.0
1.0

Re:Three one-thirds is 1, riiiiight?

[TheRaven64]

I try to avoid Python, but this result makes sense if you are doing integer division. 1/3 is 0 remainder 2. The result of the integer division will be 0, which remains 0 when multiplied by 3. You'd get the same result in C, but if you do (1.0/3)*3 then you should get something approximately equal to 1.

Re:Three one-thirds is 1, riiiiight?

[MobileTatsu-NJG]

I'm completely baffled by this (in Python):
>>> print (1/3)*3
0

I expected 1, and my FPU is from AMD, not Intel ;)

You didn't use the FPU. Try this: print (1.0 / 3) * 3;

Re:Three one-thirds is 1, riiiiight?

[Anonymous Coward]

In Perl, this outputs "0.333333333333333" either way. But it gets "0.1 + 0.3" right.

Re:Three one-thirds is 1, riiiiight?

[grumbel]

That's just integer division, nothing to do with FPU. In Python3 they actually changed this behaviour, so that now floating point is used:

>>> print((1/3)*3)
1.0

A thing that I found a little more surprising in Python is:

>>> -5 / 2
-3

When you come from C you'd expect that to be -2.

Article title rings a bell

[sco08y]

Check Goldberg's article, 'What Every Computer Scientist Should Know About Floating Point' for a more in depth discussion.

An intuitive description of floating-point

[noidentity]

Here's a one-page intuitive description of floating-point, to give a feel for why multiplication and division are fairly benign, but addition and subtraction aren't. None of the other descriptions have made it as clear as this.

Re:An intuitive description of floating-point

[Anonymous Coward]

I wouldn't say fairly benign. I'd say less likely to have as big problems e.g. adding a bunch of very small numbers to a large number can cause a bigger error compared to multiplying those bunch of numbers, but there could still be an increasing error with the latter.

Re:An intuitive description of floating-point

[noidentity]

As described in the linked text, when you multiply the relative error stays roughly the same. Obviously some error does occur, and it's cumulative in many cases, but this is the case for any imprecise operation. I still think the error is fairly benign.

No, base 10 arithmetic isn't "more accurate".

[Animats]

The article gives the impression that base 10 arithmetic is somehow "more accurate". It's not. You still get errors for, say, 1/3 + 1/3 + 1/3. It's just that the errors are different.

Rational arithmetic, where you carry along a numerator and denominator, is accurate for addition, subtraction, multiplication, and division. But the numerator and denominator tend to get very large, even if you use GCD to remove common factors from both.

It's worth noting that, while IEEE floating point has an 80-bit format, PowerPCs, IBM mainframes, Cell processors, and VAXen do not. All machines compliant with the IEEE floating point standard should get the same answers. The others won't. This is a big enough issue that, when the Macintosh went from Motorola 68xxx CPUs to PowerPC CPUs, most of the engineering applications were not converted. Getting a different answer from the old version was unacceptable.

Re:No, base 10 arithmetic isn't "more accurate".

[-brazil-]

Actually, I tried to make it very clear in several places that base 10 has just the same problems. I am open to any suggestions for improvement, though.

Re:No, base 10 arithmetic isn't "more accurate".

[MobileTatsu-NJG]

The article gives the impression that base 10 arithmetic is somehow "more accurate". It's not. You still get errors for, say, 1/3 + 1/3 + 1/3. It's just that the errors are different.

What kind of errors are you referring to?

Re:No, base 10 arithmetic isn't "more accurate".

[Anonymous Coward]

The article gives the impression that base 10 arithmetic is somehow "more accurate". It's not.

Anything that works out exactly in base 2 also works out exactly in base 10, but not vice versa. I'd call that "more accurate", but YMMV.

Re:No, base 10 arithmetic isn't "more accurate".

[Anonymous Coward]

I think it was clear. The same 1/3 example is one of the first things you mentioned.

Stop with the educational articles

[sunderland56]

Look, times are tough for programmers already. Knowing how to do things correctly - like proper floating point math - is one of the ways to separate the true CS professional from the wannabe new graduates. Articles like this just make everyone smarter, and make finding a job that much harder.

Re:Stop with the educational articles

[sohp]

Knowing how to do things correctly - like proper floating point math - is one of the ways to separate the true CS professional from the wannabe new graduates.

True, except that HR people and hiring managers neither know nor care about doing things correctly, they just want cheap and fast. Just make sure you have all the right TLAs on your resume, you'll get a job. You can put "IEEE 754 expert" down though. They won't recognized the reference so maybe they'll be impressed by it.

Re:Stop with the educational articles

[jellomizer]

Except for the fact that companies don't care about floating point they are looking for 3+ years on windows 7. 20 years of Linux. and 15 years of .NET.

Re:Stop with the educational articles

[DiegoBravo]

A long time ago I was in a meeting about the C programming language, until some guy asked about the fixed point math support OBVIOUSLY required for financial operations, which happened to work perfectly and natively in COBOL (he was a COBOL programmer.) Yet I'm looking for equivalent libraries in C (and now in Java.)

Re:Stop with the educational articles

[dcollins]

"Look, times are tough for programmers already. Knowing how to do things correctly - like proper floating point math - is one of the ways to separate the true CS professional from the wannabe new graduates. Articles like this just make everyone smarter, and make finding a job that much harder."

Except that the people doing the hiring also need to know the correct answer to this kind of stuff.

I'm pretty much convinced that I lost a job once when, on a written programming test to convert strings to numbers, I wrote down the standard algorithm from my numerical analysis class, and the reviewer (not understanding what it was doing) told me it was working backwards. I did an example code walk-through with him, and yet he was still skeptical.

Re:Stop with the educational articles

[petermgreen]

The trouble with both C and Java is they don't let you define your own types that are on par with the native types so even if you write or find such a library it will be a pain to use.

C++ OTOH gives you the tools needed to build your own types that can be used like the built in ones and a quick google search finds a couple of examples of this.

Re:Stop with the educational articles

[DiegoBravo]

> The trouble with both C and Java is they don't let you define your own types that are on par with the native types so even if you write or find such a library it will be a pain to use.

Exactly; what I don't understand is why the Java people doesn't consider this an important issue, given the "enterprise flavor" they try to inject into.

For C++, yes, I'm sure it could be done with operator overloading, but (after years of struggling) I have to admit this a too convoluted language for my brain.

Re:Stop with the educational articles

[sohp]

The trouble with both C and Java is they don't let you define your own types that are on par with the native types so even if you write or find such a library it will be a pain to use.

We could go back to using LISP...

Re:Stop with the educational articles

[sohp]

COBOL got most things wrong. The one exception seems to be the fixed point math used for financial calculations. It often comes as a surprise to COBOL programmers learning a new language that there's not built-in support for the same kinds of operations. They're also likely to be tripped up by trying to use double/float and expecting exact results.

xkcd

[alex-tokar]

They would never have gotten to Slashdot front page if they did not include that xkcd reference.

Not sure it belongs in an intro explanation, but

[dr2chase]

Other issues that might be worth mentioning:

• Catastrophic cancellation in complex arithmetic.
• Single vs double rounding, in fused vs cascaded multiply-add operations.
• Range reduction in trig functions (Intel hardware only uses a 68-bit PI, this causes problems sometimes).
• Double-rounding when converting FP precision (e.g., 64-bit mantissa to 53, or 53 with extended exponent to 53 with regular exponent when the mantissa goes denorm).
• Conversion to/from string representation.
• Issues with "constructive reals" (a=b? is not necessarily an answerable question -- you might need to look at "all" the digits in order to answer "yes").
• Distributive law DOES NOT HOLD -- a * (b+c) != a*b + a*c

Re:Not sure it belongs in an intro explanation, bu

[Anonymous Coward]

PLEASE stop this thread - interval arithmetic and all that has been around as a research topic for decades now.

If you did not know: there are search engines providing lots of links...

Re:Another potential solution is Interval arithmet

[-brazil-]

Why don't you write it up yourself and give me a github pull request? :)

Cool down, please

[Anonymous Coward]

You need a cock in your ass today.

Re:Cool down, please

[bar-agent]

You need a cock in your ass today.

I respectfully suggest that that would only worsen his mood...

Consistency across platforms

[slart42]

Another thing I think an article labelled "What every Programmer should know..." should mention is that, while "Nearly all hardware and programming languages use floating-point numbers in the same binary formats", implementations can actually be very inconsistent, in the results of computations.

For example, mac and windows programs will have different rounding behaviors, and come up with different computation results in floating point math (at least with the default compiler settings - does anyone know if it is possible to change this?). This is constantly causing us trouble, when trying to write Unit tests only to realize the expected output values are slightly different on different machines to run the tests on.

I also think that, as somebody posted above, the non-associativity of floating point operations is important and should be mentioned. But, otherwise, a well written and helpful article!

Re:Consistency across platforms

[Anonymous Coward]

Uh, floating point results can even change due to using a different compiler.
It's clearly said that you can't just do a == on floating point numbers, why would you think this does not apply to your unit tests just as much?
(Not to mention that your unit tests are obviously not testing what you want, because if they did a failing test means: you _can't_ use floating point for your problem and you have to rewrite your code to not use it).
Only way to get reliable results with floating point is assembler code (and making sure to set the floating point state in that assembler code, if applicable to your architecture).

Re:Consistency across platforms

[slart42]

It's clearly said that you can't just do a == on floating point numbers, why would you think this does not apply to your unit tests just as much?

I never said so. However, rounding errors may add up, and an epsilon test which passes on one platform may fail on another. Or operations which use floating points and convert results to integers may yield different results. One case where we ran into this where the DXT texture compression libraries from ATI, which use some floating point math internally, resulting in (slightly) different output textures on mac and windows.

(Not to mention that your unit tests are obviously not testing what you want, because if they did a failing test means: you _can't_ use floating point for your problem and you have to rewrite your code to not use it).

No, it just means that the tests needed to accept a wider range of valid results.

Re:Consistency across platforms

[TheRaven64]

For example, mac and windows programs will have different rounding behaviors, and come up with different computation results in floating point math (at least with the default compiler settings - does anyone know if it is possible to change this?).

By default, Windows compilers will target all platforms that Windows can run on and Mac compilers will target everything that OS X will run on. In x86 terms, this means that Windows compilers target i386, while Mac compilers target i686 + SSE3. The i386 target means using the x87 for floating point operations, while the Mac compiler will generate SSE instructions instead.

The x87 has eight 80-bit registers, arranged in a stack (which makes generating code for it difficult, but this is a different issue). This means that any value, float, double, or long double, will be 80 bits while stored in a register, but will be truncated when you spill it to the stack or store it in a variable in memory.

If you do a sequence of operations that fit in these 8 registers with a Windows compiler targeting the x87, you will get the whole sequence performed at 80-bit precision. If it's a float, then the result will then be truncated to 32 bits.

Compiling the same code on a Mac will give you a sequence of SSE operations. If you're using a float, then the entire sequence will be done as 32-bit operations. This will be much faster, but less accurate (although it will be as accurate as the spec requires).

You can make the Mac compiler behave like the Windows one by telling it to target the x87, but that will make things much slower. You can make the Windows compiler behave like the Mac one by telling it to use SSE for floating point arithmetic, but that will make it require a P3 or newer.

Even that doesn't necessarily guarantee the same results, because the two compilers may have different optimisations and so generate different sequences of operations from the same source code.

Re:Consistency across platforms

[Anonymous Coward]

If you can accept a wider range then why are your tests testing for a tighter one? Or is that a case of first writing the code and then adapting the tests and specifications to match the code? (Yes, yes, I know, in reality sometimes you just can't do things properly, but it sounds to me like you wrote floating-point unit tests without having the slightest clue what your error limits actually are - testing without actually knowing what is a "good" and what is a "bad" case beforehand is at least questionable).
Or in other words: if that epsilon test failed you either failed to set the proper epsilon in the test, or you had a real bug in the code that it was not sufficiently numerically stable on all platforms.

Re:Consistency across platforms

[arth1]

I've seen too many unit tests that were written to validate the code written, not to find faults in it. Including automated creation of "unit tests" which takes the output from a single "declared successful" run and uses it to write out the source code for unit tests.
This is, needless to say, brain dead, and misses the entire point of unit testing.

For a unit test to be worth anything, it should be written by someone who hasn't seen the code. And whoever writes the code shouldn't have access to the unit test source code either. Cause it's not a tool for validating the code, it's a tool for finding faults. And quite often, the fault will be in the unit test, which isn't unit tested...

Every software development job interview...

[fragMasterFlash]

...should include a sample of some code that attempts to perform logical operations based on the outcome of floating point operations. Anyone who attempts to check the result of a floating point operation for equality without normalizing precision should be shown to the door and never called back.

CS stuff

[mseeger]

This sounds a lot like the stuff i heard in CS/Mathematics classes more than 20 years ago (Hackbusch, Praktische Analysis, Summer term 1987/88). That course was mandatory then for any CS, Mathematics and Physics student. Has that changed yet? It's about the differences between a number at it's binary representation (and examples about consequences).

I completely agree, that every programmer should know about that. But this is nothing new, it was already important 40 years ago. I'm pretty sure some space probe/missile already got lost due to this problems.

CU, Martin

P.S. It's not only important for programmers (dealing with C, C++, Java, etc.). Several times i was called to debug some "Excel Error" when it turned out to be a floating point restriction.... It would be interesting if you can create a macro that checks for such conditions.

Scientist Programmers

[Anonymous Coward]

I used to do algorithm ports for science grade code to operational grade for satellite data processing. The hardest part of the job was trying to convince the phds that considered themselves 'expert programmers' that we hadn't screwed something up in the port when the answers didn't exactly match. I'm not sure if its because they didn't think we had anything of value to contribute to the process, or if they were freaked out by the fact that all of the accuracy they had been touting was only perceived instead of actual. To their credit it is pretty difficult to convince someone that everything is working just fine when the algorithm takes a single sensor value representing an 800 sq m swath of land and spits out 'desert' on their system and 'tundra' on ours.

Re:Another potential solution is Interval arithmet

[renoX]

Because I don't know how to use github and it looks to me as a really, really complicated way to make a wiki..

Comparison code seems wrong

[Anonymous Coward]

On your page about comparison, you give the following code:

function nearlyEqual(a,b)
        {
                return a==b || Math.abs((a-b)/b) 0.00001;
        }

if(nearlyEqual(a,b))

I think it will fail with a division by zero if e.g. a==2.0 and b==0.0.

Re:Comparison code seems wrong

[-brazil-]

Actually, it will yield the correct result because (nonzero)/0 = Infinity. But it will return the wrong result for a smaller than epsilon. Will correct that.

What almost every programmer should know about FP:

[Ant P.]

Nothing at all.

Someone else has already written a library for the hard part of what you're doing, and their code *works*. Use it.

Re:Another potential solution is Interval arithmet

[-brazil-]

Well, I'll do it when I get around to it. Doing it as a wiki would mean that I'd have to deal with vandalism and spam, and it's really intended more as small self-contained site than a community thing.

Joel Spolsky's Unicode explanation was a joke...

[Anonymous Coward]

I remember when I first read it: it was all wrong, utterly confusing several notions. I flamed that donkey to death and he updated/changed his Unicode explanation. It was really a "ah ah, I just understood why Unicode is good, let's blog about it"-type of article, full of errors.

That guy is not a good programmer at all. Somehow commented here on /. a while ago that Joel Spolsky was approximately saying something interesting as often as sunlight shines in one's arse. Honestly the only thing that Spolsky said that I found interesting was when he said that he created stackoverflow.com because he was tired of "looking for good speakers" and finding endless "meta" discussion about the speakers-forum ("why is my comment in bold?") instead of finding discussion about speakers itself.

stackoverflow.com kinda works (it's a hive full of retards but it kinda works once you ask "advanced", like help for some elisp piece of code) and for that kudos to M. Spolsky...

But his articles are typically "ah ah moments" full of errors. He's not a good programmer and should not be quoted nor used as a reference.

Please look here

[ctrl-alt-canc]

People interested into floating point math will find some very interesting materials and horror stories in the documents collected at the home page of professor William Kahan, the man behind IEEE754 standard.
According to my personal experience the paper by David Goldberg cited in the post isn't that difficult after all. Plenty of interesting materials can also be found in the Oppenheim & Shafer textbook about digital signal processing.

Hard to debug floating point when it goes wrong!

[Cliff Stoll]

Over at Evans Hall at UC/Berkeley, stroll down the 8th floor hallway. On the wall, you'll find an envelope filled with flyers titled, "Why is Floating-Point Computation so Hard to Debug whe it Goes Wrong?"

It's Prof. Kahan's challenge to the passerby - figure out what's wrong with a trivial program. His program is just 8 lines long, has no adds, subtracts, or divisions. There's no cancellation or giant intermediate results.

But Kahan's malignant code computes the absolute value of a number incorrectly on almost every computer with less than 39 significant digits.

Between seminars, I picked up a copy, and had a fascinating time working through his example. (Hint: Watch for radioactive roundoff errors near singularities!)

Moral: When things go wrong with floating point computation, it's surprisingly difficult to figure out what happened. And assigning error-bars and roundoff estimates is really challenging!

Try it yourself at:
http://www.cs.berkeley.edu/~wkahan/WrongR.pdf

Re:Another potential solution is Interval arithmet

[Anonymous Coward]

Nice attitude. "Hey, I think this would be a good idea". "Shut up and do it yourself instead". Typical response from an open source developer; aggressive, smug and defensive.

Thank goodness for IEEE 754

[FranTaylor]

Most of you are too young to have dealt with architectures like VAX and Gould PowerNode with their awful floating point implementations.

IEEE 754 has problems but it's a big improvement over what came before.

Not equivalent to Spolsky's article.

[John Hasler]

It's missing the irritating cutesy "humor".

Re:Another potential solution is Interval arithmet

[-brazil-]

I'm very sorry for whatever horrible things happened to you that makes you read that into my words. They were meant as an acceptance of the suggestion that a section about interval arithmetic would be a good addition to the site and an invitation to contribute.

Python

[dln385]

Python has a human-centric method of working with floating-point numbers: the Decimal class.

Don't forget...

[xded]

... to use proper test cases:

http://www.xkcd.com/217/

Re:Please look here (my horror story)

[sohp]

I was brought in a bit after the start of a state project to write a system to track about a half billion dollars in money for elderly and disabled indigent care. I was horrified to find that all the money variables were float. After raising the issue and explaining the technical details, I proposed a Money class and if they didn't want that gave them the option of a simple fix: just change all the floats to long and keep the amounts in pennies, inserting the decimal point only when displaying numbers. The tech lead (nice guy, not the sharpest crayon in the box) and the DB architect (who was the sort for which Codd was God and for which the DB2 NUMERIC datatype was all we needed) determined that they could find no discrepancies, so the problems I cited must not apply to their code.

Fast forward several months later, when the numbers started to not add up, just as I explained. Killed a significant amount of time retrofitting the code to use BigDecimal. Even then, there were problems. Not the only reason the project was shut down before tackling the remaining planned features, but certainly a factor in it.

ACM Digital Library link

[_Shad0w_]

If anyone with ACM digital libray access wants the DOI link to the original article, rather than the edited version Sun/Oracle's site it's http://doi.acm.org/10.1145/103162.103163.

It is an old article though, so it's a 44 page scanned PDF.

Re:Hard to debug floating point when it goes wrong

[daveb1]

sound interesting - are there any formal automated means of testing / checking tho? (like a valgrind for numbers?).

decimal != arbitrary precision

[Anonymous Coward]

From the guide: "If you really need your results to add up exactly, especially when you work with money: use a special decimal datatype."

No!

Use a decimal representation when you need representable numbers to match with the operator's/programmer's idea of what should be representable exactly (e.g. financial data).

Use arbitrary precision (which can be implemented with any base, but 2^32 is popular) when you need results to add up exactly.

Re:Hard to debug floating point when it goes wrong

[Lord Efnar]

That is neat, but some (math oriented) languages do just fine:
Some Mathematica code:

h[x_] := Module[{y, w, k},
y = Abs[x];
For[k = 1, k <= 128, k++, y = Sqrt[y]];
w = y;
For[k = 1, k <= 128, k++, w = w^2];
w
]
Plot[Evaluate[h[x]], {x, 0, 2}]

The result: http://www.untruth.org/~josh/real-rounding.png

Re:Not sure it belongs in an intro explanation, bu

[xded]

• Conversion to/from string representation.

Which, to most people, looks like the best way to convert a number from single to double precision in Java.

For this reason, you may also want to reference the forgotten class sun.misc.FloatingDecimal. And I still wonder why assigning floats to doubles in Java leads to a cast with no warning (which is just wrong), while they have the necessary conversion facilities available.

Example:

double myDoubleFromFloat = new FloatingDecimal(myFloat).doubleValue();

If you check the conversion code, the 3000+ lines in that class are totally not trivial.

Oops!

[Lord Efnar]

Well, so much for my smug mathy reply! Amusingly, the reason it worked so well is that my "Evaluate" statement asked Mathematica to symbolically evaluate the function prior to graphing. So, the graph looked nice because Mathematica was just graphing the "Abs[x]" function!
Without the symbolic evaluation or requesting a particular precision level, the graph you actually get is this:
http://www.untruth.org/~josh/real-rounding-oops1.png
You can get a more reasonable looking answer by messing with the calculation precision and accuracy...

Only in a perfect world, what about MS Access?

[Anonymous Coward]

And I've seen the addition of two money columns defined in Access get magical values. I'm sure somebody here can explain the situation here better than I can, but I've seen $1.00+.50 become $1.49. But in MS Access's defense, a float is a poor way to define money especially in MS Access. I was just hired to bandaid the broken solution.

fractions

[sjames]

I would REALLY like to see an "FPU" that keeps everything as fractions until explicitly forced to reduce to a floating point (if ever). Maintain a numerator and denominator register. Only reduce when the other choice is an overflow or where a common denominator operation is forced (addition, subtraction, or an explicit instruction) at least as an option!

It's truly stunning how often very detailed and compute intensive modeling where human lives depend on the correct answers will go astray because of brittle floating point. The very best case for that is when the errors compound quickly and cause obviously wrong answers, but I wonder how often the not so obviously wrong answer is accepted as correct. It's common enough in modeling to fiddle slightly with the start conditions to get the model over a rough patch and produce an answer. Perhaps go with 1.00001 second intervals rather than 1 second exactly. Perhaps bump something a micron to the left and so avoid a divide by zero on the 20,000th iteration. Or worst, compile with -O5 rather than -O6. Most of the time it works out OK, but it would be a lot more comforting if all that fiddling wasn't needed. You can only flip off the daemons of chaos so many times before you get squashed.

Re:Hard to debug floating point when it goes wrong

[zippthorne]

No adds or subtracts, but 128 invocations of a mysterious square root function that doesn't involve adds, subtracts, or divisions.

Therefore the square root function is the problem, and the reason is that it is implemented as a ghastly huge lookup table. This program never gets loaded into memory, let alone has a chance to actually run.

The "singularity theory" is just a red herring.

Re:Another potential solution is Interval arithmet

[OSPolicy]

Internal arithmetic always includes the exact solution, but only the rarest circumstances does it actually give the exact solution. For example, an acceptable interval answer for 1/3 would be [0.33,0.34]. That interval includes the exact answer, but does not express it.

Re:Hard to debug floating point when it goes wrong

[dcollins]

FTA:

"Only 257 algebraic operations, so no hordes of rounding errors."

Say what?

Thanks to Sun

[khb]

Note that the cited paper location is docs.sun.com; this version of the article has corrections and improvements from the original ACM paper. Sun has provided this to interested parties for 20odd years (I have no idea what they paid ACM for rights to distribute).

http://www.netlib.org/fdlibm/ is the Sun provided freely distributable libm that follows (in a roundabout way) from the paper.

I don't recall if K.C. Ng's terrific "infinite pi" code is included (it was in Sun's libm) which takes care of intel hw by doing the range reduction with enough bits for the particular argument to be nearly equivalent to infinite arithmetic.

Sun's floating point group did much to advance the state of the art in deployed and deployable computer arithmetic.

Kudos to the group (one hopes that Oracle will treat them with the respect they deserve)

I don't understand.

[Anonymous Coward]

Can someone use a car analogy or even a Soviet Russia one?

Re:Another potential solution is Interval arithmet

[Anonymous Coward]

Interval Arithmetic does *not* produce exact solutions, it provides upper and lower error bounds on the calculation. Most IA implementations introduce a bias on every bounds calculation to insure that the "exact" result is proven to be bounded by the interval. I think some SPARC processors had IA support.

Re:Not sure it belongs in an intro explanation, bu

[tsvk]

Nice info, but as I have understood the sun.* package hierarchy is not part of standard Java and thus the FloatingDecimal class might not necessarily exist on a non-Sun JVM.

What do you mean by "novice"?

[NotSoHeavyD3]

Because I remember reading some programmer complain about this on bug reporting web site like 10 years ago.(I think it was bugwatch.com) The guy said he had 20 years of experience and ran into it and thought it was a bug in Java. Anyway to be serious this didn't really catch me because I've taken chemistry 101. Oddly enough part of that courses was significant figures. Once you're used to sig figs you kind of get why it's not a big deal that the values seem a bit "off".

The bible of floating point

[Anonymous Coward]

All be it predates IEEE 754 there is Semi Numerical Algorithms by Knuth. It is in the nature of everything you always wanted to know about floating point but were afraid to ask.The volume is very mathematical.

Actually the fundamental problem is very simple, there are an infinite number of numbers on the real number line (between any two numbers on the real number line there is another number...) However computers are
machines with a finite word length so that the basic theory of real numbers is not valid on the computer, because it is not necessarily the case that between any two floating point numbers on the computer
there is another number. Thus the computer can not represent all numbers which is the source of the problem. The approximation required each time a floating point number is manipulated case this "error' to propagate.
Of course this problem existed even when doing calculations with paper and pencil, since one does not do an infinite length calculation, but when working by hand the number of calculations is so much less that
the problem does not become evident.

Spolsky's incompetence

[Anonymous Coward]

Typical Spolsky: speaking with authority and getting everything wrong.

For the latest version of CityDesk, the web site management software published by my company, we decided to do everything internally in UCS-2 (two byte) Unicode, ... In C++ code we just declare strings as wchar_t ("wide char") instead of char and use the wcs functions instead of the str functions (for example wcscat and wcslen instead of strcat and strlen). To create a literal UCS-2 string in C code you just put an L before it as so: L"Hello".

UCS-2 is an obsolete character set; Windows actually uses UTF-16. And wchar_t is not a Unicode character.

Re:Joel Spolsky's Unicode explanation was a joke..

[Anonymous Coward]

Modders, this might look like a troll in shape but it is very insightful.
Even now, that article isn't very good at all. It isn't even passable. Go somewhere else for a good introduction to Unicode and why it is important.
This isn't an article but has far better information if somehow hurdled by the author's need to give us his opinions on the matter.

Amazing how few programmers use real maths.

[physburn]

I bet many of you have never done any real floating point work. I cut my teeth learning how floating point was implemented from the ZX spectrum ROM disassembly, used so many numerical recipies code for my Physics PhD, entered the world of programming for a living, and sadly never had touch maths commercially again. Most computer job I see advertised, seem to think that A-level maths is as hard as it gets. Enough moaning.

As a programmer, the one most important thing you need to know about floating point is never test for equality (even with zero), almost define how near the answer need be, and test against that. double a,b; double epsilon = 1.0d-10;

if (a==b){ // Bad practice

if ( abs(a-b) .lt. epsilon){ // Better practice

--- Mathematical Programming Feed @ Feed Distiller

Re:Amazing how few programmers use real maths.

[grumbel]

The C++ FAQ can be quite an eye opener in that regard:

Why is cos(x) != cos(y) even though x == y?

The only time when using == with floats should be ok, is when comparing it to zero and only then if the zero comes from a direct assignment, instead of a calculation, i.e.:

float x = 0.0f;
if (x == 0.0f) { /* do stuff */ }

But even then its bad practice, as its abusing 0.0f as magic number, which should be avoided if possible.

Re:Amazing how few programmers use real maths.

[spitzak]

It is safe to compare to any small integer, not just zero, as long as you are checking if the the value came from an assignment. It is also safe to use small negative powers of two.

One big problem I have is with programmers who religiously add these epsilon functions and screw up algorithms. In my experience, about 99% of the == statements with floating point are explicitly testing "did the assignment to the same value earlier get executed?" Comparing the bit patterns is exactly what is wanted, stop messing it up!

Re:Amazing how few programmers use real maths.

[gnasher719]

As a programmer, the one most important thing you need to know about floating point is never test for equality (even with zero), almost define how near the answer need be, and test against that. double a,b; double epsilon = 1.0d-10;

That is nonsense. The important thing is that you know what you are doing. As an example, if you have four integers a, b, c, and d, and you want to know whether the mathematical fractions a/b and c/d have the same values, what would you think about checking (double) a / (double) b == (double) c / (double) d ? Would involving any epsilon help you?

Re:Amazing how few programmers use real maths.

[Anonymous Coward]

Except that the different between each successive representative value in floating point numbers, increase exponentially as the values increase. Thus, this "test using an epsilon" method of yours increase in error exponentially. This method is BAD patience and has been debunked many, many times, but idiots still keep regurgitating it.

Re:Amazing how few programmers use real maths.

[Anonymous Coward]

Since float is ALWAYS promoted to a double before being used in an expression, adding "f" to the end of "0.0" is worthless in this case, and would make me question the competency of any C or C++ code you wrote.

red herring

[eyeb1]

how about the FACT .. that no matter how many have been taught to use it .. or how hard you want to believe in it .. decimal math is a LIE .. it is one of if not the main reason why .. there are so many MIS-CON-ceptions in our current UNDERSTANDING about life and the universe .. and the reason for the required need of approximation when using it ..

what we call life and the universe are fractional .. fractal and absolute .. and it does not matter to how may digits you can calculate pi .. it is an exact absolute fractional relationship .. there is NO decimal equivalent .. and can be nothing but an approximation ..

as for the concept of rational and irrational numbers .. again a MIS-CON-ception .. it is not that you can not divide by zero .. it is just you do not understand the the meaning and significance of the answer .. and so most persist in calling them irrational .. interesting but expect-able that these were the terms used to describe these kinds of numbers .. rational and irrational ..but of course we were naive enough to to BASE our computing system on binary computation(great for drawing straight line but nothing else) .. so i guess it was to be expected .. until we look at both 10 base and 12 base number systems and their interrelationships .. we will persist in our MISS-understandings about life in the BIG picture ..

and just to add insult to the injury .. particle physics is also a LIE .. there is NO part of MATTER smaller that the hydrogen ATOM .. looking for them is just a colossal waste of time money and human effort .. as a consequence of our MIS-CON-ception and MISS-understanding ..

best stop there ..

as we would not want those who have partaken in the knowledge of GOOD and EVIL gaining access to everything .. but then again they might figure out why it was such a bad idea in the first place ..

hint ..

it has a EVERYTHING to do with strict binary computation .. can draw straight lines in space .. but USELESS for plainer reference or relativity ..

Re:red herring

[Fnkmaster]

EARTH HAS 4 CORNER
SIMULTANEOUS 4-DAY
TIME CUBE
IN ONLY 24 HOUR ROTATION.

Oh wait, what site are we on again?

Re:Not sure it belongs in an intro explanation, bu

[Anonymous Coward]

What are

* Catastrophic cancellation in complex arithmetic?

and

* Range reduction in trig functions?

Pointless article.

[elashish14]

Anyone that wants to know about the real dangers of computer precision just needs to watch Office Space!

Floating point excuses

[Anonymous Coward]

Floating point meth addicts need to stop asserting their point is not floating when it clearly is. Why can't I set a float to 1 and expect 1 == 1.00?. Your answer is an EXCUSE.

I'm sick of modular arithmitic, I'm sick of crappy floating point logic. I've used enough cell libraries to know why its done and why it works the way it does. It still doesn't explain why our favorite languages STILL force us to think in these crappy terms without having to resort to quirky external meth libraries to get any real work done.

An older (still good) explanation from Cleve Moler

[Hotawa Hawk-eye]

I may be biased, but I prefer this article by numerical analyst and mathematician Cleve Moler.

Re:all your floats are belong to us

[SimonInOz]

Ah - a Microsoft Sharepoint programmer

Our team came up with a better solution

[Anonymous Coward]

Don't use float. Ever. Just use Decimal and Integer types. There's never a reason to use floating point math. They should have never invented it.

Re:Our team came up with a better solution

[OrangeTide]

My system does not have any native support for decimal types. But floats can be pipelined in hardware at blazing speed.
Maybe if my computer was made from vacuum tubes I'd do it your way.

Re:Our team came up with a better solution

[Anonymous Coward]

But floats can be pipelined in hardware at blazing speed.

In other words, you prefer to get the wrong answer really fast. :P

(* note: I'm not the AC from the grandparent post.)

Re:Our team came up with a better solution

[OrangeTide]

or I know how to manage error tolerances correctly.

Re:Not sure it belongs in an intro explanation, bu

[dr2chase]

Catastrophic cancellation -- suppose you have C1 = a + bi, C2 = x + yi. Multiplying them yields (ax - by) + (ay + bx)i. Done "naively", those embedded subtractions/additions can lead to excess cancellation. The "easy" way to deal with this is to do the intermediate multiplications (ax, by, ay, bx) into higher (doubled) precision, do the add/sub in the higher precision, and then round down to get your answer. Works pretty well for single (because double is always available), not so well in double (because quad is sometimes architected, but not supported except in emulation). There are probably some tricks for doing this well with IBM Power's fused multiply-add.

Range reduction is substantially more subtle, and it requires that you understand a particular point of view. That POV is that each machine floating point number is exactly what it is -- the representation of pi (the 64-bit mantissa approximation that you get in 80-bit Intel FP) is expressly, NOT PI, it is a different number, very close to PI. So, consider sin(PI+x) for a moment, for very small x. That should be very nearly equal to -x. Now, suppose you take a 64-bit mantissa approximation of PI, call that "pie". I happen to know that it is slightly larger than PI, hence the "PI+x" formulation (bits 65-68 = hex C) (I may be making sign errors here). Suppose you want to computer sin(something), where something happens to be pie. The first thing you notice is that it is (a) larger than PI and (b) also larger than the 68-bit approximation of PI. So, either way, you need to do range reduction, subtracting either a sufficiently large approximation of PI, or subtracting a 68-bit approximation of PI. If you subtracted true PI, what you get is 64 (63?) zeroes in the mantissa, followed by 1 at the 64th position (because pie is rounded up) minus all the following digits of true PI. If, on the other hand, you subtract the 68-bit approximation of PI, you get 64/63 zeroes followed by binary 0100 (16-12), and all the rest is ZEROES. For the "pie is a real true number" POV, sin(pie) is not, not, not 4*2**-64 -- it's that number with all the (subtracted) digits of pi.

I thought I could get an example of this with comparing C and Java on an Intel box, but gcc on my Mac is too clever and keeps calling a subroutine instead (Apple was once very, very picky about floating point, and it would not surprise me if this is special to them).

Re:Please look here (my horror story)

[shovas]

Been dealing with some issues exactly like this myself. Is there ever a situation where you would work with floats rather than integers, related to what you were talking about? Only for presentation?

Re:Hard to debug floating point when it goes wrong

[Anonymous Coward]

"Why is Floating-Point Computation so Hard to Debug whe it Goes Wrong?"

That's nthng. Strngs r hardr to dbug.

Re:Please look here (my horror story)

[Nazlfrag]

Generally speaking, it's never fine to use floats in financial calculations if you want exact results. For approximations it's ok, but why bother when you can get perfect precision using fixed point integer arithmetic. You might have concerns with bounds using traditional ints but that's why you use BigDecimal or similar.

Brings back nightmares...

[Anonymous Coward]

This brings back nightmares involving macheps and cancellation for me...

It's weird because I actually have a mathematics degree, plus a lot of CS coursework, I took the numerical analysis class that covered most of this, and yet I work in a factory where I only ever use basic geometry and the ability to sort fractions very, very quickly. (Quick, is 5/32 more or less than 1/4? Now repeat similar exercises ~1000 times/night...).

Anyone need a jack-of-all-trades computer guy for something other than measuring and putting sticky labels on thousands of pieces of cut and tempered glass?

Re:An older (still good) explanation from Cleve Mo

[Anonymous Coward]

My mom also sometimes complains when I am not home in time - because of my inaccurate watch. I heard the internet was full of investigations into and solutions to this problem but I prefer DADT (don't ask, don't tell) tactics...

and Ada

[krischik]

Correction: COBOL, PL/I and Ada. Ada has both fixed and floating point BCD arithmetic. And yes I too wonder why it is not in wider use. Perhaps it has to do with the ill conceived notion of "light weight languages" - most of which are not light weight at all any more once they are on the market for decade or so.

Martin

Re:and Ada

[Anonymous Coward]

No it doesn't. Ada has both "normal" (often--but wrongly--called binary) fixed point and decimal fixed point. The standard does not dictate a particular format. BCD is one possible implementation. Either way, neither type is a "floating-type" type.

BCD floating point

[krischik]

I'm not comparing BCD to floating point

BCD and floating point do not exclude each other. There is BCD floating point as well. And this is what we are talking about. Integer and fix point are always precise.

Martin

Human readablity

[krischik]

The notion of "precise" only comes into play when the number is presented to humans. And humans expect 1/5 to be precisely 0.2. And especially when that 0.2 is to appear on your bank statement.

If you never plan to display your data to humans then it does not matter which floating point you use.

Martin

not every language uses binary.

[arthur01]

This is one of the reasons that Cobol refuses to die. It is all done in integer maths, ie no round-off unless you specifically ask for it.

Re:1.0 Post

[Anonymous Coward]

You missed it too.

A word of advice:

[kuzb]

If you want a good paper to reach as many people as possible, it's probably a better idea not to mention Joel Spolsky in the article at all. The camp is so split where he is concerned that you may lose half your readers at the very mention of his name.

Pleasant surprise.

[Sivar]

Author: I think this article is simply very well written. I plan to send a link to all the CS professors I know.

If only the Python documentation were written half as well.

Re:What almost every programmer should know about

[Anonymous Coward]

You're such an idiot. Yeah, it "works" but the math is imprecise. When deterministic math is needed, fixed point is used. When dealing with money, a decimal fixed point, like BCD is used. Floating point doesn't just "work." You aren't a professional coder, are you?