Slashdot Mirror

← Back to Stories (view on slashdot.org)

The Trouble With Rounding Floats

Posted by ryuzaki0 on from the not-the-rootbeer-kind dept.

lukfil writes "We all know of floating point numbers, so much so that we reach for them each time we write code that does math. But do we ever stop to think what goes on inside that floating point unit and whether we can really trust it?"

2 of 456 comments (clear)

  1. Re:Decimal Arithmetic by pcgabe · · Score: 0, Redundant
    But since we are not using infinite number of decimal places, you are basically asking for a bigger float no?
    No.
    --
    Don't put advice in your sig.
  2. Re:Decimal Arithmetic by grimarr · · Score: 0, Redundant
    all floating point-style numbers suffer from this problem.


    Not exactly. The real problem is that people use decimal numbers, and computers use binary numbers. Each numbering system has some numbers that the other one cannot represent exactly. If computers used floating-point DECIMAL calculations, this would be avoided. I've never seen that done, but I have seen fixed-point decimal, and it usually solves this problem nicely. But modern computers don't do decimal arithmetic as fast as they do binary arithmetic, so it's not used as often as it should be.