Has the Decades-Old Floating Point Error Problem Been Solved? (insidehpc.com)

overheardinpdx quotes HPCwire: Wednesday a company called Bounded Floating Point announced a "breakthrough patent in processor design, which allows representation of real numbers accurate to the last digit for the first time in computer history. This bounded floating point system is a game changer for the computing industry, particularly for computationally intensive functions such as weather prediction, GPS, and autonomous vehicles," said the inventor, Alan Jorgensen, PhD. "By using this system, it is possible to guarantee that the display of floating point values is accurate to plus or minus one in the last digit..."

The innovative bounded floating point system computes two limits (or bounds) that contain the represented real number. These bounds are carried through successive calculations. When the calculated result is no longer sufficiently accurate the result is so marked, as are all further calculations made using that value. It is fail-safe and performs in real time.
Jorgensen is described as a cyber bounty hunter and part time instructor at the University of Nevada, Las Vegas teaching computer science to non-computer science students. In November he received US Patent number 9,817,662 -- "Apparatus for calculating and retaining a bound on error during floating point operations and methods thereof." But in a followup, HPCwire reports: After this article was published, a number of readers raised concerns about the originality of Jorgensen's techniques, noting the existence of prior art going back years. Specifically, there is precedent in John Gustafson's work on unums and interval arithmetic both at Sun and in his 2015 book, The End of Error, which was published 19 months before Jorgensen's patent application was filed. We regret the omission of this information from the original article.

Comments

Pi

[Anonymous Coward]

So, what is the last digit of Pi?

Re:Pi

[Anonymous Coward]

Taint bit set. Accuracy not guaranteed.

Re:Pi

[LynnwoodRooster]

42

Re:Pi

[volodymyrbiryuk]

What was the question again?

Re:Pi

[Anonymous Coward]

It's the square root of -1.

Re:Pi

[Anonymous Coward]

In base pi, it would be 1. In anything else, hahacute. Same wry point available for you tau flavor preferences.

Re:Pi

[DontBeAMoran]

I believe the question was PC LOAD LETTER.

Re:Pi

[ClickOnThis]

It's easy to refute this claim. The bible is wrong since it's just a book written by people a long time ago.

No, that's not how you refute a claim. Just because something is old and written by people does not mean it is wrong. (Of course, that doesn't mean it's right either.)

pi = 3 is wrong because it is possible to calculate the ratio of the diameter and circumference of a circle mathematically. And the answer is not 3.

Re:Pi

[Chris Mattern]

In base pi

How, exactly, do you base a digital number system on anything except a natural number greater than one?

Re:Pi

[jmccue]

In binary, 1 of course

Re:Pi

[Anonymous Coward]

Oh, it's you again.

Shouldn't you be masturbating to host files or something?

Re: Pi

[Anonymous Coward]

Unlike you, the writer of the Book of Kings (it was originally one book) knew the difference between inner diameter and outer diameter.

Re: Pi

[Anonymous Coward]

Easy.

A unit in your new pi-base system is equal to pi. A half a unit is half pi.

To represent any arbitrary number you simply state the multiplier required: 10 = 3.18 in base pi, or in other words 3.18 x pi = 10.

Note that base ten works in exactly the same way as does any other base.

Re:Pi

[Anonymous Coward]

In Euclidean geometry. With other metrics and geometries, it might be true, although not very useful in day to day life, since our environment is reasonably close to Euclidean, the general relativistic corrections being very small (but between these corrections and the fact that it is futile to try to measure, say, the circumference of a solid matter cylinder to better than the distance between nuclei in said matter, having more than 20 or so decimals of pi is useless for practical purposes).

Re:Pi

[jeremyp]

pi in base pi would be 10 (same as 8 in base 8 is 10 and 2 in base 2 is 10) so there's an argument that the last digit is 0.

Re:Pi

[Anonymous Coward]

It's 7. For any digit you find after the last 7, I can find another 7 after it.

Re:Pi

[ceoyoyo]

Or maybe 0. There's some uncertainty in the last digit.

Re: Pi

[oobayly]

I'll bite. When you say inner and outer diameter, what do you mean? I somehow doubt you're referring to the diameters of something like a washer.

Re: Pi

[Chris Mattern]

That is not how bases work. So how do you count with this new system, where "1" is an irrational number (and so is 10, "3.18" is just an approximation. In fact, *every* integer is irrational!).

Re: Pi

[BasilBrush]

You can count as many pi as you like.

Re:Pi

[Anonymous Coward]

So, what is the last digit of Pi?

Trailing zeros can be truncated so It must be 1. :)

Re: Pi

[michael_wojcik]

There's plenty of extant work showing how an irrational radix works, though I wouldn't say most of it is terribly serious. Probably the most often cited example is phinary.

Admittedly, phinary does have the advantage that non-negative integers all have a terminating exact representation in it, which base-pi does not. But this thread isn't about suitability; it's about possibility. And base-pi is certainly possible.

In fact, *every* integer is irrational!

This statement is wrong in more than one way. First, representation does not determine whether a number is irrational. Presumably what you mean is "no integers have terminating representations [in base pi]".

But that statement is trivially incorrect. In base-pi, zero is "0". Zero is an integer.

More interestingly, pi**0 is one, so in base-pi, one is "1". And since pi < 3, two is "2" and three is "3". So all the integers in [-3,3] in fact have terminating base-pi representations.

In base pi, the string "120.3" means (in decimal) pi**2 + 2pi + 0 + 3pi**-1, or about 17.11 decimal. It works just like any other radix. So you have that 1's digit, which you use for integers greater than 0 and less than the radix.

[Why doesn't Slashdot support decent HTML markup like , and entities such as &pi;? Or LaTeX math, or KaTeX, or MathML, or something? What a pile of crap.]

floating point has many problems

[iggymanz]

the "bounds" also have the same issue, it's making the problem smaller but not eliminating it

Re:floating point has many problems

[gweihir]

If done right, this works. It is also more than half a decade old: https://en.wikipedia.org/wiki/...

Re:floating point has many problems

[gweihir]

That should be "century". It was discovered around 1950.

Re:floating point has many problems

[Anonymous Coward]

Hey, I have acquired another AC stalker! Nice. Confirms that I am posting things that are worthwhile. Some types of fuckup just cannot stand truth.

Re:floating point has many problems

[Anonymous Coward]

That was my thought immediately. There is lots of prior art in this area, this is just not original.

Re:floating point has many problems

[iggymanz]

more accurate to say, if done right it makes things better.

more bits also makes things better, quad precision floats are supported either in software (e.g. Fortran 2008 including gnu Fortran, Boost libraries) or hardware such as Power9 or Z series mainframes. Sparc V8 and up defines hardware but no one has implemented it yet.

Re:floating point has many problems

[ClickOnThis]

the "bounds" also have the same issue, it's making the problem smaller but not eliminating it

This. You'll still have error propagation as you combine numbers in mathematical calculations.

I haven't read the patent, but I don't see how they can get around that. The error on a calculation will grow into the more significant digits, even if you get fancy with the handling of the last digit's error.

Re:floating point has many problems

[Entrope]

It can automate the tracking of floating-point uncertainty, so that when the answer 42 pops out, you have a good indication that your result is only accurate to plus or minus 2 to the 19th power. There are a lot of operations (for example, dividing by a very small number) where the magnitude of potential errors explodes.

Re:floating point has many problems

[gweihir]

I agree. It also depends on what you need. If, for example, you must be able to compare for ordering accurately and must know when you need extra effort, interval arithmetic is the way to go. If, on the other hand, you just need a small error, more precision in everything is the way to go. It is always good to have different tools with different strength and weaknesses available and float calculations are no exception.

Re:floating point has many problems

[stoatwblr]

> more accurate to say, if done right it makes things better.

Yup.

A few years back one of the astronomy researchers where I work came to see us oiks in the IT department with a perplexing problem:

Programs which worked perfectly happily on 32-bit OSes were giving wrong answers on 64-bit ones, even on the same hardware (linux but it was the same on windows, we checked)

After a bit of head scratching and some digging we found out what the problem was: _both_ OSes were giving wrong answers.

I'm sure some of you are ahead of me by now, but it turned out that what was being done as standard practice in the space physics community when iterating multiple orbits or similar stuff was to use the output of the last calculation as the input to the next one - problem being the imprecision in the last decimal place or so rapidly adds up to major errors - and noone had noticed this for 20+ years.

(Sheesh, and I learned how this schoolboy error works 40+ years ago, when I was a schoolboy and using 4 function calculators!)

Changing to more orthodox (but longer winded) methods resulted in both OSes giving the same answers.

Unfortunately they're still slightly wrong: Originally, modelling the orbits of the solar system on 32-bit software would result in it flying apart after 11 million years or so. On the 64-bit machine it would happen in about 8 million years. Now it's more like 30 million years, but it still flies apart.

We're still here, so the calculations need more tweaking... :-)

Built-in error bars

[Anonymous Coward]

For those of you too lazy to read the summary, their method is that instead of using one floating point value, they use two and say the real answer is between those two. If the two floats are consistent when rounded to the requested precision, it declares the value correct. If they differ, it gives an accuracy error.

So, for only twice the work and a little ovehead on top, this process can tell you when to switch to a high precision fixed-point model instead of relying on the floating point approximations.

Re:Built-in error bars

[K. S. Kyosuke]

Balls and intervals have been here for quite some time, though. Not sure that this is merely "twice the work", though.

Re:Built-in error bars

[Anne Thwacks]

In the olden days (1970's) we did the computation in standard and double precision - if the answers were different, then you probably needed to change the method. (The underlying problem is "underflow" - you do not have any significant digits).

No special hardware or patents needed.

Re:Built-in error bars

[Anonymous Coward]

There is already a common software solution - use an appropriately sized long int (with optional mantissa, etc) and then perform display munging to show the simulated float value. Increase the number of bytes necessary to handle all the significant digits. If you exceed 64 or 128 bit numbers (depending on CPU), there are straightforward algorithms for splitting the ints into segments and processing them that way. The main downside here is CPU cost as you can't pack it into a chip very easily, I assume this guy has a bent toward processor design.

For example, managed runtimes like C# System.Decimal have 128 bit integers backing them to perfect precision within the numerical range (102 bits represent the number). Floats have a somewhat larger total numerical representative size, however 256 bits are not impractical for storing this kind of accuracy, and multiple long int operations are pretty inexpensive on today's CPUs.

It's basic information theory - if you need to represent every digit accurately, you need to store every digit in the information of the bits. There is no free lunch.

Re: Built-in error bars

[Anonymous Coward]

> So, for only twice the work and a little ovehead on top, this process can tell you when to switch to a high precision fixed-point model instead of relying on the floating point approximations.

Which you should already know how to do already. Getting a âoeyou dun goofedâ message is less helpful than checking the stability of your numerical algorithms and generally not being a fucking moron in the first place.

Re:Built-in error bars

[gweihir]

This is also known as Interval Arithmetic, vintage ca. 1950 and available in many numerics packages. Just putting known algorithms in hardware does not make them anything meriting a patent.

Re:Built-in error bars

[Anonymous Coward]

Even if it is "twice the work", it can be done in parallel, and hardware is cheap and easy these days.

Re:Built-in error bars

[Anonymous Coward]

So the Scud missile is somewhere between here and 1 mile away.

Re:Built-in error bars

[Anonymous Coward]

Last mile internet delivery by Scud missile is just too slow.

Re:Built-in error bars

[sjames]

So the newly patented implementation of a 50-60 year old technique that I learned in high school chemistry is a "game changer"?

Re:Built-in error bars

[Anonymous Coward]

To put it in perspective: With 259 bits you can represent the diameter of the visible universe in planck lengths.

At that point the exponent bits in the floating point are just wasteful and you should switch to integer calculations.

Re:Built-in error bars

[TechyImmigrant]

I've done 300 bit precision arithmetic using gmp, calculating uniformity bounds for a crypto thing. Computing that with bounded doubles would not give me any more precision.

I would only know it's broke. When you need more precision, more bits is what you need, usually in your mantissa.

Re:Built-in error bars

[JoshuaZ]

Yeah, I don't see how this isn't just a hardware implementation of interval arithmetic https://en.wikipedia.org/wiki/Interval_arithmetic, which is a topic basic enough that I teach aspects of it as a secondary topic to my Calc I students. It is possible there's something deeper here that we're missing but if so, it doesn't stand out.

Re:Built-in error bars

[MightyMartian]

I certainly had to do that when I wrote some financial software years ago. Floating point math, even at two decimals, was just way too error prone. Some some long ints did the trick and the decimal point was inserted when the number was formatted for display.

Re:Built-in error bars

[Impy the Impiuos Imp]

Every time you multiply two floats you lose a digit of precision. It's a little more complicated but that is the essence.

The butterfly effect was discovered with weather simulations. They saved their initial data and ran it again the next day -- and got a different result, which is impossible.

Turns out they only saved the initial conditions to 5 digits and not the entire float, or what passed for it in the 1970s.

Lo and behold! The downstream numbers, far from returning to the same value, diverged wildly. And no matter how small the difference, it always diverged.

Up until then, scientists had believed small differences would get absorbed away in larger trends. Here was evidence the big trends were completely dependent on initial conditions to the smallest detail.

That's why one stray photon would screw up time travel -- any difference whatsoever would cause the weather to be different in about a month and soon different sperm are meeting different eggs, and the entire next generation is different.

So any time you do more than a trivial number of float multiplies, you are in a whole different world. This is ok if you are looking for statistical averages over many, but miserable if you want to rely on any particular calculation.

Re:Built-in error bars

[Impy the Impiuos Imp]

Bouncing virtual molecules around in a weather simulation will eat those 256 bits, or digits, in less time than you can blink, and give different results with the 256th bit different in one molecule's position or momentum.

Re:Built-in error bars

[Impy the Impiuos Imp]

If it gets people to pay attention to "interval approaching size of the known universe" 20 multiplies down the road, sure.

Re:Built-in error bars

[Anonymous Coward]

Of course it is, now your teacher (and any other one teaching the same) can be sued for patent infringement.

Re:Built-in error bars

[Cassini2]

The reason why it is not "twice the work" is because a crude interval approach only works for linear equations. Consider: y=x^2, for small x. If the lower limit is x=-0.1, and the upper limit is x=0.1, y=0.01 in either case. However, consider the case where the actual x=0, with the actual y=0. It can be seen that for -0.1 < x < 0.1, y is outside the predicted range.

For many simple systems of equations, it is possible to get good error analysis by using some Calculus. Specifically, multiply the expected error in the inputs by the derivative (or an upper bound on the derivative).

Unfortunately, most of the people working in this area are solving complex systems of equations, often involving large matrices. This makes the problem difficult to detect with computationally fast approaches. Some people use Monte-Carlo simulations to get estimates of the likely error. These situations require far more computations than double (often thousands of runs). It is also possible to do some serious error analysis to determine when the linear interval analysis applies, and when it doesn't. However, this also requires far more calculations.

Re:Built-in error bars

[Anonymous Coward]

But the differences are meaningless. Substantial, yes; but since differences smaller than the resolution of the universe can lead to meaningful divergence in the simulation, that precision has little value.

Re:Built-in error bars

[sjames]

It won't. The technique has been available for 50-60 years and it hasn't yet. To use the new implementation, code will have to be changed and you'd have to be using a processor that supports it.

Re:Built-in error bars

[thegarbz]

So unlike their claim this works works in almost precisely half the realtime speed of traditional floating point arithmetic.

Re: Built-in error bars

[Anonymous Coward]

It's why I voted for missile neutrality. Missiles are missiles.

Re:Built-in error bars

[flargleblarg]

That's why one stray photon would screw up time travel -- any difference whatsoever would cause the weather to be different in about a month and soon different sperm are meeting different eggs, and the entire next generation is different.

s/would/could/g

Re:Built-in error bars

[whit3]

instead of using one floating point value, they use two and say the real answer is between those two. If the two floats are consistent when rounded to the requested precision, it declares the value correct. If they differ, it gives an accuracy error.

So, for only twice the work and a little ovehead on top, this process can tell you when to switch to a high precision fixed-point model ...

There's flaws in the principle other than the obvious doubles-the-work feature, though. Error 'bars' only show a pair of worst-case values that is purported to represent the error, but careful measurement processes give a distribution of errors.

One can analyze a voltage and temperature tolerance range for a CPU chip; it gives rise to 2^2 = 4 vertices for testing.

That DOES fit the model proposed. Measured-accuracy is usually less precise than a floating point number, so LSB error isn't our accuracy limit.

Taking four measured numbers, each with an associated thirty-samples selection of deviants, by calculating all the combinations: 30^4 (about a million) calculations later, you know not only the result, but a collection of deviants that can be distilled down to a new thirty samples, representing the error distribution of the result..

While the LSB granularity of a computer number is a kind of 'error bar', and a calculation on that kind of error IS just two worst-case values, there's still the nagging problem that the worst cases are overestimates of the error to be expected. In a statistical sense, worst-case is always wrong for large-scale calculations (an overestimate). When the error distribution is a bell curve, there IS no defined worst-case that really represents the situation.

By the central limit theorem, we always expect the result of a many-input calculation to have a bell-curve distribution, so the result of a large calculation is NEVER well-characterized if you propogate worst-cases.

Re:Built-in error bars

[Pseudonym]

That's not the only flaw, of course. If you take a workhorse numeric problem like integrating an ordinary differential equation, interval arithmetic can give you a bound on how accurate the calculation is relative to an infinite-precision calculation, but not on how accurate the calculated solution is relative to the true solution. I'll give you one guess which bound is the one you actually want.

In the case of ODE solving or numeric quadrature, the thing that determines the accuracy of the solution is how well-behaved the function is in the regions that fall between your samples. Neither interval arithmetic nor this "bounded floating point" is going to help you here.

Now having said that, I did read it, and the method described is not quite interval arithmetic. It's a little more subtle than that: the programmer sets the desired error bounds on the solution, and the FPU does something like a quiet NaN if the calculation exceeds those bounds.

And yes, just like with interval arithmetic, all our floating point libraries will need to be rewritten.

the result of a large calculation is NEVER well-characterized if you propogate worst-cases.

That's true, but TFA is talking up the safety-critical aspect, which I suppose is the one place where worst-case behaviour is what you want. TFA also mentions weather simulations, and that's exactly the case where the distribution of solutions is the answer you want, not worst-case bounds.

It's a little bit interesting, but Betteridge's Law definitely wins this round. I can see this as being slightly more useful than existing techniques (e.g. interval arithmetic) in some safety-critical systems, but I'd rather just have finer rounding control on a per-variable basis. We've had SIMD instructions on commodity hardware for almost two decades now, so I don't know why we don't already have essentially-free interval arithmetic within a register.

This doesn't "solve" "the floating-point error problem", as if there's only one problem outstanding. The full-employment theorem for numeric analysts has not been violated.

Re:Built-in error bars

[Anonymous Coward]

Many (many) years ago when I had a summer job at an aerospace firm, my boss had long ago developed a Fortran program for computing fluid flows through interconnected pipes/tubes using an iterative method. Unfortunately, there were some corner cases that it gave ridiculous results for or failed to converge. He tasked me to look at these cases although I didn't have a strong math background and hadn't even completed a numerical analysis course yet. I noticed he was building it with single precision floating point rather than double precision. I just rebuilt it with double precision and the identified corner cases then gave reasonable (although, who knows if they were "correct") results and/or converged on (some) answer. I went back to him and explained what I had done and that I didn't think I had the math background to do the formal analysis to figure out how to avoid the underlying error propagation but that he probably should find someone that had that background. He seemed very content and seemed unconcerned and I never heard that anyone else got assigned to the task -- but I have the uneasy feeling that he just changed the default build to double precision rather than fixing the algorithm.

It appeared that this program had been used in the design of some tools of war that peoples' survival depended on -- which was pretty scary when I thought about it.

Re:Built-in error bars

[drinkypoo]

s/would/could/g

Either events are preordained and one photon doesn't matter that much, or the world is mechanistic and one photon will change everything.

Re:Built-in error bars

[sysrammer]

Either events are preordained and one photon doesn't matter that much, or the world is mechanistic and one photon will change everything.

Or it's both and you can't tell until you open the box.

Re:Built-in error bars

[140Mandak262Jamuna]

Every time you multiply two floats you lose a digit of precision. It's a little more complicated but that is the essence

Not a whole decimal digit. Just one bit. Right?

Re:Built-in error bars

[swilver]

Up until then, scientists had believed small differences would get absorbed away in larger trends.

They do, in nature, just not in computers that treat every result as exact instead of fuzzing the results appropriately.

Re:Built-in error bars

[Shinobi]

Actually, implementing the logic gates for it can very well be worth a patent. Since there doesn't seem to be any prior art for actually implementing it with logic gates in a quick patent DB search, that definitely makes it a justifiable patent.

Re:Built-in error bars

[KiloByte]

Or, in other words, a stable system vs a chaotic system.

It appears that our world is fundamentally chaotic, and that single tray photon causes a sphere of changes that expands with the speed of light.

Re:Built-in error bars

[KiloByte]

Not if these divergences increase exponentially. Think of a billiard ball: a minute change of its position or angle won't have a noticeable effect if it doesn't interact with other balls. Every time it hits another ball, though, the divergence is amplified.

Re:Built-in error bars

[Mal-2]

That's why one stray photon would screw up time travel -- any difference whatsoever would cause the weather to be different in about a month and soon different sperm are meeting different eggs, and the entire next generation is different.

This is why I'm positing complex, hyperbolic time when I invoke "time travel" (which it only sort of is since it still doesn't involve going backward, just altering the angle at which you proceed forward). Even if you could figure out how to swim upstream, you still could never return to a previous point because the hyperbolic nature of the complex time plane magnifies the inevitable error on the complex axis. You could go back to 1933 and kill Hitler, but it would be in someone else's timeline, thus no grandfather paradox.

Why is the time plane hyperbolic? To make room for all the points of divergence from quantum collapse. Many Worlds is correct, and to handle every possible path becoming real, the time plane is hyperbolic to give them somewhere (or rather, somewhen) to call their own.

Re:Built-in error bars

[Anonymous Coward]

My professor taught me this great little gem: "If you're ever worried about the error being too large, just multiply by zero to remove the error, and then divide by zero to cancel out the scaling. Problem solved, right!?"[1]

[1] = Note: You have to give a slack-jawed grin at the end when delivering this in person.

Re:Built-in error bars

[Anonymous Coward]

I think he's assuming that you started out with a decimal number that was measured with +/- 0.5 precision for the last-place digit.

Re:Built-in error bars

[Anonymous Coward]

This is why I'm positing complex, hyperbolic time when I invoke "time travel" (which it only sort of is since it still doesn't involve going backward, just altering the angle at which you proceed forward). Even if you could figure out how to swim upstream, you still could never return to a previous point because the hyperbolic nature of the complex time plane magnifies the inevitable error on the complex axis.

Actually, this is not a new idea. It was proposed by a military research project and reported in an issue of the "Proceedings for the Society of Psychical Research" in about 1972, printed after the author died. It may have been suppressed later, I don't know.

Re:Built-in error bars

[Anonymous Coward]

Floating point crypto = You're joking, right?

Re:Built-in error bars

[Anne Thwacks]

Actually, implementing the logic gates for it can very well be worth a patent. Since there doesn't seem to be any prior art for actually implementing it with logic gates in a quick patent DB search, that definitely makes it a justifiable patent.

In most countries, patents require the invention not to be "obvious to anyone sufficiently skilled in the art". Only in America is "not blatantly obvious to a blithering idiot with some $$$" the criterion.

Re:Built-in error bars

[gweihir]

Indeed. It is time this US-based nonsense stops and patents are only granted for things that have more merit than standard straight-forward engineering work.

Re:Built-in error bars

[Anonymous Coward]

Unfortunately, you understand neither the algorithms nor even the intent of the algorithms.

For each basic operation it creates 2 floats, one rounded up, and one rounded down. Initially, they only differ by zero or one bits. Then these two go into the next operation, yielding four floats, and you take the min/max. And so on. The final two floats are almost never the same - this only occurs in the rare case that the result of the operation chain is exactly expressible as a float. This can fail to happen in a single + or - operation, and almost never happens in a single * or / operation, so after a chain of such operations it's incredibly improbable. So there's no "accuracy error" - that would trigger for every computation and become noise. Instead, they take the 2 floats and give you the nominal value plus the error bar, so you can print them to the appropriate number of significant figures, or do other groovy things.

They can also detect fundamental issues in your algorithms. e.g. if you divide 2 by the interval [-e,+e] the range of the result is +/-INF. This indicates that you have basically no information here and screwed up your numerical algorithm design.

I'm not sure why putting these ancient algorithms into hardware is a novel invention, though.

Re:Built-in error bars

[Anonymous Coward]

Um, no.

If you apply interval [-0.1,0.1] to the operation x^2, the result is the interval [0,0.01], not the number 0.01.

The algorithms know this even if you don't.

The same issue occurs for 1 / [-0.1,0.1], whose result is [-INF,+INF] and not [-10,+10], or whatever other wrong answer you can dream up.

So thanks for explaining why an algorithm you don't understand doesn't work if you implement it without understanding how it works.

Re:Built-in error bars

[Anonymous Coward]

Not a whole decimal digit. Just one bit. Right?

Depends on what you mean.

When multiplying two unsigned 8 bit values you will need 16 bits to store the result.

A 32bit float have 24 bits worth of mantissa. (With the top bit implicit.)
The product of them will have 48 bits mantissa to be exact.
When the result is packed back into a float the lower half is dropped. (Or rounded to one LSB depending on FPU implementation/settings.)

The result has as many significant bits as you started it so it wouldn't be completely wrong to say that you only introduced one LSB of error with the truncating/rounding, but it wouldn't be completely wrong to say that you lost half of your bits either.

Now, for practical purposes you don't really care about those lower bits so the normal response should be that you lost one lower digit.

When you do addition it is a lot worse, then you drop as many bits from the lower value as you need to get the same exponent as the higher value. Sometimes that is the entire mantissa.
If you are going to add a lot of floats together you should apply an algorithm similar to huffman encoding.
1) Sort the values.
2) Add the two lowest values together and replace them with the sum.
3) Goto 1 until there is only one value left.

The really tricky part is simulations.
Many simulation models are iterated from the previous state.
If you do a really really good implementation that works on a lot more bits during calculations you still drop one LSB every time you store it back into memory.

If you use 32 bit floats to store the states then you won't have any significant bits left after 16 million iterations.
Going up to 64 bit floats will help. Then you can do a billion iterations and still have 23 significant bits left. (2^(52+1) - 2^30 = 2^23)
That is about what you have if you do a single iteration with 32 bit floats.

Re:Built-in error bars

[Anonymous Coward]

The summary (and TFA) is misleading as expected. The quote from TFA "it is possible to guarantee that the display of floating point values is accurate to plus or minus one in the last digit" is also misleading. The trick is the "last digit" which is now redefined to something else. Of course, one could define how accurate the one can compute to a certain number of digit. The quote makes it looks like the new system/way of floating point can completely eliminate the current floating point error. Though, it rather suggests a new way we can handle the floating point which could have more benefits in a certain way.

Re:Built-in error bars

[elistan]

Up until then, scientists had believed small differences would get absorbed away in larger trends.

They do, in nature

Source? My understanding is that small differences lead to huge differences in nature. I'd imagine that a Pachinko game is a good illustration of that - two different balls with minutely different starting positions/velocities will follow significantly different paths eventually. It's only when you look at the statistical averages of significantly numerous paths that a pattern/trend emerges. In this way, nature is exactly like computer calculations.

Re:Built-in error bars

[stoatwblr]

The best patents are for things that weren't obvious beforehand but are obvious afterwards.

The question becomes "Why had noone implemented it in hardware before?" (ie, had it been suggested and thought too hard?)

Re:Built-in error bars

[Mal-2]

The time bit isn't the story though. It's my deus ex machina to get the people to where the story happens and not a lot more. Thus, I don't mind if the idea is not new. It's nice to see it's somewhat credible-sounding, and I don't have any entropy violations because there is no backward time travel -- it will be implied more than stated that it is still considered impossible since the tech level will only be 20 years from now. The story is more "Gilligan's Island, in space, with monsters".

Re:Built-in error bars

[Shinobi]

Just because something is "obvious" in math(that is, on the theoretical side) does not mean how to implement it in hardware is obvious. You are making a false equivalence here.

Oh, and making an "obvious" claim in hindsight is both intellectually and morally disingenious. A patent DB search yields me no results on previous attempts to implement this in hardware, thus it is justifiable. There's no prior art for an actual hardware implementation. The theory side doesn't come into it.

Re:Built-in error bars

[TechyImmigrant]

Floating point crypto = You're joking, right?

Floating point evaluation of the bounds on an integer algorithm's uniformity. It's really rather serious.

Probably not.

[Anonymous Coward]

Depends how fast it is.

No

[Anonymous Coward]

It's patented, and will be used to extort money from any company that chooses to implement it. It'll never reach any mass adoption.

Reinvented interval arithmetic

[Anonymous Coward]

As mentioned in the summary, this sounds no different from the age old interval arithmetic. The reason interval arithmetic never took off is that for the vast majority of problems where error is actually a problem, the bounds on the error become so large as to be worthless. To fix this you still need to employ those specialist numerical programmers, so this doesn't actually get you anywhere.

Re:Reinvented interval arithmetic

[Anonymous Coward]

Good. At least someone noticed. Interval arithmetic was already in some old Fotran compiler (77, I think). Let me see... ah here you go. According to this article, there was even a package for the Zuse Z23.

Old is new and new is old, I guess.

Re:Reinvented interval arithmetic

[Pseudonym]

If you read TFA, it is a novel variant of interval arithmetic. You can think of this as interval arithmetic using a more compact representation, plus something like quiet/signalling NaNs if a computation exceeds programmer-defined error bounds.

I can see a few use cases (e.g. maybe some safety-critical environments), but it isn't "the floating-point problem".

We already have a better solution!

[Anonymous Coward]

Represent them as a ratio! That way it will be *perfect*.
Of course this doesn't help with irrational numbers. But neither does this, or any other solution, ... If we could represent them lovelessly without using full-blows formulas, they wouldn’t be irrational numbers!

Then again, I'm a fan of calculating with formulas (aka higher-order functions).

Re:We already have a better solution!

[Anonymous Coward]

What about irrationals, which are as numerous as the whole of real numbers?

Re:We already have a better solution!

[Anonymous Coward]

You're irrational! The whole number system is irrational!!

Re:We already have a better solution!

[Waffle Iron]

What about irrationals, which are as numerous as the whole of real numbers?

Approximate them as 22/7.

Re:We already have a better solution!

[mi]

irrationals, which are as numerous as the whole of real numbers?

Irrationals are, actually, a lot more numerous than rational numbers. Of the latter there are just as many as there are integers (and naturals), whereas the set of the former is a lot more powerful.

Yes, the parallels with human society make me sad too...

Re:We already have a better solution!

[michelcolman]

But there is a rational number between every two irrational numbers! Words like "numerous" are tricky that way...

Re:We already have a better solution!

[ClickOnThis]

"Numerous" is not the best word to use here, because it implies countability. Rational numbers are countable, i.e., you can construct a 1-to-1 mapping with the integers, but any attempt to place the irrational numbers in a 1-to-1 mapping with the integers can be shown to have omitted at least one irrational number.

One can say that the set of rationals and the set of irrationals are both infinite, but the irrationals have a higher infinitude than the rationals. The curious reader should look up Transfinite Cardinals.

Re:We already have a better solution!

[michelcolman]

Yep, I'm familiar with all of that:
- Ordering fractions diagonally to connect them to naturals
- Assuming an ordered list of all the reals, taking the first digit + 1 behind the decimal point of the "first" real, the second digit + 1 of the "second" real, etc. yields a real that is not in the list

But it's still weird that, if there are so many "more" reals, you cannot find two reals without a rational in between. When dealing with infinities, it's really easy to take a wrong turn because assumptions valid for finite sets no longer hold for infinite sets.

You mean Intel and AMD? Who use nothing patented?

[raymorris]

> any company that chooses to implement it

This patent relates CPU design, for CPUs used in critical applications involving lots of floating point operations. So by "any company" you mean "AMD and Intel".

> It'll never reach any mass adoption.

Right, neither Intel nor AMD would EVER license any patents for use in their processors (eyeroll).

Quite float-point-ish (= inacuratish)

[CustomSolvers2]

Various thoughts about the claims in the summary, title and article:

- There are some numeric decimal types which allow a perfect precision like the .NET decimal types (C# decimal and VB.NET Decimal). Logically, there are other problems associated with these types as far as the floating point limitations aren't absolutely unavoidable, but outcomes of the pursued goal (= big values by consuming as less resources as possible).

- This approach doesn't seem to try to avoid the default problems as the aforementioned types, but just to warn about potential inaccuracies. According to the linked article:

The innovative bounded floating point system computes two limits (or bounds) that contain the represented real number. These bounds are carried through successive calculations. When the calculated result is no longer sufficiently accurate the result is so marked, as are all further calculations made using that value.

- Any floating-point-type error is certainly programmer's fault as far as being aware about these limitations is part of the basic knowledge which someone working with big decimal values should posses. So, avoiding problems like the one referred by the quote below these lines doesn't require a programming structure re-definition, just a competent developer.

the most notorious floating point error was the Patriot missile failure in Saudi Arabia...

Re:Quite float-point-ish (= inacuratish)

[HiThere]

Sorry, but while it could be the programmer's fault, this isn't certain. Different processors have different precisions, unless you are saying he should do the entire thing in software using infinite precision integers.

It's reasonable to claim that if it's still running on the platform it was designed for, that it's the programmer's fault, but I've had things migrated without my knowledge. Usually, admittedly, to a platform that had higher precision, but there have been exceptions. One time I *did* rewrite it to to calculations using integers, but I've never gone so far as to use infinite precision integers. There are cost trade-offs in everything.

Re:Quite float-point-ish (= inacuratish)

[CustomSolvers2]

Sorry, but while it could be the programmer's fault, this isn't certain.

There is something definitively certain: errors should be expected. You can do some tests under the given conditions to see the more common upper limits and go down a couple of digits to play safe. Within the 5-7 decimal digits range everything tends to work fine with eventually some rarely-problematic 0.9999 rather than 1.0 situations. Alternatively, you might use different approaches to get an almost perfect precision under virtually any scenario; logically, at some expense like everything else, including hiring more competent programmers.

Programming languages/algorithms allow tons of different ways to accomplish the same thing and there is always some uncontrollable-error-free option. So and unless under extremely unlikely conditions (programming language in-built problem), all the programming errors are the given developer's fault. On the other hand, some situations can be really complex and human errors might be somehow excusable, but this cannot justify ideas on the lines of "let's avoid future errors by modifying the programming language". You can certainly improve the programming experience, but a developer being perfectly aware about the given conditions will always be the best way to deliver reliable software.

Re:Quite float-point-ish (= inacuratish)

[HiThere]

And once, when it was appropriate, I did replace the floating point approximation by an integer based approach. But there are costs in every choice. Usually that's a bad idea, even if it does mean the thing will either work properly or fail noticeably (e.g., out of memory error). But any design makes assumptions about the hardware it's running on. It's usually possible to check, but that's usually not a good approach...or wasn't in the environment in which I was working. If I'd been writing a library or program intended to public distribution the appropriate choices would have been different.

E.g.: Is it appropriate to represent a year with two digits? These days the answer is clearly no, but 40 years ago the answer was "probably yes". And that was the correct answer. Memory was a lot more limited and computer time was a lot more expensive. And storage was also more limited and expensive. The problem is that things written 40 years ago that weren't expected to keep being used kept being used without being updated. And also in the intermediate period at some point the default assumptions should have been changed. I was luck during that period. We kept re-writing our programs and updating our assumptions. But this wasn't true everywhere. Until a year ago I still had a computer system that sorted files by the two digit year number...and it was a mass market system. By 1995 the two digit year should have been outdated, but I'm not sure it was the programmer's decision. In 1985, however, disk and memory space was still expensive enough that a two digit year was the right decision.

So. If I write a program in, say, Fortran that makes certain assumptions about how floating point numbers are represented, say I write it for a 32 bit CPU, and someone migrates that code to a 16 bit CPU (this is a guess, not what happened) is that my fault? You can't predict everything, and I was predicting migration to a 64 bit CPU, but it needed to currently fit in a small amount of RAM.

Well, as I said, that's not what happened, but what happened was on mainframes back when nobody was using personal computers, and time was *expensive*, so I designed things to run efficiently. And that means you don't do things like checking array bounds, or floating point precision. You *design* them correctly. And you depend on human checks to catch the errors (which they did, or I wouldn't have found out about it). But sometimes someone else decides to move something from one machine to another, and different machines have different designs for floating point units. IIRC this was between IBM and CDC, but it's been a long time, and I'm not sure anymore. I presume analogous things still happen, though things have changed enough that a lot of features of the analogy will be quite different.

Re:Quite float-point-ish (= inacuratish)

[CustomSolvers2]

But there are costs in every choice. Usually that's a bad idea,

What you are suggesting then? Relying on a faulty approach when you do need to have the maximum precision? Floating point types exist because they are fine for most of scenarios, but there might be cases where their limitations aren't acceptable and the given programmer would have to come up with the best solution under the given conditions.

The person developing a program sending a rocket to the wrong place was undoubtedly responsible. If certain precision was required, that person would have had to rely on whatever methodology to accomplish it (+ eventually sacrifice any side advantage like ease of implementation or performance). If in the given conditions there is no way to accomplish whatever goal under whatever conditions, you would have to accept it. It might be slower, harder to developer or whatever, but sending a rocket (or performing whatever other actions) within the expected precision is extremely likely to be doable and, consequently, that error is programmer's (or manager's or client's or whoever-made-the-final-decision's) fault.

Is it appropriate to represent a year with two digits? These days the answer is clearly no, but 40 years ago the answer was

I am not saying otherwise! As said above, for most of scenarios, you don't need even to care about floating-point precision problems (who cares about what happens in the 10th decimal position!!), that's why this is a widely used numerical type. But that person in that development should have taken into account the accuracy peculiarities. Exactly the same that a person developing nowadays a piece of software dealing with years shouldn't just account for 2 digits.

On the other hand, people's knowledge, resources, past problems, etc. aren't the same everyime and everywhere. What is evident today might not have been too clear some years ago. But again this is a matter of context and this is exactly were my words are expected to be understood: creating a new programming approach (this one or any other one) is rarely the solution for whatever problem, but properly knowing how to deal with the given environment and conditions. Theoretically, the developer is always the one to be blamed, but usually that blame has to be properly understood within the given situation (would have other person under equivalent conditions done the same?).

say I write it for a 32 bit CPU, and someone migrates that code to a 16 bit CPU (this is a guess, not what happened) is that my fault?

This seems like a quite extreme scenario, but the answer would be it depends: was you being aware about that eventuality (your software having a relevant floating-point dependence and being potentially used under different conditions) part of your due diligence? Because if you are developing a piece of software to calculate the coordinates where a rocket has to land, where each digit is relevant and you blindly use a type who has well-known limitations on this front without deeply testing the approach, it would sound quite negligent to me. But for over 99% of the all the possible developments, not caring about floating-point limitations seems even the usual proceeding. As said, below 5-7 decimal digits, everything should be fine.

Re:Quite float-point-ish (= inacuratish)

[Anonymous Coward]

Did you mean to say, "(= big values by consuming as few resources as possible)"? You're welcome.

Re:Quite float-point-ish (= inacuratish)

[CustomSolvers2]

Did you mean to say, "(= big values by consuming as few resources as possible)"? You're welcome.

Thanks for the remark, other AC. I will try to repay your noble action by pointing out that the comma in "did you mean to say, whatever" is wrong.

Re:Quite float-point-ish (= inacuratish)

[david_thornley]

Exact decimal representations only work if all the numbers are exactly representable in a decimal representation of bounded length. That isn't the case with something as simple as 1.0/3.0. The decimal numbers are primarily designed for financial calculations, in which all the numbers are normally set up to only require a few decimal places, and where the rounding methods are fixed. In any situation where we're dealing with numbers that haven't been deliberately picked to be easy special cases, regular floating-point arithmetic is better.

Floating-point arithmetic is inexact. What the decimal types you mention do is be inexact in the exact same way financial calculations were done by hand, which is exactly what you need for some calculations and irrelevant in most others. Floating-point error is almost inevitable, no matter the skill of the programmer. Figuring out what the errors do gets increasingly difficult real fast, and most people just use the algorithms the numerics people recommend.

Re:Quite float-point-ish (= inacuratish)

[CustomSolvers2]

Exact decimal representations only work if all the numbers are exactly representable in a decimal representation of bounded length. That isn't the case with something as simple as 1.0/3.0.

Logically. But this isn't what the floating-point problem is about. Imagine that the real value is 0.12345679101112131415 (...) and that getting up to the 8th decimal position is acceptable under the given conditions, a floating-point type might mess this number up and deliver something like 0.12345621, unlikely a non-floating-point type which will always deliver 0.12345678.

The decimal numbers are primarily designed for financial calculations

Certain not. Similarly to what happens with pretty much everything else, there are tons of other scenarios where this might become useful. Trivial examples are most of trigonometrical calculations which involve Pi. As explained in some other comments, you (meaning a competent programmer) should understand properly the situation and deliver the best solution under the given conditions, what includes meeting the expected accuracy within the given precision, input conditions and further issues.

What the decimal types you mention do is be inexact in the exact same way financial calculations were done by hand

Not sure what you mean with that, but the aforementioned decimal types deliver a perfect representation of the given number up to the maximum supported precision (around 25 decimal positions). So and unlikely what had happened with the floating-point alternatives (the main one in the .NET languages is double), the aforementioned sample representation of a decimal number would be stored and dealt with perfectly (= all its 20 relevant decimal digits will always be considered).

Floating-point error is almost inevitable, no matter the skill of the programmer

Inaccurate or, at least, misleading statement. Floating point errors are unpredictable (or, at least, difficult, not cheaply predictable) and uncorrectable (same clarification than before), but they happen within a relatively-known range and that's why they are controllable. A proper programmer working on a development where up-to-the-last-digit accuracy is very important should be perfectly aware about all that and only deal with floating-point types when being completely sure that no error would be provoked. I have never developed a piece of software failing because of a floating-point error + said that this was unavoidable rather than my full responsibility and am quite sure that will never do. Same ideas are applicable to SQ-injection-vulnerable code, wrong memory allocations or any other scenario involving not properly understanding and caring about all the possible conditions. I can certainly make mistakes, but will always accept the associated responsibility (= I am the one to blame, not the programming language/software as a whole).

and most people just use the algorithms the numerics people recommend

I don't know who these numerics people are, but in principle I could consider myself one of them. And I recommend you the same than with any other kind of algorithm: have full control over all the I/O, potential scenarios or anything else which is expected to affect it. If you rely on third party codes, you should accept the consequences and wish that their developers do apply these ideas. In any case, whatever error on these lines (= not provoked by extreme conditions like bugs in the programming language or computer) is certainly avoidable + responsibility of its author; anyone telling you otherwise is either lying or isn't competent/knowledgeable or isn't working under the best conditions (e.g., unmotivatedly rushed milestones and similar managerial nonsense).

Re:Quite float-point-ish (= inacuratish)

[david_thornley]

Um, yes. If you use a floating-point number of insufficient precision, it won't work well. People generally know this stuff.

As a competent programmer, I do know that decimal calculations do not do well with transcendental numbers or trigonometric functions, and you should just go with regular floating-point. It's just as accurate (given enough precision), it's faster, and it's probably more predictable.

As you say, the decimal data type supports decimal places exactly. (Duh.) So, what's that useful for? If you use it for calculations in general, it's no more accurate than a regular floating-point number of adequate precision, and it's slower. It works great for financial calculations, which are done in decimal and have to come out the same way every time. Financial calculations involve numbers with exact decimal representations (such as a 3.25% interest rate, not 22/7% or pi%), and when there is necessary inaccuracy there are rules saying how to handle it. Financial calculations were built around base 10, unlike most calculations, which are merely expressed in base 10 when convenient.

You really don't know much about floating-point, do you? Here is a copy of "What Every Computer Scientist Should Know About Floating-Point Arithmetic". I strongly suggest that you read it. Floating-point calculations are by nature inexact, unless by chance the numbers have an exact representation in the base being used, and most don't. It's hard to know exactly where the errors come in, and in complex calculations it becomes pretty much impossible.

There is a discipline in computer science that deals with this, and that's what I mean when I refer to "numerics" people. You are obviously not one of them. They are going to come up with better ways to calculate things than you or I will. As far as third-party code, if you're going to be consistent about it you'll have to write your own .NET runtime and program directly in its intermediate language, without using a compiler.

Re:Quite float-point-ish (= inacuratish)

[CustomSolvers2]

you should just go with regular floating-point. It's just as accurate (given enough precision), it's faster, and it's probably more predictable.

As previous said to you and to others before, it is a matter of context!!! If I need to have a full control on each single digit, I have to rely on decimal values and I am using a .NET language, I would certainly rely on decimal. In fact, I do rely on that type a lot in quite a few scenarios.

As you say, the decimal data type supports decimal places exactly. (Duh.) So, what's that useful for? If you use it for calculations in general, it's no more accurate than a regular floating-point number of adequate precision, and it's slower. It works great for financial calculations,

You are repeating your first sentences over and over, despite all my comments?! (like quite a few times before).

You really don't know much about floating-point, do you?

Where have you got that statement from what I wrote?! Seriously, what is the matter with you?! (not the first time I am saying this!).

Floating-point calculations are by nature inexact,

Seriously?

There is a discipline in computer science that deals with this, and that's what I mean when I refer to "numerics" people. You are obviously not one of them.

Why? Because you say so? A person unable to understand the few ideas I have written in my previous post.

As far as third-party code, if you're going to be consistent about it you'll have to write your own .NET runtime and program directly in its intermediate language, without using a compiler.

Why would I do such a thing? Aren't you seriously able to understand the difference between writing a whole algorithm yourself or relying on parts developed by others vs. being forced to trust whatever programming language (+ computer + building where you are coding + everything else!!).

Except for a couple of isolated cases, you have done pretty much the same quite a few times (10, 15 already?) before! Why?! You respond to one of my posts by saying that all/most of what I wrote is wrong and by repeating a set of ideas since the first moment until the end when I usually stop replying you?! Some times you seem to understand more or less fine some concepts and to have certain knowledge of certain aspects, but most of the times you are blindly repeating ideas which are partially or completely wrong and not even reading/understanding what I write?! It is like you having a conversation with yourself!! I am starting to have serious doubts regarding your background and motivations (to talk to me). Now, I will stop talking to you and will not be understanding in the future. The next time you will start your routine and show an even slightly weird-to-me behaviour, I would stop replying you right away and without saying anything else.

Re:Quite float-point-ish (= inacuratish)

[david_thornley]

I understand what you're saying. You're wrong about a lot of things.

Decimal floating-point isn't really different from binary floating-point. Having control over each digit buys you nothing once you start calculating. Calculations that work out exactly in decimal will be exact, but most won't. If you're working in decimal, try 1.0/3.0 + 1.0/3.0 + 1.0/3.0. If you get 1.0, then the computer is using hidden digits, which by definition you don't have control over, because with fixed numbers of decimal digits you get 0. followed by infinitely many 3s, and adding three 3s together gives you 9.

If you don't understand this, you know very little of floating-point, and you should learn more. I posted a cite of a classic article.

Re:Quite float-point-ish (= inacuratish)

[CustomSolvers2]

You're wrong about a lot of things.

Firstly, I am pretty sure that you have misunderstood most of my statements, you rely on quite a few absolute truths which are either completely wrong or inapplicable under quite a few scenarios and, as said, I have serious doubts regarding your background and motivations to talk to me. Secondly and even by ignoring the previous point, precisely the fact that you don't see this whole situation (= you considering that what I say is wrong, I am not considering that you know what you are talking about, but this going on anyway) as a clear indicative of this conversation not going anywhere seems a quite big problem.

Having control over each digit buys you nothing once you start calculating

This is precisely the whole point of having accurate results!! Each digit behaving as it should. You are still mixing up ideas!!! One thing is the inherent inaccuracy of incomplete decimal representations; a different story is the floating point problems which provoke digits to get wrong values!! For example, 1.23456789 being an imperfect representation of 1.23456789101112131415, but 1.23456723 being an error provoked by the floating point limitations!! Second time I repeat the same example and you continue proving that you haven't still got it!!!

1.0/3.0 + 1.0/3.0 + 1.0/3.0. If you get 1.0

Again!!! You are not understanding it!!!! You are showing the limitation of dealing with incomplete versions of decimal numbers (because the only way to perfectly represent 1/3 is by relying on its fractional form) and mixing it up with the difference between floating-point/non-floating numeric types. The most perfect decimal numerical type with the highest precision ever will not be able to represent 1/3 perfectly! You can have 0.3333333 or 0.33333333333333 or 0.333333333333333333333333333333333. All of them are imperfect and none of them have anything to do with the floating-point limitations!!! Something like 0.33333435 would be the typical floating point problem! Randomly wrong digits!!

I expressly said you that I don't want to continue this conversation and even highlighted my doubts regarding your behaviour; you have simply ignored all that and continued behaving in the same way!!! Why are you doing that?! You only have two options: either behaving in a compatible-with-me way (as explained quite a few times before) and talk to me or avoid dealing with me, but you choose the third, making-no-sense one. Why? Anyway, I will now seriously stop talking to you (now and in the future unless you change your behaviour completely). I haven't been able to refrain myself from answer you one last time because your example was sooooo descriptive of your misperception!!

Re:Quite float-point-ish (= inacuratish)

[david_thornley]

I see that the personal insults start because you apparently don't understand what I'm saying, which is bog-standard computer science.

I am not going to change my behavior to adhere to whatever twisted reality you've got. I will reply to you as I see fit, to try to point out things to others.

Re:Quite float-point-ish (= inacuratish)

[CustomSolvers2]

I see that the personal insults

?! I am honestly sharing with you my concerns. I am honest. Perhaps you aren't used to deal with honest people. I recommend you to give it a try, but please don't try it with me.

don't understand what I'm saying,

?! I am speechless. No idea what to answer here.

I am not going to change my behavior to adhere to whatever twisted reality you've got

?! Having a civil conversation with people with similar knowledge/expectations and/or trying to understand others properly and/or accepting that better avoiding chaotic nonsense is a twisted reality?! OK.

I will reply to you as I see fit, to try to point out things to others.

You are free to do whatever. I have only tried to help you understand how you could avoid what I only do under extreme circumstances: ignoring someone talking to me.

Interval arith.

[Anonymous Coward]

Idiocracy. This is really disquieting...

patent troll?

[Anonymous Coward]

Just another wannabe patent troll?

Re:patent troll?

[Anonymous Coward]

Probably got jealous of all the money winners in Las Vegas.

You can patent Math?

[gweihir]

The perversions of the US patent system are truly astounding.

Also sounds very much like they re-invented Interval Arithmetic, which was discovered originally around 1950 and has been available in numeric packages for a long time. And, to top it off, the title is lying: Interval Arithmetic does not give you an accurate representation. It just makes sure you always know the maximum error.

Pathetic.

Re:You can patent Math?

[Anonymous Coward]

>You can patent Math?

You cannot patent abstract math.

>The perversions of the US patent system are truly astounding.

No, it's just the stupidity of the people commenting on the US patent system that they know nothing about.

Re:You can patent Math?

[PPH]

No.

But this appears to be an implementation of something like Interval Arithmetic inside of a math co-processor. Previous implementations have been in math libraries.

Re:You can patent Math?

[Anonymous Coward]

Keep on stalking, boy. You just validate me.

Re:You can patent Math?

[Anonymous Coward]

The idea of interval arithmetic has been around for a long time. The real question is whether this patent describes a significantly better way to implement it.

Re:You can patent Math?

[Anonymous Coward]

Oh I see.

So when you move maths from software to hardware it becomes patentable.

Re:You can patent Math?

[Anonymous Coward]

http://www.freepatentsonline.com/crazy.html

That method of swinging on a swing (#6368227) patent is truly a novel and unique mechanism that should allow its creator to profit from it for the betterment of society. All the patent trolling that happens too... totally fine. Not perverting the point of patents at all.

The more I know of the system the more I tend to agree with that guy you inferred was stupid. I guess I'm stupid too, huh? Good job for me I hold myself to my own standards instead of those of fellow anonymous cowards otherwise that might hurt my feelings.

Re:You mean Intel and AMD? Who use nothing patente

[Anonymous Coward]

Right, neither Intel nor AMD would EVER license any patents for use in their processors (eyeroll).

Haha, he probably doesn't know Intel licensed x86 to Cyrix so they would not fall under monopoly regulations.

Large scale omissions

[khb]

Earlier prior art, https://en.m.wikipedia.org/wik...

Patents, implementations etc. Not that there isn’t room for innovation as well as popularization.

Standards efforts (IFIP, IEEE) are ongoing. Yearly conferences in reliable computing, so both the original article and most likely the patent itself gloss over engineer-decades (if not centuries) of work.

For twice the work, I can use ratios too.

[Anonymous Coward]

And get perfect numbers for everything but irrational numbers, for which there is no easy precise solution anyway.
Would probably be faster too, since the ratios could be two integers.

Re:For twice the work, I can use ratios too.

[TechyImmigrant]

And get perfect numbers for everything but irrational numbers, for which there is no easy precise solution anyway.
Would probably be faster too, since the ratios could be two integers.

Yes. I've used rational types in python. When you are doing arithmetic in the rationals, it's nice to use this and get precise answers.

The question is

[goombah99]

what does "MOUNT TAPE U1439 ON B3, NO RING" mean?

is the answer who gets the last slice of Pi?

Re:The question is

[HiThere]

What was in tape U1439? The rest of that makes sense.
O, yes, was it 7-track? 9-track? or 6250?

Re:The question is

[Anonymous Coward]

a lot of people have the unix Fortune as a sig. I have no idea why.

Re:The question is

[baegucb]

6250 Bits per inch was an IBM standard on 9 track tapes from 40 years ago. The 9th track was a parity bit iirc.

back to basics ?

[swell]

In the 1950s, when the public was first becoming aware of computers, computers were considered to be large calculators. They could do math. They could be used by the IRS to compute your taxes or by the military to analyze sensor inputs and guide missiles. Few people could envision a future where computers could manipulate strings, images, sounds and communicate in the many ways that we now enjoy.

But today we have all those unimaginable benefits but one: They can't really do math well. Oh, the irony!

Re:back to basics ?

[ClickOnThis]

That depends on what you mean by "do math." If you mean arithmetic and symbolic manipulation, then yes they can. In fact they're damn good at it.

But if you mean possessing a sentience with a motivation to seek out theorems and construct proofs for them, then not so much, but let's wait and see what AI brings us.

Re:back to basics ?

[postbigbang]

It would be an advance for "AI" to suss when it's appropriate to go down the rabbit hole, into the weeds, into a cpu-intensive series of loops when precision becomes important. The bounds of a series of linked algorithms coupled to its dataset, makes for an accurate day.

This also means unleashing AI, then making it do what all of us have had to do since the beginning of time: show proof. This is where software test is supposed to catch errors, where QA decides that the boundary of inputs deigns the precision of outputs, and most importantly: why.

I eagerly await that.... if only because most organizations are pretty slothful, and imprecision leads to problems in accuracy of results. They imprecision may never be revealed, or be important, or given the dependency on blossoming/mushrooming lambda, maybe trains crash, pipelines burst, surgery goes wrong, etc etc.

A boundary algorithm inside a CPU may be heaven-sent if only to tell imperfect coders that results could be squishy, or to confine the results to a probablistic domain.

Re:back to basics ?

[Anonymous Coward]

They can't really do math well?

They do exactly what they're told. Program it and it'll do whatever math you're capable of telling it to do.

If your language of choice doesn't support the mathematics that you want to do then you can write a library or a language that does. This article is about putting in to hardware a ... workaround... a method of improvement... that has been utilised for a long time in software. If you want accurate floats then you can get them. You need to tell the computer exactly how to handle it so you can get them how you want them... but how is that different from anything else you tell a computer to do?

Not good at math... foolish! They're not good at ANYTHING..

It's the people making them dance that have to be good.

Wrong Title

[Anonymous Coward]

There's a major difference between "solving floating point error" and "solving the detection of floating point error"

And the second one has been solved for quite some time.
The first one is highly dependent on how accurate your original data is, and is likely never to be solved because of that.

solved already

[Anonymous Coward]

It's called called continued fraction arithmetic.
0 error. Or at the very least only as much error as you allow.

Re:solved already

[HiThere]

You've got an answer to the problem this was trying to deal with, but what he actually did was just put bounds on the error. As others have said, this has been known for a long time.

The only thing that's been said which "sort of" justifies this patent is that this is the reduction of a known algorithm (interval arithmetic) to a hardware implementation. Why that's worth a patent I'm not sure, but at least, unlike copyrights, patents eventually expire.

Re:solved already

[Pseudonym]

The only thing that's been said which "sort of" justifies this patent is that this is the reduction of a known algorithm (interval arithmetic) to a hardware implementation.

If you read TFA, it's clear that there are at least two things which can reasonably be called "new": a compact representation of a value-plus-bounds (i.e. an interval), and some way to mark (analogous to quiet NaNs, if I'm reading it correctly) a calculation that may have violated its precision requirements.

Maybe if you're working in a safety-critical field it might help with some calculations, but I can't see how it would help with most numeric problems.

Re:solved already

[RhettLivingston]

From the patent:

Using current technology error can be reduced by increasing computation time and/or memory space. The present invention provides this error information within the inventive data structure with little impact on space and performance.

This expresses the key development better. This patent specifies a means by which this error tracking and warning mechanism can be added to floating point HW with little addition of hardware or performance penalty. The operation is not performed twice as other posters here have imagined.

The article's wording definitely misleads as this just provides a means of identifying precision violations efficiently. But, I can imagine that someone could then go further at the assembly level and automatically respond to the new flag by repeating the operation with a higher precision instruction or algorithm.

Re:solved already

[HiThere]

IIUC, the existing methods already had a way to mark when the bounds were violated. Compact, however, might well be new. And perhaps the way he marks that bounds have been violated is new...though I wouldn't bet on that...not given what I read in the summary.

For that matter, I'm not expert in the field, but it wouldn't surprise me if some existing library already used an analogous "compact representation". Probably not the same representation, as implementing stuff in hardware usually causes changes, but something close enough that no reasonable patent agency would consider it worth a patent. (That said, this is just me being cynical. As I said, I'm no expert in the field.)

Not solved

[chromaexcursion]

This is totally theoretical. There is no implementation on silicon.
Until someone does this on a chip it's meaningless, and as others have pointed out, the solution isn't new.

Re:Not solved

[Anonymous Coward]

You could say the SSE implements this in silicon. Or at least I made an interval arithmetic package several years ago (you can use it really well to see if a CSG object intersects with a Voxel).

You configure SSE to round up, and then you negate the lower limit and store the lower and upper limit in an SSE register, then you do the standard set of mathematical operations on them. Only for a few operations you need to make adjustments, but it is very efficient,

Interval arithmetic isn't remotely new

[Anonymous Coward]

It was well-known when IEEE 754 was developed in the early 80s, and is the reason that directed rounding modes are included in the standard (published 1985).

The big problem is that naÃve implementations grossly overestimate errors and thereby produce useless results.

Suppose you have a variable A with value x±e. The value of A-A is exactly zero, but a simplistic subtraction would compute a lower bound of (x-e)-(x+e), an upper bound of (x+e)-(x-e), and say the result is 0±2e. But if you have a different value B, not derived from A, which coincidentally happens to have the same value x, then 0±2e is the correct difference.

Maybe they have a new way to address this problem, but interval arithmetic is not new

Re:Interval arithmetic isn't remotely new

[Anonymous Coward]

Idiocracy. You better get accustomed.

Re:Interval arithmetic isn't remotely new

[Anonymous Coward]

Penis in butt. You better get accustomed.

can you say ...

[Anonymous Coward]

a new round of litigation by a new patent troll! or perhaps it's an old patent troll with a new toy!

"Representation of real numbers accurate"

[Chris Mattern]

"to the last digit." Cool. I'd like a representation of pi, please.

Re:"Representation of real numbers accurate"

[Anonymous Coward]

"to the last digit." Cool. I'd like a representation of pi, please.

8=========D

Re:"Representation of real numbers accurate"

[DCFusor]

Pi in what time-space environment. Time-space is only flat if there is zero mass energy around - all the way out to infinity. The official value of pi is wrong if there is any mass-energy warpage of time-space. There is no place in the universe where there is no such warping. It's a pretty math construct, but so is a flat space-time. Doesn't mean it exists. Yeah, I know, no one likes a smart-ass - especially if they're right.

Re:"Representation of real numbers accurate"

[ClickOnThis]

"to the last digit." Cool. I'd like a representation of pi, please.

8=========D

Wrong gender.

Re:"Representation of real numbers accurate"

[Anonymous Coward]

As a physicist... you've missed the point entirely. Sure, one of the many things Pi does is describe the radius of a circle relative to its diamter, but it shows up as a fundamental constant in all sort of functions that have nothing to do with geometries on space-time. Anything involving complex numbers is riddled with pi terms all over the place.

Re:"Representation of real numbers accurate"

[Pseudonym]

Please go back to undergraduate computer science and re-learn the difference between accuracy and precision.

"3" is a representation of pi accurate to the last digit. It's just very imprecise.

Perl - bigrat

[DCFusor]

There's this perl5 module that actually solved this exactly quite some time ago - no compromise at all, but as Damian Conway says, we suck at marketing - if you need a bigrat to solve your problem. Perl 6 (which should have a different name and many now call Rakudo) - has this as a built-in. Rational numbers are kept exact all the way through calculations, period, if built from...rational numbers in the first place (won't help with pi or sqrt 2 of course). One of the best conference presentations ever given about anything happens to cover this - skip past the boring intros. Computer scientists will get the dry humor in say "the Alan Machine" reference. https://www.youtube.com/watch?...

Re:Perl - bigrat

[david_thornley]

The set of rationals is closed under ordinary arithmetic operations. It isn't closed if you add square roots or trigonometric functions. Such rational arithmetic was required in the Common Lisp standard of about 1994, and present in implementations going a good deal further back, long before Perl 6.

Re:Perl - bigrat

[DCFusor]

Agreed. Just saying something I know about myself solved this long ago (it's an add-on in perl5, builtin in 6). Doesn't surprise me if Lisp had it too - even slower and more memory intensive than perl perhaps. My point was, this "news" didn't even solve it...and if it did, it'd be "old news" as you seem to agree.

We have this

[Anonymous Coward]

It's called fixed point and / or BCD.

Re:We have this

[Anonymous Coward]

It’s also called me jackhammering your tight, white pooper.

Submitter didn't RTFA

[Kremmy]

The original article has a comment by someone who developed the method and released it openly.

So, they've patented interval arithmetics?

[Lisandro]

From what little i've read it certainly seems to be the case.

For those too lazy to view the patent...

[RhettLivingston]

He specifies the hardware algorithms that can utilize this new representation directly, track the error amount and flag violations with little addition of work. This is not just a representation patent. It is a patent on the algorithms to utilize the representation efficiently.

This is both (a) old and (b) simply wrong

[Terje Mathisen]

(a) The first example of this (decades before Gustafsson's unums) is interval arithmetic.

(b) A trivial counter-example is any kind of Newton-Rapson iteration to calculate the value of a given function: Even though the NR iteration for sqrt(x) or sqrt(1/x) both have quadratic convergence, i.e. the number of correct digits double on each iteration, any kind of interval arithmetic will tell you that the error instead grows to infinite levels.

I.e. the claimed benefits are totally bogus except for really trivial calculations, and for those the error bounds are equally trivial.

Full disclosure: I am currently a member of the 2018 ieee754 floating point revision working group, so you could claim that I am biased here due to having spent a year or two working in the subject area.

Terje

Accurracy!=precision

[plopez]

People tend to forget that.

For 10x the work, I use assembly!

[EETech1]

We used to use a lot of fixed point math that you could configure to get the range and resolution needed.
It's amazing how well optimized these systems were. Much of the expensive math and rational number problems were invisible because of the internal data representations being carefully planned.

There was a dll generated at build time that provided information to properly convert the internal representation to engineering units. It was very interesting doing the conversions
Divide 63,000,000 by foo, then add bar times 0.015625 and divide the result by pi and you get RPM!

Re:Accuracy!=precision

[Anonymous Coward]

Apparently people also forget this when they spell the word accuracy. X^D

This sounds a lot less innovative than claimed

[mysidia]

The innovative bounded floating point system computes two limits (or bounds) that contain the represented real number. These bounds are carried through successive calculations. When the calculated result is no longer sufficiently accurate the result is so marked, as are all further calculations made using that value.

This sounds like interval arithmetic.. A.K.A. using a vectorized format for numbers that carries forward an upper and lower bound through each operation, and I wrote a Matlab datatype for that in college, but others are published in the subject already, so this is not an innovation.

I would also point out that including two limit bounds increases the size storage costs, and the complexity/number of basic elements that will be required for computation.

APK just likes showing he is a retard

[Anonymous Coward]

Alexander Peter Kowalski just likes to show the world he is a retard.
That is why he posts stuff like that just to make sure any new readers know what kind of stupid he has to offer.
He also frequently doesn't claim his work so that it won't be directly associated with him.
It doesn't change the fact that Alexander Peter Kowalski is a retard.

Re:You mean Intel and AMD? Who use nothing patente

[martinfb]

Don't forget the ARM CPUs out there!

Comment

[WallyL]

Yes, but does it also perform speculative execution? If so, do not want!

Not for heavy-duty floating point

[raymorris]

ARM isn't used for serious floating point work. Very much the wrong tool for the job.

Rational?

[Doctrinsograce]

Irrational numbers always bothered me anyway. Now, if we could get rid of imaginary numbers, that would really be something!

Ministry

[JThundley]

Damn, I didn't know the guy from Ministry was into computer science!