IBM Claims Breakthrough Energy-Efficient Algorithm

jitendraharlalka sends news of a claimed algorithmic breakthrough by IBM, though from the scant technical detail provided it's hard to tell exactly how important the development might be. IBM apparently presented its results yesterday at the Society for Industrial and Applied Mathematics conference in Seattle. The breathless press release begins: "IBM Research today unveiled a breakthrough method based on a mathematical algorithm that reduces the computational complexity, costs, and energy usage for analyzing the quality of massive amounts of data by two orders of magnitude. This new method will greatly help enterprises extract and use the data more quickly and efficiently to develop more accurate and predictive models. In a record-breaking experiment, IBM researchers used the fourth most powerful supercomputer in the world... to validate nine terabytes of data... in less than 20 minutes, without compromising accuracy. Ordinarily, using the same system, this would take more than a day. Additionally, the process used just one percent of the energy that would typically be required."

Comments

Wat

[Anonymous Coward]

reduces the computational complexity, costs, and energy usage for analyzing the quality of massive amounts of data by two orders of magnitude.

I guess they stopped using Windows Vista?

just trying to be relevant

[pydev]

Sounds like someone found a faster algorithm (maybe just constants), and since energy efficiency is the hot new thing, "faster" is now translated into "saves energy".

Re:just trying to be relevant

[Dunbal]

I find it interesting on a philosophical level to think about what computing is doing to us. CPU's require energy to perform calculations. Then there's the system overhead, a fixed energy cost that included the assembly and set up costs, and the running and maintenance/replacement costs. Now obviously humans have been almost taken out of the equation. Where before you had thousands of workers all requiring to be fed, all requiring furniture and space and light and reasonably cool/warm air, all of them needing transport, and all of them victims of entropy and therefore needing accident and health insurance, taking sick days, etc. We've come a long way.

Now you just need the brains. Brains to design the system, brains to drive the investigation, and brains to try to improve the algorithms the system uses. To save even more energy. Of course eventually physical limits will be reached. There's no escaping the fundamental laws of our universe. But the energy "savings" from doing it the "old way" is translated into the ability to essentially brute-force the universe with raw computing power. Er, but what are we going to do with all the people who just don't "have" the brains? They get a free ride?

Sorry I'm waxing philosophical today.

Re:just trying to be relevant

[biryokumaru]

Er, but what are we going to do with all the people who just don't "have" the brains? They get a free ride?

That seems to be the way it's been working so far.

Re:just trying to be relevant

[zippthorne]

In a post scarcity economy? Yeah, everyone gets a free ride. Everything changes if you can get to that.

Re:just trying to be relevant

[pydev]

Did you actually read the article?

Well, that's hard to do since there was no reference. But the guy seems to be talking about "Massively Parallel Low Cost Uncertainty Quantification". This is probably the same work as this:

http://portal.acm.org/citation.cfm?id=1645413.1645421

The work has nothing to do with energy savings, it's just about a fast, approximate algorithm for a fairly common operation.

Common sense tells you that spending 20 minutes to do something takes less energy than taking up to a day doing the same thing.

My point exactly. The whole press release is analogous to saying that you save a lot of energy by compiling with "-O0" instead of "-O4".

Impressive but

[QX-Mat]

Can it organise my porn?

Awesome!

[daceaser]

I'll buy three!

What do they do exactly?

Clarification?

[Twinbee]

Can someone please clarify exactly what they've achieved here? All I hear is that they can somehow sift through large quantities of data much quicker. What kind of data? What are they trying to extract? And for what end?

Re:Clarification?

[Xest]

"What kind of data? What are they trying to extract? And for what end?"

The web. Porn. Fun.

In that order.

Re:Clarification?

[godrik]

The conference proceedings are not online yet. So I am not sure. I could not even find the title of the talk on the conference web page

I know people who are at SIAM PP and they are all : "why are they talking about PP on slashdot ?". There was no major anouncement. I'll check the proceedings again next week, but I believe there is no major improvement. IBM is probably just trying to get some more light.

We can find the following IBM talks in yesterday page :
http://meetings.siam.org/sess/dsp_programsess.cfm?SESSIONCODE=9507
The paper have the same author and name than this paper published last year :
http://portal.acm.org/citation.cfm?id=1645413.1645421

So they are probable publishing an improvement on their 2009 work.

TFA is worthless

[Anonymous Coward]

This would be a real story if it gave implementation details, but it doesn't even tell us what the algorithm does; therefore it's totally worthless. Get this crap off the front page.

Empty statements

[qmaqdk]

...for analyzing the quality of massive amounts of data...

I have an algorithm that does that in O(1):

return "Not the best quality, but pretty good.";

Re:Green-washing

[gandhi_2]

With faster algorithms, the machine can just get more jobs done in the same amount of time. But the jobs will just keep coming, so the energy use never changes.

Or are the new algorithms SO fast that all processing needs of humanity will be done in a week, thereby allowing us to turn off all supercomputers? Now that would save energy.

Re:I'd expect this

[momerath2003]

The funny thing about energy efficiency is that it saves companies money, but they get to spin it as being "green." [For example, when grocery stores eliminate plastic bags to be "green," what they really mean is they're eliminating bags to be "cheap."] If this new algorithm has no penalty associated with it, then it saves time and energy, therefore money and "the environment."

Zombie Computer Says:

[WrongSizeGlass]

Now you just need the brains. Brains to design the system, brains to drive the investigation, and brains to try to improve the algorithms the system uses. ... Er, but what are we going to do with all the people who just don't "have" the brains?

Mmmm, brains ...

Re:This is a pretty good energy-saving algo...

[biryokumaru]

Running this once should reduce your PC's energy consumption to near zoro:

The only problem is, with all that jumping around and swordplay, Zorro ends up using tons of energy.

Re:Green-washing

[biryokumaru]

The fans on servers have variable speed. Case closed.

I believe that you will find those fans can still run at variable speeds with the case open.

What?

A more informative link

[jbuhler]

Here's a link with actual content on what the algorithm does:

http://www.hpcwire.com/features/IBM-Invents-Short-Cut-to-Assessing-Data-Quality-85427987.html

Here's the actual paper.

[mattdm]

"Low cost high performance uncertainty quantification", full text available in PDF.

http://portal.acm.org/citation.cfm?id=1645421&coll=GUIDE&dl=GUIDE&CFID=77531079&CFTOKEN=42017699&ret=1#Fulltext

And, here's the abstract:

Uncertainty quantification in risk analysis has become a key
application. In this context, computing the diagonal of in-
verse covariance matrices is of paramount importance. Stan-
dard techniques, that employ matrix factorizations, incur a
cubic cost which quickly becomes intractable with the cur-
rent explosion of data sizes. In this work we reduce this
complexity to quadratic with the synergy of two algorithms
that gracefully complement each other and lead to a radi-
cally different approach. First, we turned to stochastic esti-
mation of the diagonal. This allowed us to cast the problem
as a linear system with a relatively small number of multiple
right hand sides. Second, for this linear system we developed
a novel, mixed precision, iterative refinement scheme, which
uses iterative solvers instead of matrix factorizations. We
demonstrate that the new framework not only achieves the
much needed quadratic cost but in addition offers excellent
opportunities for scaling at massively parallel environments.
We based our implementation on BLAS 3 kernels that en-
sure very high processor performance. We achieved a peak
performance of 730 TFlops on 72 BG/P racks, with a sus-
tained performance 73% of theoretical peak. We stress that
the techniques presented in this work are quite general and
applicable to several other important applications.