Slashdot Mirror
IBM Claims Breakthrough Energy-Efficient Algorithm
jitendraharlalka sends news of a claimed algorithmic breakthrough by IBM, though from the scant technical detail provided it's hard to tell exactly how important the development might be. IBM apparently presented its results yesterday at the Society for Industrial and Applied Mathematics conference in Seattle. The breathless press release begins: "IBM Research today unveiled a breakthrough method based on a mathematical algorithm that reduces the computational complexity, costs, and energy usage for analyzing the quality of massive amounts of data by two orders of magnitude. This new method will greatly help enterprises extract and use the data more quickly and efficiently to develop more accurate and predictive models. In a record-breaking experiment, IBM researchers used the fourth most powerful supercomputer in the world... to validate nine terabytes of data... in less than 20 minutes, without compromising accuracy. Ordinarily, using the same system, this would take more than a day. Additionally, the process used just one percent of the energy that would typically be required."
And that color would be blue! Hopefully us mere mortals will be able to benefit from such algorihtms.
I tried to think of a good sig, and this wasn't it.
I guess they stopped using Windows Vista?
After we moved from MySQL to PostgreSQL, we saw similar performance improvements. Then we doubled our performance again when we moved to FreeBSD from Linux. We never expected a few software changes to have such a big impact, but were happy that we could reuse all of our existing hardware.
Sounds like someone found a faster algorithm (maybe just constants), and since energy efficiency is the hot new thing, "faster" is now translated into "saves energy".
Can it organise my porn?
I'm all for energy-efficient algorithms and datacenter but this PR is nothing but green-washing. IBM's algorithm is just faster so it uses less energy. Duh.
Automatically spreading loads across datacenters in multiple locations to take advantage of local environmental conditions so you don't have to use chillers, now that's something.
Nobox: Only simple products.
I'll buy three!
What do they do exactly?
-- There are three kinds of mathematicians: those who can add and those who can't.
Can someone please clarify exactly what they've achieved here? All I hear is that they can somehow sift through large quantities of data much quicker. What kind of data? What are they trying to extract? And for what end?
Why OpalCalc is the best Windows calc
No explanation of how these two orders of magnitude of improvement were achieved. Less space than a Nomad. Lame.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
So what kind of porn was that, and why is this of interest to the enterprise?
If you're running the fourth most powerful supercomputer in the world you are not saving energy. Period.
Get this type of 'efficiency' running on mobile & portable devices and laptops first. Then you can claim some sort of energy related victory.
This would be a real story if it gave implementation details, but it doesn't even tell us what the algorithm does; therefore it's totally worthless. Get this crap off the front page.
Without more information, this really sounds like they just had a horribly-slow-but-at-least-it-works algorithm in the first place and now done some work on making it more efficient. They don't even say what type of processing was being done on the data..
which is totally what she said
...for analyzing the quality of massive amounts of data...
I have an algorithm that does that in O(1):
return "Not the best quality, but pretty good.";
My UID is prime. Hah!
The funny thing about energy efficiency is that it saves companies money, but they get to spin it as being "green." [For example, when grocery stores eliminate plastic bags to be "green," what they really mean is they're eliminating bags to be "cheap."] If this new algorithm has no penalty associated with it, then it saves time and energy, therefore money and "the environment."
I had but a simple dream, to destroy all humans.
Now you just need the brains. Brains to design the system, brains to drive the investigation, and brains to try to improve the algorithms the system uses. ... Er, but what are we going to do with all the people who just don't "have" the brains?
Mmmm, brains ...
just in case you don't know how marketing looks like. Until there is a technical paper from IBM we could just assume that someone said "I have an idea! Lets use quicksort instead of bubble sort!".
and Business Inteligence software. Things that large corps use to help make decisions (Goldman-Sachs?) and manipulate the banks/markets even faster today so Yea! This is a big deal to corps. Not so big a deal to individuals other then the damn corps can make idiot decisions even faster now.
Mod me up/Mod me down: I wont frown as I've no crown
Implemented for Linux, but analogously applicable to other systems. Running this once should reduce your PC's energy consumption to near zoro:
#!/bin/bash
#
# save-energy.sh
#
# Save enormous amounts of energy, irrespective of cost in lost computation
# must be run as root
Of course, effeciency will be lost if you do anything else with your PC (like turn it back on), but hey, no algorithm is perfect for all use cases.
The Future of Human Evolution: Autonomy
On their mainframes, IBM still charges for 'MIPS', which is processor usage ... and they charge rather a lot. This could potentially cost them a lot of money, although it's unlikely the efficiency can be applied to the sort of simple business transactions that are typically done on that sort of hardware.
Is there hope for a merge of all virtual fragmented universes into a single universe? We we can explore strange new worlds; seek out new life and new civilizations; boldly go where our avatar hasn't gone before?
Build your own energy sources from scratch. http://otherpower.com/
that would be my own word smithing
regularly produce this magnitude of algorithm speedup...
The MAFIAA is a bunch of mindless jerks who will be the first up against the wall when the revolution comes
As a computer engineer, I'm fascinated by the potential improvements in performance.
As a wired citizen, I'm terrified of the additional data-mining capabilities this will provide to our corporate overlords.
We are the 198 proof..
./configure
What was the algorithm? For all I know (having not read TFA), it could be that they replaced bubble sort with quicksort.
Here's a link with actual content on what the algorithm does:
http://www.hpcwire.com/features/IBM-Invents-Short-Cut-to-Assessing-Data-Quality-85427987.html
AH you are not seeing the real potential of this.
It took them 20 mins to do. Meaning they can do 70 more customers in one day. Charge 10% more for the same job. Poof 70x1.1xcost gross profit. Oh and it cost them 99% less power wise. Margin just went up by 99%. Meaning they can also undercut competitors in the field. Or resize the computer so it still takes a day and still sell by the same price point. So it just costs them 99% less to do powerwise and customers pay the same amount.
Wont cost them a thing. In fact I would be willing to bet they make even more. Wait until the MBA's are done spinning it.
For their cloud computing stuff, absolutely, but customers typically 'own' their own mainframes.
I think he gets his insults from moveon.org.
You're talking about a company that builds supercomputers to play chess. They're hardly greenards.
They are, however, moving to greener pastures in response to the demands of the marketplace.
That sounds smart to me.
"Lame" - Galaxar
are they marketing the price of the new algorithm? Peak oil trumps software. If they open source it...never mind.
"Low cost high performance uncertainty quantification", full text available in PDF.
http://portal.acm.org/citation.cfm?id=1645421&coll=GUIDE&dl=GUIDE&CFID=77531079&CFTOKEN=42017699&ret=1#Fulltext
And, here's the abstract:
Uncertainty quantification in risk analysis has become a key
application. In this context, computing the diagonal of in-
verse covariance matrices is of paramount importance. Stan-
dard techniques, that employ matrix factorizations, incur a
cubic cost which quickly becomes intractable with the cur-
rent explosion of data sizes. In this work we reduce this
complexity to quadratic with the synergy of two algorithms
that gracefully complement each other and lead to a radi-
cally different approach. First, we turned to stochastic esti-
mation of the diagonal. This allowed us to cast the problem
as a linear system with a relatively small number of multiple
right hand sides. Second, for this linear system we developed
a novel, mixed precision, iterative refinement scheme, which
uses iterative solvers instead of matrix factorizations. We
demonstrate that the new framework not only achieves the
much needed quadratic cost but in addition offers excellent
opportunities for scaling at massively parallel environments.
We based our implementation on BLAS 3 kernels that en-
sure very high processor performance. We achieved a peak
performance of 730 TFlops on 72 BG/P racks, with a sus-
tained performance 73% of theoretical peak. We stress that
the techniques presented in this work are quite general and
applicable to several other important applications.
Not to nitpick, but plastic bags are a lot cheaper than paper bags.
Grocery stores often stop using plastic bags because they're banned on a city or township level.
You can't legislate goodness. Let each to his own destiny, by will of his freely made choices.
which means now i need one or more less to get the same job done - OR i don't need to add one for an increased work load or i could power extras down for later days or or sell them to another group.
any time you can make existing equipment more efficient the gains are wonderful because you have already made the investment in buying and maintaining the hardware
Total cost of ownership for equipment is far more than the initial cost to purchase it - and it is a cost i'm going to pay - so if i can get more out of it for that cost - that is a benefit
'...if only "Jumping to a Conclusion" was an event in the Olympics.'
20 minutes is roughly 1% of "More than a day",
so it's not only using "1% of the energy required",
it's just using 1% of the time required,
so this is not a breakthrough of energy efficiency, it's a "CPU time" breakthrough, LOL
I don't mean replace with paper, I mean get rid of bags altogether. Presently, you can see several instances of companies eliminating some service in the name of "green", and the fact that it will save them millions of dollars by shorting their customers is just a convenient benefit.
I had but a simple dream, to destroy all humans.
Burn 'em.
Obviously are not releasing details until the Patent application goes through and the Patent Troll company set up. They certainly would not release the information so other people could just steal their idea. Maybe they will package it in sealed application and rent it out. Hmmmm anyone remember the Chess Playing Mechanical Turk? (1770).
From TFS:
"reduces the computational complexity[...]by two orders of magnitude[...]
Additionally, the process used just one percent of the energy that would typically be required"
Well, duh, what's so shocking about a computation taking 1% of the time previously needed now only takes 1% of the energy as well?
... and that was exactly my point. This will cost IBM money ... it will save customers money.
So, you'd rather them not be Green and waste money? And by "Penalty" you mean Government interference?
People like this have no idea of the consequences of their stupid insane policies. Here in California, we're losing Dairy Farms at an alarming rate, because of so-called "penalties" to raising cows.
Even Dairies that have tried to comply and work with the "Greenies" are folding, because the pound of flesh they demand never satiates them. There is always MORE that "needs" to be done. Green is never Green enough.
Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
...and it automatically fired three project managers.
It will make money for IBM if potential customers look at it and realize that they too, will save money by buying IBM.
The description of this "new algorithm" is pretty sparse.
Any word on if it allows faster solutions to encryption problems so that we now all need longer passwords?
I shouldn't be telling anyone this but I found the algorithm. Here it is in Java, please convert it to your preferred language.
public int computeData(Data data) { return 42; }
Oh crap, I forgot to post this anonymously. Why are there mice all over the pl@#M *^&I RCS$WE^%
[CARRIER LOST]
Well, there's spam egg sausage and spam, that's not got much spam in it.
Um, fundamentally, green IS cheaper, provided you properly account for ALL costs, negative externalities included.
The only reason not-green seems cheaper is when there's no charge for dumping heavy metals in the rivers, etc.
Smart brains ...
Well you could be up here in Canada, where my wife and I are paying about $90 a month for our health care insurance. That would be horrible wouldn't it? Living under an inefficient Socialist healthcare system and saving all that money while getting very good healthcare services?
Now if my income was lower, I would pay less. If it fell below a threshhold I can't recall, it would be free. If I earn more my payments will go up but not onerously so.
For some examples:
1) My mother died of cancer 2 years ago. She received full health care, hospital time, radiation treatment etc. Total cost: $50 for the ambulance that took her to the hospital the last time she went.
2) A friend of mine was diagnosed with a brain tumour. He was in and out of the hospital inside of a few weeks, perfectly healthy after the tumour was removed. Total cost to him: nothing beyond his payments.
3) I am 50 years old, I have been to the doctors all my life. I have never had to pay a dime for a visit. As a result, when I am sick I can go to the doctors, not worry about whether I can afford to find out if its something serious etc.
I know we have some problems from time to time up here, we get just as horrified by mistakes in the system as the anti-healthcare folks who post all those stories do down in the US when they are denegrating our system up here. However, overall it works very well.
Oh its worth pointing out that anyone who wanted different health care, say if my friend wanted treatment for his brain tumour down in the US, is capable of getting that treatment instead - they just have to pay for it.
The system benefits those at the bottom of the economic scale, it doesn't penalize those at the top of the scale
Nothing new, probably just using caching and a makefile-like dependency-checking algorithm.
Kind like you can speed up the calculations in the game of life if you calculate the sums of 3 adjacent cells over x first, and calculate the sums over y of the previous sums. So you use 4 addition operations instead of 8 (and some array indexing operations which I'm too lazy to calculate, probably 6 instead of 8).
Really sad that they will probably patent it so no one else can use it for 20 years.
Hey don't blame me, IANAB
If going green means that I never have another cheap plastic bag fall apart in my hands, so much the better. I love the $0.99 tougher bags branded with the store's name; I have a set of 6 that has lasted two years. It's probably ended up being cheaper than the cost of all those plastic bags and they are way more reliable and reusable.
Random Thoughts From A Diseased Mind (Not For Dummies)
> The only reason not-green .., etc.
Well, the 'etc.' implies more than one, as is, indeed, the case.
I'd submit that using energy up-front is easily overlooked; look at all the extra technology in a hybrid - did it cost nothing to research and does it cost nothing to manufacture? We are told that the net result is to save energy, but I've never seen a complete breakdown of the energy costs of producing hybrids. I'm not saying it doesn't exist, just that I haven't seen it, and I'd like to.
*Still* negative function...
"Er, but what are we going to do with all the people who just don't "have" the brains? They get a free ride?"
This question was already answered by Saturday Morning Breakfast Cereal:
http://www.smbc-comics.com/index.php?db=comics&id=1760#comic
They will patent the algorithm, include it in their software packages, then sell the software as a premium upgrade, offsetting the loss in sales.
OR they will patent it and sit on it.
That sounds nice but let's focus cost. If you're only paying $90/month, there's no way you are paying the full cost of health care. According to Wikipedia, the Canadian government pays about 70% of health care, so your $90 represents about $300 in actual cost. (Still a lot better than the US.)
According to Wikipedia, Canada has 6% of the population above 65 years of age. The US has 12.8%.
Eh, what's the point. It's very difficult to compare costs between different countries. I don't even know whether your $90 is converted to US dollars or still in Canadian dollars (hey 5% makes a difference).
I know the US has high health care costs, but the important question is why. People like you don't even pretend to analyze the situation, you just think it's "the system" that makes it cheaper. Give me some facts if you have them! I look for them but it's hard for one person to become an expert, or even just become knowledgeable, about the health care systems of a dozen different countries.
The price of the vehical should factor in the energy costs to create it, otherwise, a lot of people are losing a lot of money.
Low Cost High Performance Uncertainty Quantification
http://portal.acm.org/citation.cfm?id=1645421&jmp=cit&coll=portal&dl=ACM&CFID=62671076&CFTOKEN=92670385#CIT
Keywords:
Inverse Covariance Matrices, Stochastic Estimation, Iterative
Renement, Iterative Solvers, Quadratic Cost, Massive
Parallelism
All rites reversed 2010
You managed to optimize your SQL query.
When I optimize my queries, why don't I get press releases?!
I'm not an expert on this subject but I wonder.
What pollutes more? 100 oxo-biodegradable plastic bags or 1 non-biodegradable vinyl bag?
Also, when you use the same bag over and over again and use it to carry food, how sanitary is that?
"in less than 20 minutes, without compromising accuracy. Ordinarily, using the same system, this would take more than a day"
This just means that they were doing a pretty damn lousy job before they fixed the problem.
The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.
You're right that spinning this as a green result is trendy. But it can't be just a change in constants, since they say they reduced the complexity. "Reducing the complexity" is a technical term that means the speed, as a function of input size n (or l,m,n, etc.), keeps improving as n gets bigger. That is,
lim time_required_new(n) / time_required_old(n) = 0.
n --> infinity
An example is that an O(n log n) algorithm like Heapsort has reduced complexity compared to an O(n**2) algorithm like selection sort. But you can't legitimately claim to have "reduced the complexity" just by cleaning up the code or cutting out a constant factor of work (like going from bubble sort to selection sort).
$META_SIG_JOKE
For anyone who's interested in what these guys have done- the WHPCF'09 paper by Bekas and Fedulova (and going back a bit further, their 2007 paper by Bekas et al.) give the details.
In many statistical problems we end up with the problem of finding the diagonal entries of the inverse of a known symmetric and positive definite matrix A. For example, in linear regression the variances for the fitted parameters are found on the diagonal of inv(X'*X). When this matrix A is very large, the computation can be very expensive, since it requires O(N^3) time by conventional methods (Compute the Cholesky factorization of A and then use the Cholesky factors to solve for N right hand sides.)
Bekas et al. have developed a Monte Carlo approach that can give good (e.g. 2-3 digits of accuracy) estimates of the diagonal entries in inv(A) by using an interative method to approximately solve systems of linear equations involving A. The approximate iterative solutions take roughly O(N^2) time, and there are s of these systems to solve, where sN. Thus the computational complexity is lowered from O(N^3) to roughly O(N^2). Furthermore, you can solve these s systems of equations in parallel. Going one step further, you can do a lot of the computation in single precision, so it can be done on GPGPU's and other machines that don't do double precision floating point efficiently.
funny ... i had exactly the same dream ...
beware he who denies you access to information for in his mind, he already deems himself to be your master (SMAC-ish)
So imagine if you had a Beowulf cluster of these, they could like, self-power forever?
This likely involves cats strapped to buttered bread, for maximum effect.