Slashdot Mirror
Harvesting & Reusing Idle Computer Cycles
Hustler writes "More on the University of Texas grid project's mission to integrate numerous, diverse resources into a comprehensive campus cyber-infrastructure for research and education. This article examines the idea of harvesting unused cycles from compute resources to provide this aggregate power for compute-intensive work."
Google's desktop search is one example where the timing and recovery back to the user is really done well.
__
Laugh daily funny adult videos
REusing idle cycles? Really?
brwski
"Because without beer, things do not seem to go as well''
Yeah, but it still draws a lot less letting a some computers burn some cycles, than you would have to use if you built a shiny, new, cluster. And you don't have to pay for the hardware either, because you already have it...
I have a really elegant proof for Fermat's last theorem. If this sig was only a bit longer...
This is a very insightful post, but has two crucial counterarguments
- Does anyone realize the cost of buying extra computers to handle peak computing loads?
- Does anyone realize the cost of idle high-tech, high-paid labor while they wait for something to run?
The proper decision would balance these three (and other factors) in defining a portfolio of computing assets that can cost-effectively handle both baseline and peak computing loads. Idle CPUs aren't free, but then neither are idle people or surplus (turned-off) machines.Two wrongs don't make a right, but three lefts do.
That's a very valid point, we should not assume that this usage comes at no cost to the environment. However, the cost of building and running a separate CPU dedicated to the same purpose is even higher - twice the hardware infrastructure (motherboards, cases, power supplies, what else? monitors, gfx cards, etc.), twice the number of cycles wasted loading software infrastructure (OS, drivers, frameworks eg. Java/Mono). Add to that the fact that hardware is not easily recycled and the "green" part of me suggests that cycle-sharing is a better idea than separate boxes.
The next question is - who pays for the electricity then? University departments are notorious for sqabbling over who picks up the tab for a shared resource - and that's not even considering the wider inclusion of home users...
Does anyone realize that running a CPU at 100% takes more electricity than running a CPU at 10%?
Yes, I do, the same for RAM being accessed and for a hard disk drive when it's seeking. But this is insignificant compared to the overhead of the power supply, fans, hard disk drive spindle motors, other circuitry that runs continuously, and dare I mention all those fancy-dancy computer case lights that are popular now.
The incremental cost of these otherwise-unused cycles is so low that they can be considered free.
So someone prove me wrong, what's the electricity cost of running a CPU at full cycles for a year vs. running at typical load? What's the cost of the lowered processor life due to running at a higher temperature. Chip makers will tell you this is a real cost, but practically, the machine is likely to be replaced with the next generation before the processor has a heat-related problem.
Regardless, the cost is MUCH lower, in both electricity and capital, than buying other machines specifically to do the work assigned to these 'free cycles'.
Tag lost or not installed.
What you are saying was perfectly correct even 3 years or so ago.
But case in point: My Athlon64 computer doubles its wallplug powerdraw (including everything:PSU, Mainboard, HD, ect) at 100% load compared to idle desktop (ok, cool%quite helps pushing idle power down).
The cpu IS the biggest chunck besides some high-end GPUs (and even those need MUCH less power when idle), and modern cpus need 3-4 times as much power under full load compared to idle.
HI O WISE PRINCE. WHT TOOK U SO DAM LONG?
PVM offers both the spec and the implementation, MPI offers a newer spec with several solid implementations. But no, NIH-syndrom prevails and another piece of half-baked software is born.
Where I work, the monstrosity uses Java RMI to pass the input data and computation results around -- encapsulated in XML, no less...
It is very hard to fight -- I did a comparision implementing the same task in PVM and in our own software. Depending on the weight of the individual computation being distributed, PVM was from 10 to 300% faster and used 5 times less bandwidth. Upper management saw the white paper...
Guess, what we continue to develop and push to our clients?
In Soviet Washington the swamp drains you.
Good point. Having to rewrite the application to make use of a parallel MPI can be a pain. Condor is a free full-featured batch system that allows you to run apps on remote machines without having to recompile them
Ya, let's countries such as China and N. Korea have such access to free engineering. After all, we want oppressive regimes to have as much power over their own citizens. I mean, when was the last time YOU could fly your own jet? Such gaps between non-democratic governments and it's citizens make much-needed revolutions that much harder to achive.
Life is not for the lazy.
What all of you working from the electricity cost issue are missing is that at most universities, money for capital is different than money for operations. Capital money is hard to get. An increase in your operations cost just kind of get ignored if they're not too big.
This has political ramifications.
The goal: get a great, powerful, cluster of compute power.
You can't go to the administration and say, "We need to spend $150k on a compute cluster". The answer will be "we don't have one now, and everything's just fine. No."
So, you, being resourceful, implement this campus-wide cluster system that taps spare resources. Power bills go up a bit - nobody cares.
Now, a couple years later, lots of projects are using the cluster. But the thing isn't working well because the power's not there during normal peak usage.
At his point you go the administration, "we're losing tuition-paying students, and several grants are at risk because our compute cluster is not powerful enough. We need to spend $250k on a new compute cluster.
And THAT is how you manipulate your operations budget to augment your capital budget.
So what makes the wikipedia the be all and end all of information? A collection of user contributed information. I would hardly use it as proof in an argument.
Suffice to say however AMD calculates it's PR rating really doesn't change the fact that it's to provide a comparison between Athlons and P4. I can guarrantee you if intel released a P4 processor that changed that correlation, AMD would change their PR rating on new processor to match it. Of course now that Intel itself is going to a PR rating of sorts that all changes.
--
Free Credit Report
What does your Credit Report look like?
But it isn't that expensive. Let's call it 100W extra. 24 hours in a day gives us 2.4 kWh per day. For a year, call it 876 kWh. Approximate cost of electricity (in Texas) is about 10 cents per kilowatt-hour. That's $87.50 per year. Let's assume that you can extract half of the computer's horse-power for your cluster (the rest is lost in overhead of the cluster software etc, and of course, whatever the actual user of the PC does, which is often just word processing, email, and surfing the web). For an extra ~$175 per year you get the equivelant of another computer.
If you wanted to get that computing power in a stand alone system, you'd not only have to purchase the PC (up front capital), but you'd have to pay more for electricity. From the reference link, only about 30% of a computer's power is used by the CPU, the rest is doing nothin'. The computers referenced, at full bore use 185W (best case). That's $162 per year at my 10 cent per kilowatt hour quote. Cheaper, sure, but by the cost of a computer? Not even close.
Of course, there are other (hidden) costs involved in both methods, of which I'm not including in my (overly?) simplified model. And I'll just brush under the rug the fact that this kinda assumes that the average secretary has a top of the line system to surf the web with.
--Xandu
The Wisconsin Condor Project has been harvesting unused compute cycles for over a decade. The software is free to use and deploy, and is used by various corporations including Western Digital and others.
You mean like a thermostat?
This isn't just about parallel computing - in fact if you'll read the article you'll see that they're using MPI for handling parallelism! Grid computing isn't about reinventing inter-node communications - it's more about inventing inter-node scheduling.
Your cluster - is it so fast that you're never stuck waiting for jobs to finish? If not, then you could probably benefit from being able to borrow time on someone's larger system. Is your cluster so well-utilized that the load's always around 1? If not then you've probably got spare capacity that someone else could benefit from. The fact that both you and those others are using MPI is necessary but insufficient to allow you to cooperate.