Slashdot Mirror
Price of Power in a Data Center
mstansberry writes "Much like the rest of the country, IT is facing an energy crisis. The utilities are bracing companies for price spikes this winter and according to experts and IT pros, those prices aren't going to come down any time soon. This is thefirst article in a four-part series investigating the impact of energy issues on IT."
P.S. The submitter has a nice fishing web site and is holding about a 12" trout on his main page. Nice catch ... but I'd
recommend he go on a
fishing charter
in Seward Alaska if he wants to catch some mongo fish.
This trip was a major slayfest and my brother was
Captain
Crudd who knows how to fish with a beer in his hand.
Energy prices are going to hurt everybody.
From here:
"EIA expects energy expenditures will be 18% higher this winter compared to last winter, which will be 8.3% of the annual gross domestic product, a record since 1987 when it was 8.4%."
And for those of you who want to find a way to save energy: Here's 60 Tips To Save Energy This Winter
Materials needed: Fans. Flexible duct. Duct tape (of course).
Procedure: Place fans in datacenter. Tape duct to fans. Route duct to office spaces.
Results: Save money on heating and cooling bills.....
Just on the cusp of hydrogen fuel cell techonology becoming available, we're about to be hit hard with spikes in both gas and electricity. The SK crown corporation SaskEnergy asked the rate review panel for a 41% increase, but the review panel recommended "only" 27%. Auto gas prices have soared as high as $1.20/litre but have settled back at about a $1 CDN. Natural gas though is what scares Canadians, since most heat their homes with either that or electricity.
Sask Power is running advertising imploring people to unplug their underused second fridge, turn off their computers when not in use, and upgrade to LDC screens to save about 66% the power expense over CRT technology. They claim savings of $50/year if you turn off your computer when it's not in use.
Saskboy's blog is good. 9 out of 10 dentists agree.
This is about data centers.
Trust me, the cost of a roomful of PCs running Seti is nothing compared to keeping a 20-ton Liebert running 24x7.
No folly is more costly than the folly of intolerant idealism. - Winston Churchill
Let's be realistic, they won't come down...ever. If they can get another 20% (example) out of you this year, do you think they're going to drop it 20% next year after the "crisis"? 10% even? No way. Just like any other energy business that is at a near-monopoly level (gasoline), they can raise it whenever they feel like it and blame it on whatever they want. What are you going to do, go to the competition? In the area I live in (Midland, Michigan) and the surrounding cities (Saginaw, Bay City, Flint, etc) we get ONE choice for gas and electricity - Consumer's Energy. That's it. You don't like their service or prices? Tough shit. You're stuck. There have been "alternative companies" in the past, but all they do is resell energy for Consumers Energy - it's all going through the same pipes and wires.
It sucks, but that's the way it is.
Content Management System: A pretentious way of saying "text editor."
It sounds like Linux was running the laptop at a higher clock rate. Many laptops have a configurable clock rate, and will turn the rate down when power savings are needed (for example, when AC power disappears and the laptop switches to battery power).
A little fiddling with the power controls of Linux would probably get it to the same power consumption as Windows. While you measured something real, it's probably a configuration issue more than a builtin Linux vs. Windows difference.
I'd love the government to step in and set things straight but I don't see that happening anytime soon. For now we'll either have to bare the pricing or start pushing alternatives which have been available for a good long time now.
What a bunch of bullcrap. The cost of pumping oil varies from well to well. Sure, it might cost Saudi Arabian Oil Company $15 per barrel, but if they only release enough oil for half the world's demand, other producers have to fill that supply. It can cost those other suppliers much more to pull oil out of the ground. And that high price is going to lift the market price.
but instead of the $25 price before we invaded Iraq, it's pushing $70+ as a "permanent high". Maybe Congress and the White Hosue can exercise some accountability for their totally failed energy policies (including sending us to war) by stopping the price gouging the oil corporations are abusing us with.
Oh really? So they're just going to tell Saudi Arabia or Venezuala to lower their prices? How are they going to force them to do that? Oh, you mean force American Oil companies. Well here's a clue: American oil companies are bench-warmers in the global oil market. The biggest American company, ExxonMobil, ranks just 16th in the world in total reserves. They control about 2% of the worlds oil. Hell, even Petronas, a Malaysian company, is bigger than America's biggest oil company.
And looking at the table you see that the market is dominated by state-owned, national oil companies like Saudi Arabian Oil Company, and Petroleos de Venezuela. The only way you're going to lower the price they charge for oil is to invade and force them. Otherwise they'll sell their oil to the highest bidder.
I know those corporations are their best bribers^Wcontributors, and their foreign sources are our best traitors^Wallies, but Americans will vote on the entire House of Representatives and 1/3 of the Senate in elections next year. We might be willing to put up with a lot of BS on faith, but there's no denying we're not getting the spoils of all of our "superpower" status.
So your complaint is that Bush hasn't invaded enough countries yet to lower oil prices. Interesting.
The fact is state-run foreign oil companies set the price for oil. There is very little the government of the USA can do about it aside from rushing in with tanks to take their oil fields. Any kind of price control on this oil would mean it would get sold to someone else at a higher price, like the Chinese, for example.
Leading Oil and Gas Companies Around the World
Rank by 2004 Oil Equivalent Reserves Company Worldwide Liquids Reserves, Million Barrels Worldwide Natural Gas Reserves, Billion Cubic Feet Total Reserves in Oil Equivalent Barrels, Million Barrels
1 Saudi Arabian Oil Company (Saudi Arabia) 2,3 259,400 234,500 299,485
2 National Iranian Oil Company (Iran) 2,3 125,800 940,000 286,484
3 Qatar General Petroleum Corporation (Qatar) 3 15,207 910,000 170,763
4 Gazprom (Russia) 0 988,892 169,041
5 Iraq National Oil Company (Iraq) 2,3 115,000 110,000 133,803
6 Abu Dhabi National Oil Company (UAE) 3 92,200 196,100 125,721
7 Petroleos de Venezuela.S.A. (Venezuela) 3 78,998 149,891 104,620
8 Kuwait Petroleum Corporation (Kuwait) 3 99,000 55 99,009
9 Nigerian National Petroleum Corporation (Nigeria) 2,3 35,255 176,000 65,340
10 National Oil Company (Libya) 2,3 39,000 52,000 47,889
11 Sonatrach (Algeria) 2,3 11,800 160,500 39,236
12 OAO Lukoil (Russia) 23,215 39,089 29,897
13 Petronas (Malaysia) 5,290 85,200 19,854
14 PetroChina Co. Ltd. (China) 10,941 44,554 18,557
15 Petroleos Mexicanos (Mexico) 14,803 14,807 17,334
16 ExxonMobil Corporation (United States) 8,395 31,843 13,838
17 BP Corporation (United Kingdom) 5,775 46,650 13,729
18 Egyptian General Petroleum Corp. (Egypt) 2 3,700 58,500 13,700
19 OAO Yukos (Russia) 10,950 7,800 12,283
20 Petroleo Brasilerio S.A. (Brazil) 2 9,945 11,247 11,868
What percentage of a data center's revenue does power consume? What percentage of its expenses? If a cost that accounts for 2% of a company's revenues doubles, that's not good, but it's hardly a crisis.
Fuel accounted for 17% of Southwest Airlines' revenue, for example. A data center would have to be less than that, wouldn't it?
More and more players are entering the virtual market (look at the success of Citrix over the past decade, which is a technology that comes from a similar paradigm) - and that means that more and more datacenters are converting. While the cost per kwh might be rising, the costs of running a data-center are coming back under control.
It is difficult to free fools from the chains they revere.
-Voltaire
Who among us doubts that one AMD64 with a few gigs of RAM could, if programmed properly, calculate the payroll for the entire USA every night?
Interesting question. Let us consider a simplified universal payroll system and see where this goes. I'll stipulate roughly 200 million US payroll employees and 52 pay periods. Lets say individuals require 200 KiB of storage (historical deductions, contributions, etc. necessary for YTD results,) and generate 1 KiB of storage each period. The necessary software doesn't exist to accommodate the payroll requirements for every conceivable employment situation, but we'll just pretend it does.
You'll need 41 TiB of storage on day one. Each day you'll need to perform 28.5e6 payroll calculations generating 29.2 GiB of new data.
The storage requirements alone are going to blow your hardware budget. It's understood the data will need reliable storage, so a rack of 82 500 GB SATA drives won't do (with a new drive added every 16 days.) Lets factor in 25% storage overhead for (poor) redundancy. We need just over a hundred 500 GB drives. Storage at this scale will inevitably result in SAN or some such technology, so you'll need some storage switches. The storage hardware will consume many times the power of your AMD64.
Data will have to arrive via some network. Payroll is an aggregate calculation of many different forms of detail data. Lets say the time sheets, vacation requests, etc. represent 10 times the volume of the result. You'll need to handle 292 GiB of inbound (we'll assume outbound is negligible) data per day. That's 2 DS1s running at capacity at all times. You'll need redundant switching hardware for this. Also, we're taking for granted that fraction of Internet capacity necessary to move this data; it's not really free, after all.
Computational load is much harder to estimate without an accurate model of typical payroll calculations. I'll use my experience with OLAP consolidations. 2.4GHz Xeon will compute an 8 GiB cube in 45 minutes. This time is mostly (95%) spent in the CPU. Its also far simpler than payroll, consisting mostly of simple aggregates and hash calculations. Lets throw a factor of 3 at this to cover the extra computation necessary for payroll. We'll need 8.2 hours of CPU time per day. That seems quite feasible for an AMD64 CPU.
There is no wiggle room in the above estimates. No backups, no test system, no fail over. In the real world batch computation is highly synchronous, so you need a lot of spare network and compute capacity. You're doing a lot of co-processing here; the CPUs running the SAN switches and routers are all essential to the process, so you can't give all the credit to the AMD64 CPU. The interesting thing is that communications and storage are the real power hogs.
Disclaimer: To call the above 'back of a napkin' is probably flattery.
Lurking at the bottom of the gravity well, getting old
People cite Hydrogen as a source but most the available Hydrogen production is a byproduct of fossil fuel refining
Actually Iceland is doing quite well in working with hydrogen, "Iceland launches energy revolution". However Iceland has a big advantage over other countries, they have an abundance of geothermal energy they can use to generate hydrogen from water.
FalconShould there be a Law?
If you examine the 2005 Energy Bill you will discover how deeply flawed it is. It is a bill written to protect the domestic oil companies at the expense of the middle-class taxpayer. Our government simply does not care about us. We need to make them care before they sell out our future to the highest bidder.
Are you kidding me? I overclock my computer by 10% during the winter and increase the monitor power save timeto 30 minutes (from 10) just to warm my apartment up a little bit more. Given the double cost of natural gas this winter, keeping the area around my computer warm (where I'm sitting most the time anyway) might be more cost efficient.
Perfectly reversible computing does not produce heat.
Ever wondered what happens to bits that you erase out of memory or a register? They get dumped out of the chip and turn into heat.
Reversible logic reuses the electrical charge for your next computation, or for storing the next 1 that comes along.
On the downside, reversible hardware is much harder to design, but any addition of reverible logic on today's CPUs would decrease the amount of electricity needed and heat produced.
Electricity bills would be lower, and heat output would be smaller.
Laptops would last much longer, desktops wouldn't need a CPU cooler.
Even better, we could continue increasing the speed and diesize of CPUs.
One problem right now is that AMD, Intel, IBM, etc are perfectly able to produce a CPU that they have no hope of cooling. If reversible logic were used instead, you could have a 6GHz chip with the heat output of a 4.77 MHz 8086.
Shae Erisson - ScannedInAvian.com
I don't know what electricity bills cost in the States, but I can tell you that rates in Quebec are also increasing. Hydro Quebec sends a lot of juice down the pike to the US, so increasing demand in the US means increasing demand for Hydro Quebec. In effect, we pay higher rates so that Hydro Quebec can export electricity to the US. Joy.
One of the selling points for Sun's Niagara is that a single Niagara processor can do the work of a bunch of single core servers - for about the same amount of power as one single core server.
A Shadeless room is a brighter room.
For a financial data center ("Tier-4" class, 2(N+1)), the cost per kW is $10-15,000 for infrastructure alone. That quickly matches cost of WinTel boxes (although the depreciation cycle is considerably longer).
Energy Consumption works out close to 3x server power consumption, so 1kW of load is equal to 3kW total energy input. That comes close to $2,700 per year in energy costs for 1kW of server power.
This is just the kind of screwed-up priorities that cause companies to lose all competitive edge.
Good (?) accounting tends to highlight grand total costs of small things. Good Lord, we spend $27,000 per year on paper clips! Better control them under lock & key. The lost-opportunity cost of the contract bid missed because somebody was hunting for paperclips does not, of course, appear on and ledger.
Now somebody has summed up the electrical costs of a really large server room and come up with a sum close to a human salary. That always impresses people. (Man is the measure of all things.)
But what is it as a fraction of total operations and capital?
At 11 cents per kilowatt-hour (a common residential cost except in badly-gouged locales; but high for major consumers, at least until lately and those 27% increases) your rule-of-thumb for 7x24 consumption is:
a buck per watt per year.
500-watt average constant consumption from a basic 3u rack server = $500/year. Easy, no?
But that's a pretty serious machine, home machines don't commonly have over 400W power supplies - and certainly don't use the 400W all the time. So we're allowing for air conditioning power in the estimate.
But a serious server starts at $10,000 and you won't get five years out of it, so the capital cost alone is $2000/year and up.
All but the most automated shops surely have a salaried sysadmin (and/or DBA, backup specialty guy...) for every ten machines. And those guys all cost $50,000 dead minimum. So that's another $5,000 per machined per year for care & feeding.
So that's $7000/year, plus power at $500. Maybe skyrocketing to $700 and a full 10% of costs.
And of course I had to assume that the $10,000 included 5 years of vendor support to keep it that low. Never mind insurance, rent on the space, huge UPS's, fire systems, air conditioning (not the power for it, the machinery). In truth, I can hardly imagine power reaching 10% of the operations cost.
Also, I'm taking some place like NCAR as my site: gargantuan computing power at the service of a dozen professors and their retinue of grad students. Totally running their own programs, not million-dollar software packages like SAP on Oracle. In short, the normal "IT" costs of programmers, analysts, support techs, software vendors, don't exist.
Because when they do, they dwarf the cost of running the server room and power dwindles down to being 10% of 10% of your total IT budget. Which in most companies is 5%-9% of total operating expenditure.
Wailing about this cost - which springs out on the accounting spreadsheet because it is up a large percentage from last year - leads to classic penny-wise, pound-foolish decisions.
Perhaps: "we'll use less power if we consolidate a dozen servers down into one big one". A lot of this has been done by IT departments, whom I swear are pining for the days of the mainframe.
But at least where I work, business didn't move off the mainframe because it was such a high cost per compute cycle - often enough we were increasing our total computing costs to go PC and small server. We did it for the flexibility.
And loss of flexibility could cost a business big - for want of a paperclip.