Slashdot Mirror
Shortage of Electricity Drives Data Center Talks
Engineer-Poet writes "Per the San Jose Mercury News, competitors such as Google and Yahoo are meeting to discuss the issue of electricity in Silicon Valley. How much of the USA's 4038 billion kWh/year goes into data centers? Enough to make a difference. Data centers are moving out of California to spread the load and avoid a single-point-of-failure scenario. This is a serious matter; as Andrew Karsner (assistant secretary of energy efficiency and renewable energy for the Department of Energy) asked, 'What happens to national productivity when Google goes down for 72 hours?' I'm sure nobody wants to know." From the article: "Concern about electricity pricing and volatility has led Microsoft to talk with its network manufacturers about building more efficient servers. IBM and Hewlett-Packard -- which both build data centers -- want to improve efficiency at the facilities. AMD promotes changing the design of data centers to increase airflow to keep the supercomputers cool."
One thing that needs to be looked at with the congregation of data centers is why are they like that? Here in the North East, any kind of bandwidth will cost an arm and a leg compared to the North West area. I've recently been involved in pricing out Colocations for one of our webservers and a simple T1 costs 4-5 times in the N.E. that it costs in the N.W. I'm sure we'd see more evenly distributed data centers if costs we evenly distributed too. How about taking some of those new 40% efficiency solar panels and moving some data centers down to the S.W. for a start?
Data center consolidation. ESX. Good Stuff.
I'm surprised no one has mentioned Sun Microsystem's CoolThreads technology. The company I work for had a problem with too much heat and too much power in our co-lo data center. We managed to replace 2 V880 servers with one T2000, not only did we free up an entire rack with a single 2U box but we also eliminated over 5000 watts of power consumption. (estimating 2800 watts per V880 and T2000 consuming around 250) Obviously those boxes are not for everyone, there is only one FPU even though the box has 8 cores. In our case it was processing incoming streaming data, all integer math. We went from needing two V880s with 8 1.2Ghz USIIIs running at nearly 50% CPU usage to one T2000 at about 20%.. When it comes to serving up massive amounts of data that don't require a lot of floating point calculations they cannot be beat.
The 500 pound gorilla in the corner is that in a typical Silicon Valley datacenter only 50-60% of the power goes to the computers while the other half goes to the support equipment. It does not have to be this way, and things are changing. I have not yet walked into a datacenter that could not cut its total power usage by at least 25% (albeit, in some cases the design damage is done and the simple payback required to make it work would stretch to 4-5 years)(I'm looking at you, datacenters with dozens of 20-30 ton air-cooled compressors on the roof).
On the gross kWh/yr side, the vast majority of datacenters are unable to use outside air directly for cooling. A 24 hour a day load and they can't 'open the windows' to cool it at night (with appropriate filtration and redundant humidity control lockouts of course)? Come on people! It would even improve reliability (even 70F outdoor air could hold a well configured hot aisle/cold aisle datacenter). But that doesn't help trimming peak load, to do that you have to get the airflow right.
Efficiency in datacenters starts with just a basic understanding of airflow. You want it very hot behind the racks; you want that hot air to go directly back to your cooling unit not get recirc'd to a rack intake. And you have to have airflow controlled based on the cold aisle temperature to harvest energy savings (fan energy wastage is ridiculous in these things)(oh, and watch out for those server fans that ramp up if you push the cold aisle temp too high - not efficient to provoke a rack of those guys to start screaming).
You have to know hot aisle / cold aisle to properly design and operate an efficient datacenter, even if that exact configuration is not applicable. Period.
Of course, its not "that simple," but to the design engineers it certainly should be pretty straightforward work. The information is out there and more is in the pipeline. A good start on the basics of efficient datacenters is available here (full disclosure, I was associated with producing that report, so I am not impartial)(but don't blame me for the blurry graphics - I did not create the pdf!).
And for god's sake people, quit keeping these places at 55-60F - I'm freezing my butt off and you're making a mockery of your own 'tight humidity control' (70-90% RH at the server intakes, but a good 45% +/- 2% at the air handler return).
The problem is such that PG&E is actually offering rebates of about $150 for every physical server that is virtualized. The rebates can go up to $4MILLION for each company. Then there is the additional savings companies will see in reduced power consumption by the servers themselves and cooling.
More info HERE
There is nothing inherently safe about liberty. That's why so many people died protecting it.
The Sun T2000 stuff was obsolete the day it launched when compared to competing x86 solutions.
;).
http://www.anandtech.com/printarticle.aspx?i=2727
The CPU power/watt wasn't really that much better compared to x86 stuff of that time.
It is now nearly 9 months from that, and AMD and Intel have improved significantly. Where is the T2000 or T1 now? Look at Intel - their latest CPUs now trash AMD's by about the same margin which AMD used to trash Intel's offerings.
As long as you skip the Intel P4 stuff, and the silly AMD FX stuff (esp the quad one), the recent x86 stuff is pretty decent.
Go do performance/watt stuff yourself. Sure the Sun wins in some niche situations and in situations when you can actually use the crypto engine, but for most cases the T2000 isn't worth the bother.
Sun doesn't even bother doing specint rate for the T2000/T1 (maybe you can guess why looking at Anandtech's benchmarks) - they only do it for their SPARC IV+ and that gets:
Sun Fire E25K (72 processor) 144 cores, 72 chips, 2 cores/chip: 1413, 1644
144 cores, how much money and watts to get a score of 1413?
In contrast Intel's CPU gets a score of 64 with just 2 cores.
Intel(R) DG965WH motherboard( 2.93 GHz, Intel(R) Core(TM) 2 2 cores, 1 chip, 2 cores/chip: 64.3, 64.4
Maybe AMD will have an answer next year, but whatever it is, AMD and Intel in their frenetic race with each other, have left Sun's CPUs behind in the dust.
If your app works much better with a single system image with 144 cores then I guess you could buy Sun, but if rest of us need the processing power of 144 SPARC IV+ cores we'd get about twenty-two single CPU x86 servers with a total of 44 cores (or eleven dual CPU x86 servers), and figure out a way to make do with such "restrictions", like having money left over for storage, UPS, backups, generators, party for everyone etc.
You can still run Solaris on a Sun x86 server y'know
That sounds like a perfect reason for nearly all of Google's servers to live distributed around the US, and the globe. With local operators for physical access, and global remote admins for most normal operations. Uh, they are already doing that, I.e. moving to The Dalles (Real close to Bonneville hydroelectric plant on the Columbia River.)
Silicon Valley (where I live) isn't hot. The reasons why a company would locate their data center here are numerous:
No, I will not work for your startup