Slashdot Mirror
Shortage of Electricity Drives Data Center Talks
Engineer-Poet writes "Per the San Jose Mercury News, competitors such as Google and Yahoo are meeting to discuss the issue of electricity in Silicon Valley. How much of the USA's 4038 billion kWh/year goes into data centers? Enough to make a difference. Data centers are moving out of California to spread the load and avoid a single-point-of-failure scenario. This is a serious matter; as Andrew Karsner (assistant secretary of energy efficiency and renewable energy for the Department of Energy) asked, 'What happens to national productivity when Google goes down for 72 hours?' I'm sure nobody wants to know." From the article: "Concern about electricity pricing and volatility has led Microsoft to talk with its network manufacturers about building more efficient servers. IBM and Hewlett-Packard -- which both build data centers -- want to improve efficiency at the facilities. AMD promotes changing the design of data centers to increase airflow to keep the supercomputers cool."
When Google goes down, productivity probably goes up.
One thing that needs to be looked at with the congregation of data centers is why are they like that? Here in the North East, any kind of bandwidth will cost an arm and a leg compared to the North West area. I've recently been involved in pricing out Colocations for one of our webservers and a simple T1 costs 4-5 times in the N.E. that it costs in the N.W. I'm sure we'd see more evenly distributed data centers if costs we evenly distributed too. How about taking some of those new 40% efficiency solar panels and moving some data centers down to the S.W. for a start?
We just started switching from Intel to AMD hardware in our servers (HP DL385). Not that we pay per Kw/h, but I figure less power consumption means less heat and less fried hardware.
AMD has a website on the topic: Real Efficiency in the Data Center
"No matter where you go, there you are." -- Buckaroo Banzai
Locals and guest workers would be hired to pedal for one-hour shifts each, generating some portion of the needed power and giving a boost to the local economy. Don't think "galley" -- think "self-sustaining"!
If you'd like to use this idea, please contact me via my Slashdot account. Thanks.
The difference between stupidity and genius is that genius has its limits.
So Google has more money than it has electricity. And it's HQ'd in some of the most expensive real estate in the country. And its servers are remote to practically every user in the world.
That sounds like a perfect reason for nearly all of Google's servers to live distributed around the US, and the globe. With local operators for physical access, and global remote admins for most normal operations.
The past year or so we've heard all kinds of wild rumors about "googled in a box": supercomputers in a shipping container for rapid deployment around the world. How about just a briefcase of money dropped on the local economies to build datacenters in-place, the old fashioned way, without the alien assault tech strategy?
Cheaper, more redundant, more energy efficient (at least not overloaded). Sufficiently distributed, they could use lower-density energy generation, like solar/wind/environmental.
Google should force manufacturers and designers to make all our power consumption more efficient, using their buying power to improve the tech. Then they should use that tech in the more economical, reliable, power efficient way. Share the wealth and power with the rest of us who are keeping them hot.
--
make install -not war
Now, I hope, people will start to understand why Sun and Intel are focussing so hard on performance-per-watt, and not just performance.
I am TheRaven on Soylent News
Data center consolidation. ESX. Good Stuff.
Given the abundance of geothermal power in iceland (hence why aluminium ore is transported there for refinement) perhaps a few trucks of fibre need to be put in place - Reykjavik becoming the next big hub for data centers... Lots of power on tap, lots of cooling easily available (ie its bloody freezing there), and the good old days of meetings in hot tubs could come back too - though obviously thermal springs rather than hot tubs....
$_="Slashdotter";$syn="OTT";s;..;;;sub _{print shift||$_};s!ash!Perl !;s=$syn=ack=i;tr+LLEd+BLAH+;_"Just Another ";_
Google had the right idea when they located their datacenter in Oregon, in a colder climate so they don't need as much air con power, and right next to a big hydro power plant.
What's the point of locating your datacenter in an area with high ground prices, a history of electric power supply problems and a hot climate?
It's still a good idea to reduce server power because it reduces both the operating power AND the cooling power required.
On another note, has anyone noticed that language used impacts performance per Watt?
I recall from a /. comment from the son of a congressman(Congressional brat?), where he said that his father worked rather more than six days a week, due to the necessary reading of bills in his committee, and such.
Ninjas and pirates. How piquant.
You gotta remmeber that, when a blackout hits a huge swath of area, it also brings down the *client machines* in that area as well, so your backup centre doesn't necessairily have to handle your entire peak load.
Google only needs one of two redundant data centers (one in the East, one in the West, one Mid-Central) to basically ensure they can whether any power loss scenario. If they had 3 such separate centers (which I have no doubt they already have), the only way they're going to be totally off line is if the whole national grid goes down - in which case Google should be the least of your worries if you're a lawmaker.
People who manage and run data centers have to think it through before making changes. Many servers that are more than a year old were not designed for energy efficiencies. To top that, they weren't designed to take advantage of natural efficiencies in telecomms data centers. Most telecomms equipment is designed to run off of -48VDC. This has the effect of reducing the number of wasteful 115VAC to DC conversions along with the subsequent losses to heat that have to be removed by A/C systems. I've seen estimates that show the possibility of up to a 35% reduction in power and A/C costs simply by converting the AC power supplies in servers to DC power supplies.
Additionally, much of the forced air (from the floor upwards) A/C systems I've seen in data centers is not configured properly. There are vented tiles in places they shouldn't be, and not where they should be... causing hotspots and A/C problems in general.
I see datacenters with a wide variety of rack types. This can work, but often leads to inefficient use of the A/C systems. Its expensive to change racks, if its even possible (some vendors don't like their kit in someone else's rack) but this problem also needs to be looked at. A/C accounts for a huge energy drain in datacenters.
Using older hardware rather than buying new hardware saves on the short term, but the savings in energy costs by buying newer, more efficient hardware is something that datacenter managers HAVE to look at if this problem is to be solved. Its not just a matter of being 'green'. Its a matter of saving money that can then be used to bolster other parts/systems of the company.
I think that we'll see Google et al running VM clusters soon, where unused servers in the cluster are shutdown till they are needed for heavier traffic. In much the same way that complex automotive engines shut off several cylinders during low power requirement times, servers can be shut down (sleep mode) to save power until they are needed.
These are just some of the ideas that are currently the talk of datacenter managers and the vendors who support them. Try perusing the APC website, or other datacenter vendor's websites.
Support NYCountryLawyer RIAA vs People
I think we all know where 'analists' are pulling their numbers from though...
Another interesting tidbit for comparison: a typical high-density rack puts out something in the neighborhood of 15KW of heat. An average home electric oven puts out about 7-8KW of heat. So each high-density rack is like having two ovens going full blast, 24x7.
Data centers need to figure out a way to use the "waste" heat and turn it back into something useful, namely electricity. The problem is they generate a lot of heat, but it isn't hot enough, which seems screwy but for co generation you want as hot as possible. So the tech that needs to be developed (along with the obvious not generating so much waste heat through efficiency gains), is to find better ways to accumulate/move and use the low temp stuff they do have lots of. There are some alternative energy projects out there doing that with solar thermal to stirling engines for example.
Given US laws these days I suspect companies would prefer their data OUTSIDE of US jurisdiction....
That aside latency is not really a distance issue - its a network design issue.
If you put a big trunk of fiber (as my original comment was saying) from iceland to NY and iceland to london (thus making a nice redundant triangle with the current transatlantic connections) and connected it the the existing back bones sensibly the extra distance would not really be noticed.
Hops add far more latency than distance, most of the hops being to get up from your home/office connection to the back bone, then back down from the back bone to the company - hence why data centers are built where its very easy to get trunk connections - the proximity is a cost issue - fiber is expensive to lay.
$_="Slashdotter";$syn="OTT";s;..;;;sub _{print shift||$_};s!ash!Perl !;s=$syn=ack=i;tr+LLEd+BLAH+;_"Just Another ";_
The 500 pound gorilla in the corner is that in a typical Silicon Valley datacenter only 50-60% of the power goes to the computers while the other half goes to the support equipment. It does not have to be this way, and things are changing. I have not yet walked into a datacenter that could not cut its total power usage by at least 25% (albeit, in some cases the design damage is done and the simple payback required to make it work would stretch to 4-5 years)(I'm looking at you, datacenters with dozens of 20-30 ton air-cooled compressors on the roof).
On the gross kWh/yr side, the vast majority of datacenters are unable to use outside air directly for cooling. A 24 hour a day load and they can't 'open the windows' to cool it at night (with appropriate filtration and redundant humidity control lockouts of course)? Come on people! It would even improve reliability (even 70F outdoor air could hold a well configured hot aisle/cold aisle datacenter). But that doesn't help trimming peak load, to do that you have to get the airflow right.
Efficiency in datacenters starts with just a basic understanding of airflow. You want it very hot behind the racks; you want that hot air to go directly back to your cooling unit not get recirc'd to a rack intake. And you have to have airflow controlled based on the cold aisle temperature to harvest energy savings (fan energy wastage is ridiculous in these things)(oh, and watch out for those server fans that ramp up if you push the cold aisle temp too high - not efficient to provoke a rack of those guys to start screaming).
You have to know hot aisle / cold aisle to properly design and operate an efficient datacenter, even if that exact configuration is not applicable. Period.
Of course, its not "that simple," but to the design engineers it certainly should be pretty straightforward work. The information is out there and more is in the pipeline. A good start on the basics of efficient datacenters is available here (full disclosure, I was associated with producing that report, so I am not impartial)(but don't blame me for the blurry graphics - I did not create the pdf!).
And for god's sake people, quit keeping these places at 55-60F - I'm freezing my butt off and you're making a mockery of your own 'tight humidity control' (70-90% RH at the server intakes, but a good 45% +/- 2% at the air handler return).
The problem is such that PG&E is actually offering rebates of about $150 for every physical server that is virtualized. The rebates can go up to $4MILLION for each company. Then there is the additional savings companies will see in reduced power consumption by the servers themselves and cooling.
More info HERE
There is nothing inherently safe about liberty. That's why so many people died protecting it.
The Sun T2000 stuff was obsolete the day it launched when compared to competing x86 solutions.
;).
http://www.anandtech.com/printarticle.aspx?i=2727
The CPU power/watt wasn't really that much better compared to x86 stuff of that time.
It is now nearly 9 months from that, and AMD and Intel have improved significantly. Where is the T2000 or T1 now? Look at Intel - their latest CPUs now trash AMD's by about the same margin which AMD used to trash Intel's offerings.
As long as you skip the Intel P4 stuff, and the silly AMD FX stuff (esp the quad one), the recent x86 stuff is pretty decent.
Go do performance/watt stuff yourself. Sure the Sun wins in some niche situations and in situations when you can actually use the crypto engine, but for most cases the T2000 isn't worth the bother.
Sun doesn't even bother doing specint rate for the T2000/T1 (maybe you can guess why looking at Anandtech's benchmarks) - they only do it for their SPARC IV+ and that gets:
Sun Fire E25K (72 processor) 144 cores, 72 chips, 2 cores/chip: 1413, 1644
144 cores, how much money and watts to get a score of 1413?
In contrast Intel's CPU gets a score of 64 with just 2 cores.
Intel(R) DG965WH motherboard( 2.93 GHz, Intel(R) Core(TM) 2 2 cores, 1 chip, 2 cores/chip: 64.3, 64.4
Maybe AMD will have an answer next year, but whatever it is, AMD and Intel in their frenetic race with each other, have left Sun's CPUs behind in the dust.
If your app works much better with a single system image with 144 cores then I guess you could buy Sun, but if rest of us need the processing power of 144 SPARC IV+ cores we'd get about twenty-two single CPU x86 servers with a total of 44 cores (or eleven dual CPU x86 servers), and figure out a way to make do with such "restrictions", like having money left over for storage, UPS, backups, generators, party for everyone etc.
You can still run Solaris on a Sun x86 server y'know
I run a rather large department for one of the Investment Banks, with users / developers / support staff dispersed between London / Amsterdam / Cairo / Milan and Rio.
About one year ago the folks maintaining our applications infrastructure were advised by the companies responsible for the municipal grid to reduce our hardware footprint in London.
The reason? The grid was close to if not already overloaded, and increases in consumption were to be discouraged.
So we've been putting all new build into Central Europe, and slowly migrating existing systems over as we can.
A strange situation all around, if you ask me.
A message from our sponsor