Slashdot Mirror


Benchmarking Power-Efficient Servers

modapi writes "According to the EPA, data centers — not including Google et al. — are on track to double power consumption in the next five years, to 3% of the US energy budget. That is a lot of expensive power. Can we cut the power requirement? We could, if we had a reliable way to benchmark power consumption across architectures. Which is what JouleSort: A Balanced Energy-Efficiency Benchmark (PDF), by a team from HP and Stanford, tries to do. StorageMojo summarizes the key findings of the paper and contrasts it with the recent Google paper, Power Provisioning for a Warehouse-sized Computer (PDF). The HP/Stanford authors use the benchmark to design a power-efficient server — with a mobile processor and lots of I/O — and to consider the role of software, RAM, and power supplies in power consumption."

97 comments

  1. ummmmm by djupedal · · Score: 2, Insightful

    When someone considers the impact, end-to-end, from carving copper oreout of the ground to throwing the out-of-date server chassis into a furnace, then I'll pay attention...maybe.

    Until then, this is just marketing 101...

    1. Re:ummmmm by kebes · · Score: 4, Insightful

      Your basic point, which is that we need to consider not just operating costs also manufacturing and disposal costs, is a valid one.

      However the way you've worded it amounts to "since we can't account for all aspects of impact, I'm not going to worry about any aspect of impact." That's a bit extreme. Surely reducing our power consumption during the operating lifetime of our servers is a step towards greater environmental and fiscal responsibility.

      Now, if you can show that the "energy saving" chips generate more pollution during production than the "normal" chips (and that this increase in pollution/energy-use/cost is greater than the savings during the lifetime operation of the chip), that's important. However I doubt that is the case. Thus, to ignore the potential advantages of power-saving measures in the data-center, simply because such measures don't address the orthogonal concerns of production impact, is silly.

    2. Re:ummmmm by Ryan+Amos · · Score: 1

      This is important in the IT industry for reasons other than being a tree-hugging hippie (not that there's anything wrong with that.)

      More efficient, lower power servers directly relate to a cash savings on your electric bill. One server operating at 10% greater efficiency may not be a big deal, but it starts to matter when multiplied over a room of servers. Servers that use less power (generally) put off less heat, so you also save electricity because you don't have to cool them as much, and you can cram more servers on a UPS without overloading it. In the end it's really about density.

      It is marketing, but it's not coming out of left field. There are real business reasons to want servers that are cheaper to operate.

    3. Re:ummmmm by Ironsides · · Score: 1

      However, if those more efficient servers cost twice as much to purchase per unit of work, not to mention the energy used in manufacturing, the savings are reduced.

      --
      Fly me to the moon Let me sing among those stars Let me see what spring is like On jupiter and mars
    4. Re:ummmmm by Bearhouse · · Score: 1

      You're right - let's do what we can, with what we've got. But at the same time, the rush to convert to so-called energy-efficient, 'sustainable' and/or renewables has already gotten out of hand.

      The crazy pork-loaded policy of subsidising turning feedstuff into ethanol is already distorting world food prices and policies, causing harm to the poor.

      The Toyota Prius (pious?) uses more fuel than a good small diesel car, and is less functional. In fact, you'd be doing more good for the planet if you just bought a good, recent used small car. (The pollution the ships cause when transporting the cars to market is massive). But hey, plenty of LA stars have a Prius, (and maybe also a Hummer, and a Ferrari...)

      What we need is more information on the end-to-end impact, so we can make informed decisions, not marketing waffle.

    5. Re:ummmmm by Azghoul · · Score: 2, Insightful

      ... except that the Prius promises far more benefit long term than yet another diesel, by way of popularizing the idea that a car can, in fact, be driven by something other than burning dead shit.

      Never misunderestimate [:)] the power of technological progress - you gotta start somewhere.

    6. Re:ummmmm by trolltalk.com · · Score: 1

      "However, if those more efficient servers cost twice as much to purchase per unit of work, not to mention the energy used in manufacturing, the savings are reduced."

      I think you missed something obvious - the cost of the servers includes the cost of the energy used to make them.

      That said, they could help reduce everyone's energy consumption by posting their stuff as plain html instead of pdf. Less data to transfer, no need to open up a pdf reader, etc.

    7. Re:ummmmm by Bearhouse · · Score: 1

      Urm...like the fuel it burns? Remember, the Prius is not, unless you hack it, a 'plug-in'...it uses fuel like any other car, and more than some. Also, it's hard to recycle (battery pack) and performs poorly in a crash. So, not really a good role model then.

      But a Tesla, http://www.teslamotors.com/, hmmmm

      Of course, the electricity to recharge the cells is mostly generated by coal-fired power stations. Damn...

      The sad fact is, (and yes, I mean 'sad' - I have kids, so I'm concerned about the future of the planet, which has visibly gone even further to shit in my lifetime), that hydrocarbon fuels are still the cheapest and LEAST polluting of all the options for personal transportation... If you talk to the non-nutjobs, the best plan seems to be hydrogen and electricity produced by nuclear power, but, of course, we can't say that in public...

    8. Re:ummmmm by Spoke · · Score: 1

      The Toyota Prius (pious?) uses more fuel than a good small diesel car, and is less functional. Please elaborate, because I don't believe you.

      I don't see any diesel cars on the market (or any diesels for that matter) that are similar in fuel economy, functionality and emissions, let alone more efficient or less polluting.
    9. Re:ummmmm by orangesquid · · Score: 1

      I've always wanted to see better benchmarking systems that show how efficient various architectures are for various types of tasks, rather than just lists of MFLOPS and MIPS numbers. I know there's always a business push to be able to compare everything from vector-oriented supercomputers with huge registers to commodity clusters to massively parallel RISC arrays using just one single "magic" number (typically MFLOPS), but, this has so little effect on real-world performance. I want a benchmark to spit out dozens of numbers, and different architectures should show their particular strengths readily through those numbers, if the benchmark was designed well.

      --
      --TheOrangeSquid Is it any wonder things seem so awry? We swim in a sea of confusion and don't have to think to survive
    10. Re:ummmmm by Bearhouse · · Score: 1

      No problem, here in Europe we've got the Volks Polo for a start. I'm not sure if you guys in the sates get the same models...
      Also, checkout the latest Volks, Merc & BMW 'super efficient' models - low rolling resistance, engine cut-off on coast and at red lights...in 'real world' driving, (not EPA bullshit - yup, I'm an avid Car & Driver reader too), they beat the shit out of the Prius.

    11. Re:ummmmm by Spoke · · Score: 1

      But none of the latest Volkswagons, Mercedes and BMWs are clean enough for for sale in the USA (yet, should be able to in the next year). With fancy (and expensive) urea injection and particle capture exhaust systems they manage to meet emissions requirements, but they still aren't for sale.

      BTW, did you remember to take into account the fact that diesel has ~10-20% more energy than gasoline and as a result ~10-20% more CO2 emissions for same fuel economy?

      Hybrids won't really get good until they start putting LiIon batteries in them and give them plug-in capabilities, but even now your typical hybrid is worth 15-30% in fuel economy depending on the system.

      BTW, anyone regularly getting less than 40mpg on their Prius either needs to have their car or driving checked out, everyone I know gets 45+mpg in real world driving.

    12. Re:ummmmm by redcane · · Score: 1

      Looking at http://www.greenvehicleguide.gov.au/, The Prius comes first in terms of greenhouse gasses and air pollution, and equals the diesels in terms of fuel consumption. The Prius also offers more usable space than it's most fuel efficient competitors. Sure the embodied energy of any new car isn't amortized across as many passenger/kms, so you are nearly always better off putting the kilometres on a used car than a new one in terms of total energy use, but the Prius amortizes those manufacture costs much more quickly than almost any other cars. Your actually better off buying the *most* used car rather than a recent one, since it's manufacture costs have already been spread across a lot of mileage. I'd be surprised if the transport of cars to market is a big factor, especially spread across the volume of cars. I think you'd find those ships are reasonably efficient for the amount of cargo they move (the very nature of their scale makes this an important factor for the companies operating them).

    13. Re:ummmmm by Azghoul · · Score: 1

      Yes of course the Prius burns fuel. Did you think we'd ever be able to one day just flip a switch (pun) and never use oil again? My point is that the Prius is a stepping stone towards making alternatives feasible in the marketplace. You can't mandate change.

      And I'm with you on nuke power, absolutely.

  2. Units? by niceone · · Score: 3, Funny

    I'm hoping the units are going to be kWh/slasdotting.

  3. virtualization? by stoolpigeon · · Score: 1

    I'm not a hardware guy - but in an all hands meeting the other week, we were told that virtualization was going to save us a bunch of money on power. Our data center isn't all that big and they were talking about 2-3 thousand (US$) a month or something like that.

    --
    It's hard to believe that's how Micronians are made. Why don't we see it right now by having you both kiss one another?
    1. Re:virtualization? by eln · · Score: 2, Interesting

      Everyone is looking at virtualization for this sort of thing, and it does hold some promise. Currently, though, virtualization still comes with some very significant performance penalties. I think if virtualization can further mature, and we can get more cores and cheaper memory, we will be where we want to be.

      Eventually, if RAM continues to get bigger and cheaper, more cores get packed into chips, and virtualization becomes what it is intended to be in terms of performance and stability, we will start to see a lot of movement away from lots of small pieces of commodity hardware and back to fewer large systems in the data center. Of course, cheaper and faster disks on Enterprise storage systems like Netapp or EMC will help that transition too.

    2. Re:virtualization? by jon287 · · Score: 1

      Absolutely. I've seen a dozen or more corporate datacenters with racks and racks of outadated windows servers, mostly pentiums 2's and 3's that are nothing more than desktops turned on their sides with metal brackets from the harware store holding them in.

      Literally dozens of those things could be displaced by single modern VMWare or Xen hosts. Its all a matter of manpower and know-how. (As well as convincing the PHB that his initial outlay will be made up quicky with power savings and administration cost savings). Add to this the fact that the department that pays for the power is usually different than the one ordering new servers and the politcal nonsense praticly guarantees another decade of PII's.

      We've calculated that a single server running full blast can use up to $30 of power per month all by itself. That doesn't even count the power the AC's need to get rid of the heat it makes.

      --
      To boldly use to and too two times and get it right too! They're not gonna believe their eyes when they see it there!
    3. Re:virtualization? by Znork · · Score: 1

      "we were told that virtualization was going to save us a bunch of money on power."

      Sortof. Unfortunately the ease of deployment and price reductions accomplished tend to result in a vast expansion of virtual servers instead. You're likely to end up with as much hardware except it's doing several times more than what it used to.

      At least from what I've seen of virtualiztion your bill isnt going to get smaller, you're just going to get more for it. Which isnt too bad anyway.

    4. Re:virtualization? by growse · · Score: 1

      So you mean, big boxes with loads of CPUs and tonnes of memory, all connected to a huge storage system?

      Sounds like IBM did a good thing keeping their mainframe business open :p

      Commodity hardware was sold over big mainframes on the basis that it's much more scalable. If you want to do something else, just buy a couple of relatively cheap boxes and away you go. The thing that no-one mentioned is that it suddenly starts to cost a lot more $$$ to keep the things in power and cooled properly, so now we're seeing a potential shift back towards the big-powerful-single-box-model. Interesting that.

      --
      There is nothing interesting going on at my blog
    5. Re:virtualization? by bunratty · · Score: 1

      In case anyone is wondering how to estimate the cost of power for running a server, I've found that simply plugging the server into a Kill A Watt EZ is quite effective. Just enter the cost for power and let it measure the power usage for a few days to get a good average, and it will calculate the cost for power per month or year automatically. To account for the cost of cooling, you may need to double that amount.

      --
      What a fool believes, he sees, no wise man has the power to reason away.
    6. Re:virtualization? by Ironsides · · Score: 1

      It makes a lot more sense when you look at the cost of the computers over the past 20-30 years. In the 80's, a low end desktop cost $3,000. Today, you can get a high end desktop for that cost. Mainframes used to have seven figure prices. Today, you can get them for six figures. As the price of the mainframe came down, their advantage has grown, especially when you take into account the ease of maintaining redundancy on a single mainframe vs. tens (or more) of servers.

      Basically, mainframes are cheaper for base load computing power today, while server are better for incremental upgrades.

      --
      Fly me to the moon Let me sing among those stars Let me see what spring is like On jupiter and mars
    7. Re:virtualization? by Ironsides · · Score: 1

      Write up a paper/memo/whatever and send it to the guys who pay for the power over the possible cost reductions in power. Write up a whatever and send it to the guys who pay for the replacement parts and management of the aging servers on the costs of managing 50 P2s over one larger server.

      If at that point they don't get together and work (apply pressure) to reduce the number, your company is full of idiots.

      --
      Fly me to the moon Let me sing among those stars Let me see what spring is like On jupiter and mars
    8. Re:virtualization? by Bearhouse · · Score: 1

      Yippee, back to the mainframe! I wonder if I still have my old MVS and CICS manuals in the garage...

      I think we have the virtualisation technology we need - in fact we've had it for a long time. I just don't see massive adoption happening until there's a fat, cheap and secure pipe everywhere... Until then, I'll stick with my laptop, and home PC, and server, and think my kids will too. OK salesforce.com works, but it's still peanuts compared to the PC users worldwide - and what do people connect to salesforce.com with? Not VT22Os, methinks...

    9. Re:virtualization? by suggsjc · · Score: 1

      First, I want to state that I haven't actually worked directly with any virtualization implementations.

      Now, when the whole "virtualization is the answer to everything" wave started rolling in, I got excited and thought it was going to actually make good on all of its promises (and it still may). However, what is keeping me from actually putting it to use is that when you put several different VMs inside one box, then all of those VMs can be taken down by a single failure (disk, power supply, nic, etc) that would have normally only taken down a single "real" machine. Granted its all about the architecture and application, but I'm still leaning toward smaller cheaper boxes that all do specific tasks. They can also be mirrored by small cheap boxes, and the points of failure be completely isolated from each other.

      Given that this whole topic is about reducing power consumption, my latest off the wall idea is to build my system entirely out of NSLU2's. They each have a small finite amount of computing power, but if you could cleverly (and I do think I'm pretty clever) break up the application into different components (in my case database sharding to the extreme) then you could distribute your work load evenly across them. You could upgrade your system in
      Again, my application is specific (a database driven website) but being able to spread a small measured load across multiple machines could also be more efficient at the hardware level. Consider that in the traditional database model, you have a db instance that services queries to all of the tables. Therefore the system load is the aggregate of queries to all tables. If you could fairly accurately break down your usage patterns, then you could move specific functions to different boxes to maximize the overall load. I do realize that when sharding you lose some of the abilities to create joins, etc. But that doesn't mean it still can't be overcome, especially when considering that when the distribution/index size of the data, less computation is required to find the desired result (searching 100 records is more efficient than searching 10000 records).

      Whew, whether or not I actually use NSLU2's or some other low-cost/low-power server is irrelevant. I think that when the application and architecture can support it, then multiple small servers can out perform larger ones on cost and consumption...but I will concede that space *might* be sacrificed.

      --
      When I have a kid, I want to put him in one of those strollers for twins and then run around the mall looking frantic.
    10. Re:virtualization? by dtouchet · · Score: 1

      "Write up a paper/memo/whatever and send it to the guys who pay for the power over the possible cost reductions in power."

      We did just that a few years ago. It was justification for switching from CRT to LCD monitors.
      In one year we saved enough in electricity to pay for the difference in price. Like many businesses, our electricity costs are based on our highest month's bill. By reducing that we ended up saving for the whole year.

      We're regretting our move from a single IBM pSeries to 10 HP rack-mount servers because of the extra A/C we had to install to keep the temperature level. Engineering took a couple of months to install it. Southern Louisiana in the summer without enough A/C...not a pretty sight.

      You are right about idiots in the company. Our 'old' IT director kept shooting us down when we wanted to move to LCD's. The new guy took a real look and did it. Now he's getting attaboys for saving the company some real money.

      --
      void r() { printf("recursion is "); r(); }
    11. Re:virtualization? by Ironsides · · Score: 1

      Any chance your new IT director has enough "street cred" (sorry, can't think of a better term), to convince enough of the right people to consolidate and eliminate the old servers? I'd think that a modern server could replace at least 10 P2 machines and consume far less power.

      --
      Fly me to the moon Let me sing among those stars Let me see what spring is like On jupiter and mars
    12. Re:virtualization? by dtouchet · · Score: 1

      Unfortunately, this would require a lot of convincing. These servers run our mission critical 24/7 operations. They use Marathon solution for high availability and re-configuring them would be an big task. The last time we did a change (reconfiguring hard drive arrays), we had 2 hours of down time during the backup and restore and the Operations dept wasn't very happy. I'm going to try and talk him into replacing the servers with a single one next time we have a major software release, usually every 2-4 years. Hopefully virtualization will be even better by then.

      --
      void r() { printf("recursion is "); r(); }
    13. Re:virtualization? by petermgreen · · Score: 1

      However, what is keeping me from actually putting it to use is that when you put several different VMs inside one box, then all of those VMs can be taken down by a single failure (disk, power supply, nic, etc) that would have normally only taken down a single "real" machine.
      If you have lots of tasks that don't take much computing power by todays standards and you don't have a massive budget then you have two choices.

      1: put each one on it's own cheap shit box
      2: put them all on one higher grade box which has raid storage, redundant power supplies, a decent quality motherboard etc

      opton 1 means that only one service will go down at a time but you can almost gaurantee you will have failures frequently, option 2 allows you to spec out decent hardware that is far less likely to fail. However many services have strange requirements that make it difficult to agregate them, virtualisaion provides a way arround this.

      for big homogeneous workloads like say google has you have the option of designing your software so machine failures are tollerated but I can't see how you can do that with a mix of legacy applications.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    14. Re:virtualization? by suggsjc · · Score: 1

      you have the option of designing your software so machine failures are tollerated but I can't see how you can do that with a mix of legacy applications I really want to post my proposed architecture, but without going to too much detail, I am working on designing a system that uses mainstream software on commodity hardware in such a way as to break down each component into trivial tasks. Each of the tasks are stateless and therefore can be spread across different machines so that parallel requests (even by the same user) can be handled by different hardware components concurrently. By having at least 2 (more in most cases) machines that can perform each function of the stack combined with a load balancer (that can also detect failures) in front of each layer of the stack you can provide high availability as well. My entire system is designed so that hardware failures are not only allowed, but expected (although rarely).

      Now, what is somewhat interesting is being able to profile each piece of the stack and provision the correct amount of resources on a virtualized machine. Then, you can add capacity linearly by adding in a new piece of hardware that performs all roles of the stack (via VMs).
      --
      When I have a kid, I want to put him in one of those strollers for twins and then run around the mall looking frantic.
  4. a la simpsons by Anonymous Coward · · Score: 2, Funny

    EPA Official: You've gone mad with power!
    Russ Cargill: Of course I've gone mad with power! Have you ever tried going mad without power? It's boring and no one listens to you!

  5. Efficient design by Neil+Watson · · Score: 1

    When looking at all the servers in datacenters I often wonder how many servers are not actually needed. I do not mean redundant or due to be decommissioned servers. I mean there are to many dedicated servers. It is a common theme to have every service using a dedicated server. Is there any reason a server can not do more that just DNS? A properly built, configured and maintained server should be able to fill multiple roles thereby reducing space, power and cooling requirements.

    1. Re:Efficient design by Calinous · · Score: 1

      We have several web servers with IIS. Due to different software requirements (.Net 1.1 and .Net 2.0, some add-on or another), the functionality of the web servers is not guaranteed. Adding another web domain (developed with other tools) could be tricky.
            I would really like to have one server for each of the web sites running on Windows. Too bad virtualisation is out of discussion, as Windows is a big memory hog.

    2. Re:Efficient design by samkass · · Score: 1

      This is exactly the question answered by virtualization. Mixing lots of server processes on one OS instance makes it difficult to maintain status, monitoring, fault tolerance, etc. And before virtualization, a different OS instance required a different piece of hardware. When you hear about companies using VMware to save thousands on server power and air conditioning bills, this is where that savings is being realized.

      --
      E pluribus unum
    3. Re:Efficient design by LWATCDR · · Score: 1

      Well I question if the X86 is the way to go. This one size fits all mentality with CPUs needs to go. For things like web serving why not something like the SunT1 or T2. For NAS the PowerPC, ARM. and MIPS might be more power efficent.
      For render farms, database servers, and HPCs the X86-64 and Power5, UltraSparc T2.

      The X86-64 does a good job at about everything but it is not the best at anything. The new low power laptop cpus are not terrible but I don't think they can match the ARM, PowerPC, or Mips in the low power area yet.

      --
      See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
    4. Re:Efficient design by Anonymous Coward · · Score: 0

      There's a fundamental trade-off in these things. I work for a company which leases one dedicated server, which handles DNS/email/spamassassin/web/ftp/PHP/Perl CGI (some of this is *old*) for over 100 sites, some of which we are hosting for clients. Everything is on one box, going into one MySQL instance, etc., which makes it obnoxiously fragile: if service X goes down, it takes all the rest of them with it. And you can't clamp down too hard on service X to stop it from taking the box down, because some of the other sites may need more resources than the amount which guarantees the server keeps running if everyone uses up to the limit.

      With one box, you spend time either cleaning up crashes, or trying to prevent crashes and cleaning up the problems you caused by doing that. Reliability suffers; I bet our uptime is only two nines, if that.

    5. Re:Efficient design by silas_moeckel · · Score: 1

      Cramming lots of things onto one server running a single OS is generally problematic at best, it's the classic small IT shop mistake for long term reliability. Vmware can alleviate that by dedicating an OS to a function so you can do things like reboots without affecting a small pile of ancillary systems add a san and now your not adding piles of disks with each new server and can do vm level load balancing and clustering, have to replace some hardware move all the vm's off without any outage and fix the gear move them back when it's tested good. Windows servers are generally not good at running multiple anything, a unix can use it's better shared library system and chroot environments to run anything that can run under than kernel next to each other but VM's can do more and be easier to setup while consuming only a little more resources.

      --
      No sir I dont like it.
    6. Re:Efficient design by jmitchel!jmitchel.co · · Score: 1

      Why? Reliability!!!

      If every service / customer is independent of every other service / customer, then outages tend to stay small and simple. And with a simple, stupid datacenter your failure modes also tend to be simple and stupid. 99.999% uptime is often worth the cost to keep too many servers running.

    7. Re:Efficient design by SlamMan · · Score: 1

      I've found Windows to do better under virtualization than one would expect. ESX server does some nice tricks for shaving overhead amounts when you have multiple clients of the same OS.

      --
      Mod point free since 2001
    8. Re:Efficient design by sunking2 · · Score: 1

      For the most part this is due to software vendors not wanting to deal with support calls on more complicated systems. It's simply easier for them to support if you take out all of the variables introduced by other applications running on the machine.

      The simplest thing for them to do is say or recommend it goes on a dedicated server. Much less of a headache come support time. Of course businesses spend millions because of it, and Dell/HP/IBM rake in the bucks.

      I mean really, do license servers need to be dedicated? Not to mention the number silly little web apps that I've seen that management is told needs a dedicated server by the vendor. It's really pretty crazy.

  6. Not really by WindBourne · · Score: 2, Insightful

    The lifetime costs of the chips are what you can control directly. As it is, the manufacuering of the chip (and even the systems) are going to be relatively close to each in terms of energy. The CPU/GPU is the single largest means of our being able to control energy.

    --
    I prefer the "u" in honour as it seems to be missing these days.
  7. Don't worry by Colin+Smith · · Score: 2, Funny

    "The Singularity" predicts that processing power will continue to increase exponentially for ever. So obviously, electricity generation will also do the same. Not a problem.

    --
    Deleted
  8. Network Queue Systems by Colin+Smith · · Score: 2, Insightful

    Have been optimising server resource utilisation for decades.

    The real problem is that most I.T. staff are either as dumb as bricks and have no idea how to make use of one or have plenty of profit to burn and just don't care.

    --
    Deleted
    1. Re:Network Queue Systems by cerberusss · · Score: 1

      Network Queue Systems Have been optimising server resource utilisation for decades.
      Now, I've been in the IT industry for ~ 5 years now and I've never heard of something like "Network Queue Systems". And definitely not in connection to power savings. Could you expand on this a little more so I know what the +1 insightful was for?
      --
      8 of 13 people found this answer helpful. Did you?
    2. Re:Network Queue Systems by Colin+Smith · · Score: 1

      Now, I've been in the IT industry for ~ 5 years now and I've never heard of something like "Network Queue Systems". And definitely not in connection to power savings. They've been around since the early 1980s.

      See:
      http://www.google.co.uk/search?num=100&hl=en&safe= off&q=Network+Queueing+Systems&btnG=Search&meta=
      or
      http://en.wikipedia.org/wiki/Job_scheduler

      Modern free and commercial examples:
      http://gridengine.sunsource.net/
      http://www.cs.wisc.edu/condor/
      http://www.clusterresources.com/pages/products/tor que-resource-manager.php
      http://www.platform.com/Products/Platform.LSF.Fami ly/Platform.LSF/
      http://www.gridwisetech.com/content/view/123/90/la ng,en/
      http://www-03.ibm.com/systems/clusters/software/lo adleveler.html

      In a Unix server environment, pretty much any of the above can be used to run pretty much any application on the least loaded machine, including GUI/desktop apps or things like SQL queries and with a tiny bit of effort it can be made almost completely transparent. It means you can increase your server utilisation from 5% or less on average to around 90%. In a Windows server environment, you're pretty much fucked.
      --
      Deleted
    3. Re:Network Queue Systems by cerberusss · · Score: 1

      Great post! OK, I know this as a grid :-) Terminology changes (thanks, Markering), principles don't. I've played around with Oracle 10g, installing database "heads" on various machines where the database files themselves remain on a SAN. Same idea except more specialized than you meant.

      --
      8 of 13 people found this answer helpful. Did you?
    4. Re:Network Queue Systems by Anonymous Coward · · Score: 0

      In a Unix server environment, pretty much any of the above can be used to run pretty much any application on the least loaded machine

      Wouldn't you want to turn OFF your least loaded machines? Adding a little incremental load to already busy systems doesn't use as much power as the overhead of keeping a bunch of near-idle machines on.

  9. It's going to be interesting by HangingChad · · Score: 1

    If we ever run into wide-spread power availability issues. In the event of a natural or economic disaster, perhaps a series of them or we just degenerate into a civil war between political factions. No one ever imagines we could go through a near-collapse and fragmentation similar to the old Soviet Union.

    We'd likely have bigger worries than whether we could keep our data centers running but it's an interesting scenario to contemplate. I honestly had no idea data centers in the US consumed that much power.

    --
    That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
  10. DC power by rhaig · · Score: 2, Insightful

    everything uses DC internally. Some hardware allows for DC inputs. using DC across the board would greatly reduce cooling costs.

    --
    "We are not tolerant people. We prefer drastically effective solutions"
    1. Re:DC power by Wesley+Felter · · Score: 1

      OTOH, the cabling will be more expensive and complex. AFAIK there are no standard connectors for -48V DC.

    2. Re:DC power by pathological+liar · · Score: 1

      Sort of.

      The smaller gear we use has standard-ish connectors. Positive, negative and ground go into a terminal block, which in turn gets plugged into the device. That may just be a Cisco thing, I'm not sure -- even then, it's only on the relatively low draw devices with high gauge wire that use it.

      The bigger stuff is all lugs, which I guess you could call a standard, but it's a nightmare to wire. Cut and crimp positive, negative and ground, fight with heavy gauge wire, find suitable ground points, or make your own if you have to... and then rinse and repeat for the B feed.

      DC sucks.

    3. Re:DC power by javaxjb · · Score: 1
      --
      Programmers in mirror are brighter than they appear
    4. Re:DC power by Doctor+Memory · · Score: 2, Interesting

      Cut and crimp positive, negative and ground, fight with heavy gauge wire, find suitable ground points, or make your own if you have to... Nonsense. You just run the wires over to the buss bars that run between the racks, drill a new hold for each one and bolt it up. No muss, no fuss, and your wires are easy to manage. Plus, open buss bars hold serious comic potential if one of your cow-orkers likes to wear a big key ring on their belt...
      --
      Just junk food for thought...
    5. Re:DC power by TinheadNed · · Score: 1

      Hmm, I've had this suggested to me before, and the main thing that worries me (apart from MASSIVE cable losses - seriously, you work them out), is that when you turn something on or off that's really meaty you'd get some lovely magnetic fields developing. I've not checked the maths for that yet, because it's not quite back-of-envelope for me, but I'd want to double check them pretty urgently before suggesting this near your hard drives.

    6. Re:DC power by david.given · · Score: 1

      everything uses DC internally. Some hardware allows for DC inputs. using DC across the board would greatly reduce cooling costs.

      Weeeelll... not necessarily. When you start dealing with long wires, you end up having to deal with voltage drops across those wires. If your computer needs 5.000V to run reliably, you simply can't feed it with 5.000V produced by a power supply ten metres away, because by the time the electricity reaches the computer it won't be 5.000V any more.

      Which means you need to feed it with some other voltage, and convert it to 5.000V at the computer itself. Which in turn means either DC-to-DC converters, which are relatively inefficient, or AC-to-DC converters, which are far more efficient. Add to this that AC can be trivially stepped up or down in voltage by using a transformer, and that transmitting high-voltage AC is substantially more efficient than low-voltage AC, and suddenly the picture gets more complex.

      It may well turn out that it is more efficient to transmit your power to each chassis as 120 or 240V AC, and then convert it to 3.3V/5V/12V DC on the chassis.

      Alternatively, just use superconducting power cables and most of the problem goes away.

    7. Re:DC power by evilviper · · Score: 1

      using DC across the board would greatly reduce cooling costs.

      Except that it wouldn't...

      Switching (AC) power supplies have the potential to be just as efficient, if not more so, than DC power supplies.

      With a DC datacenter, you have to have a big central AC/DC converter, and then a bunch of DC/DC voltage converters. There's very little to gain, even in theory.

      In practice, you'd probably do far better if you took a fraction of the money it would cost to make a DC datacenter, and instead replace all the PSUs with more efficient models.
      --
      Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
    8. Re:DC power by TClevenger · · Score: 1

      That's what blades are for. Run a ton of machines off a pair of redundant power supplies.

  11. Google Cache by Anonymous Coward · · Score: 0

    According to the EPA, data centers -- not including Google et. al -- are on track to double power consumption in the next five years

    Google will just cache the power from the other data centers.

  12. bias by clarkn0va · · Score: 1
    Kudos to HP for taking the initiative on this, but one must question the validity of any benchmark designed by a corporation that will use the benchmark, whether it will be a true reflection of real world performance and efficiency, or just shine false light on HP's design practices.

    If nothing else, maybe this will spur the design of other relevant energy efficiency benchmarks.

    db

    --
    I am literally 3000 tokens away from the chaotic crossbow --Stephen
  13. Mainframes, anyone? by spookymonster · · Score: 1

    Seems to me someone left out another viable alternative.... especially since we just had that article last week about IBM consolidating several hundred PCs into a handful of Big Iron boxen...

    --
    - Despite popular opinion, I am not perfect.
  14. Power Metering in IT by P3NIS_CLEAVER · · Score: 1

    I think a better solution would be for corporations to meter, document and budget power expenditures in the server room. Once costs like this are recognized corporations will be more motivated to do something about it. You can justify the cost of switching to a 12 volt setup if you have the numbers to back it up.

    --
    Please sign petition to restore sanity to our banking system!!!

    http://financialpetition.org/
  15. Software by necro81 · · Score: 1

    Two sentences I would highlight from the StorageMojo article:

    1) Developers, the time may not too far away when your code is measured on power efficiency.
    2) Software effects will be found significant as well because widely used software affects so many systems.

    This reminds me of an article here on /., about how Microsoft could become the world's greenest company with a few small changes in code to be more aggressive about using power saving modes by default. Hardware makers have been harping on about power efficiency for ages, driven largely by the mobile computing and gadgetry market. As a result, there are a lot of things that the hardware can do to save power when under-utilized or idle, even in the server market. However, those features are only useful if the software (particularly OS and firmware) takes advantage of them. The more widespread the software, the greater the impact of incremental improvements in power-efficient code. A tiny fraction times GWh of energy is still a large amount. There are whole realms of power efficiency that have yet to be tapped.

  16. The case for Smart Appliances by maillemaker · · Score: 2, Interesting

    Years ago we heard how PCs were going to be embedded in everything from the dishwasher to the refrigerator, and I was left wondering, "why?"

    Perhaps now I know.

    It would be nice if I could set my house up on a "power budget", and let my appliances vie for electrical power and load-balance themselves to stay within that budget. If all appliances spoke over the in-house wiring (or perhaps wireless) and could turn themselves off or adjust their power usage that would be awesome.

    You could implement something similar to this today with an X10 system or the like, but this is more of an off/on scenario, and is not based on actual power demands.

    It would be great if all of my electrical things in my house could get together and say, "OK, guys, we have X amount of electricity to use today between all of us. Let's figure out, based on past usage patterns, who needs to be on and when in order to hit this budget".

    --
    A work that expires before its copyright never enters the public domain and thus enjoys eternal copyright protection.
    1. Re:The case for Smart Appliances by illumin8 · · Score: 1

      It would be great if all of my electrical things in my house could get together and say, "OK, guys, we have X amount of electricity to use today between all of us. Let's figure out, based on past usage patterns, who needs to be on and when in order to hit this budget".
      It sounds great now, but wait for the screams you hear from the shower when the hot water heater cuts out 2 minutes in...
      --
      "When the president does it, that means it's not illegal." - Richard M. Nixon
    2. Re:The case for Smart Appliances by RAMMS+EIN · · Score: 2, Funny

      ``You could implement something similar to this today with an X10 system or the like...''

      Dude, we've been using X11 for some time now! X10 has been obsolete for almost exactly 20 years...

      --
      Please correct me if I got my facts wrong.
    3. Re:The case for Smart Appliances by Crispy+Critters · · Score: 1

      It would be nice if I could set my house up on a "power budget", and let my appliances vie for electrical power and load-balance themselves to stay within that budget.

      I wonder if you are mixing up two things - power and energy.

      Power is important. Enough power plants and transmission capacity have to be built to handle the peak power load. Leveling out power usage can save money in construction costs and reduce the footprint of the electrical infrastructure.

      As individuals, most of us pay for electrical power by the kilowatt-hour regardless of when we use it (this is not true for everyone). Personally, I would like to reduce my energy usage, because it would save me money.

      There are lots of tricks to level out the electrical load. Many people do this anyway by running things like the dishwasher at night. Hot water heaters could be smarter - if everyone in the house takes showers in the morning, we could let the water cool down a little during the day and heat it back up again at 3 am. But none of this reduces the total energy use significantly.

      Other than adding insulation, buying more energy-efficient appliances, or staying dirty, it is a lot less clear what can be done to reduce energy use in a normal house. (There are lots of ways people waste small amounts of energy, leaving lights on and wall warts plugged in, but anyone can fix that if they care. CF bulbs may save a lot if everybody switches, but the effect on my bottom line is minimal.)

    4. Re:The case for Smart Appliances by ChrisMaple · · Score: 1

      Smart-Power appliances sound good until you consider that each of 50 appliances may be dissipating 5W while in standby. Oops! There goes another 2000 kW-hr per household per year.

      --
      Contribute to civilization: ari.aynrand.org/donate
  17. At the cost of massive transmission losses... by Anonymous Coward · · Score: 1, Insightful

    The only place DC power makes sense is large data centers, where AC is converted to DC in only a few places, instead of in each machine.

    That's because DC power distribution suffers from massive losses if it's transmitted across any decent distance.

    1. Re:At the cost of massive transmission losses... by mjgraham · · Score: 1

      What is it about DC that makes losses lower? AFAIK HVDC is used widely in power transmission, especially underseas where AC suffers losses due to induced currents in the water. I'd have thought corona losses would also be lower with DC, though correct me if I'm wrong.
      Modern solid state equipment means DC-DC conversion is more efficient than ever - AC was originally chosen because of how hard it was to convert DC between different voltages (the high ones required for transmission and the low ones required for it to be safe in the home), but this isn't true anymore.
      In fact, with the necessity of maintaining a good power factor, and power factor correction required in any sizable AC system, added to balancing phases to keep generation and transmission working efficiently, AC is something of a legacy standard.
      If the world needed to be wired today, I'd have to say DC would be a better choice.

    2. Re:At the cost of massive transmission losses... by mjgraham · · Score: 1

      note that this should read 'what is it about DC that makes losses higher'?

    3. Re:At the cost of massive transmission losses... by Big_Breaker · · Score: 1

      It isn't that it's DC. It is that the DC is low voltage compared to 110V AC. Wire losses are inversely proportional to the quare of voltage so 110 AC is about 100 times as efficient as 12V DC (Think I^2xR for the resistive power loss). It is very difficult to change the voltage of DC and very easy to do it with AC. For that reason high(er) voltage AC is preferred for power distribution.

      HVDC makes the most of the voltage limit in the wire insulation and the relative lack of inductive losses to ground from the magnetic fields emited by AC lines. The voltage quoted for AC is actually the DC voltage amount with equivalent power. The true AC voltage oscillates with an amplitude about 40% higher (and 140% lower) than the quoted figure. The peaks would exceed the insulation limits and corona discharge into the air.

    4. Re:At the cost of massive transmission losses... by aaarrrgggh · · Score: 1

      The only place DC power makes sense is large data centers, where AC is converted to DC in only a few places, instead of in each machine.


      Actually, traditionally the reverse is generally true-- it's only the small (That's because DC power distribution suffers from massive losses if it's transmitted across any decent distance.

      Low voltage infers high current; high current causes losses in the wiring. Traditionally, it was hard to convert DC voltages as is done in AC with transformers. This precluded having transmission, distribution, and utilization voltages in a DC system.

      Today, when there is talk of DC data centers, it is generally focusing on a distribution system at 300-1kV DC, with local converters at a rack level to go from the distribution voltage to -48VDC. The challenge is still that at a 48V level you are talking about up to 600A for a high density blade center rack. This creates all kinds of interesting fault and safety concerns, as well as getting back to the original problem of too much copper.

      The value of using DC distribution is that the AC-DC conversion is inherently inefficient. If you have a UPS, then you are doing an extra AC-DC rectification at the UPS front-end, plus a slightly more efficient DC-AC inversion at the output, plus the AC-DC conversion at the utilization equipment. In contrast, DC-DC voltage conversions can be almost as efficient as an AC-AC conversion. The cost of an efficient DC-DC conversion is significantly higher than the typical voltage regulator, where excess voltage is burned off as heat.
  18. Programming languages and system architecture by chthon · · Score: 3, Interesting

    Answer me this : how much power is lost through the use of inefficient programming languages and architectures which only emphasize processor speed, instead of balancing memory, processor and IO ?

    Python, Perl and PHP all suffer from one big drawback : when you scale up you need that much extra processor power. One programming language I know (Common Lisp) offers the advantages of them, but can be compiled to near C/C++ speeds. I suppose there are others. Don't come saying that programmers are expensive. It seems that what you gain on programmers, you lose in the cost of your datacenter. I don't know how Java matches here, it probably depends upon the deployment of more recent JIT compilers.

    If you see how much a process has to wait on IO, how come there are still no good solutions in providing enough IO bandwidth that the processor can use fully ? (Unless you buy a mainframe or iSeries system that is)

    Just asking.

    1. Re:Programming languages and system architecture by DamonHD · · Score: 3, Interesting

      As to Java: I have just moved a rack of (Solaris) servers @670W on to a single (Linux) laptop @18W (~25W from mains, but sometimes it runs off-grid on solar PV).

      http://www.earth.org.uk/low-power-laptop.html

      I actually now control the CPU-speed control with another small Java app (see update for 2007/08/20 on same page) and in particular watching it with strace() can't see the JVM doing anything that hand-crafted C wouldn't in the main loop.

      In fact, the whole machine, including several Java and static Web servers, sendmail, etc, comes in at under 1% utilisation.

      So Java and C at least are capable of being reasonably efficient providing the apps are written not to be profligate (I did a set of tweaks to minimise power-hungry CPU use unless there is 'excess'/spare power available from solar).

      Rgds

      Damon

      --
      http://m.earth.org.uk/
    2. Re:Programming languages and system architecture by Laser+Lou · · Score: 1

      That depends on the application. I do believe that most of the heavy processing in Python, Perl, and PHP apps normally occurs in the database, so they are probably responsible for most of the power consumption in those apps.

      --
      No data, no cry
    3. Re:Programming languages and system architecture by Laser+Lou · · Score: 1

      I should also add that if the database leaves any spare cycles, XML processing will use them up, long before the scripting code will.

      --
      No data, no cry
    4. Re:Programming languages and system architecture by chthon · · Score: 1

      What would be the key here ? Are there optimisations possible to speedup XML processing ? Can guidelines be written to enhance XML designs for speeding up processing ? Have there been profiling tests to see where in the XML processing the bottlenecks are ? Should you use XML even ?

    5. Re:Programming languages and system architecture by Dystopian+Rebel · · Score: 1

      I'm interested in knowing more about how you are booting from flash while mirroring to disk. Would you explain?

      --
      Rich And Stupid is not so bad as Working For Rich And Stupid.
    6. Re:Programming languages and system architecture by DamonHD · · Score: 1

      There is no live mirroring going on.

      The 4GB SD card was/is essentially dd-ed mirror of the first three hard disc partitions (/, /usr, + spare) sized to fit 4GB, though now it has drifted away as I have not applied all recent changes to the hard disc copy (manually, by mounting the partitions and hacking files with vi, etc).

      The idea is that if/when the SD card dies from too many writes (I have no idea how long this will take even though I have minimised writes) then I can boot off hard disc with minimal work to get right back up to date, and/or stick a new SD card in, dd the partitions onto it (ext3, BTW), do the small fixes, eg in grub and fstab, and reboot off the new SD.

      I have kept those changes that would not be trivial to re-apply in SVN on the hard disc.

      Rgds

      Damon

      --
      http://m.earth.org.uk/
  19. At least one company is tackling the issue by White+Flame · · Score: 1

    SiCortex

    They're more focused on computation than giant racks of storage, but their 2 systems are rated at max 3W/core total power consumption including drives, power supplies, interconnect, etc. I suspect the actual power draw will be much less.

    How much storage does a "typical" datacenter have? (I know any answer would have huge variances.) For probably over two million USD or so, you should be able to get their larger system with 8TB of RAM and run their RAM-based Lustre filesystem along with the HD storage. Even with more places getting into petabyte ranges, I suspect 8TB of RAM and 5832 CPU cores would make it fly (16TB on 11,664 cores at $4-5M or so for redundancy). To me, that cost looks like a whole lot less than the facilities construction, cooling, and administration of the typical rows of racks... if it would meet the need.

    (not affiliated with them; just a customer looking forward to their products)

  20. Some numbers don't jive by Anonymous Coward · · Score: 0

    Today, a typical data center with 1000 racks, consuming 10MW total power, costs $7M to power and $4-$8M to cool per year . . .

    So if there are 1000 racks that consume 10 MW of power, that means each rack consumes 10 W ? That seems a bit low. If you take the $7M power cost and do some division with power at about 7 cents per MW/hour to make the math easier, it comes out to about 33 W per rack, which is still a bit low. At 4 cents per MW/hour ( our local cost ) it comes to about 200 W per rack. That places the total MW of 1000 racks at 200 MW, not 10 MW.

    Is my math suspect, or some of their numbers ?

    1. Re:Some numbers don't jive by Wesley+Felter · · Score: 1

      You're confusing mega and kilo in several places.

    2. Re:Some numbers don't jive by belg4mit · · Score: 1

      Umm, you lost a few zeroes, that's 10 kW per rack.

      Total power is a misnomer though, since the broke out cooling separately.

      --
      Were that I say, pancakes?
  21. Re:Conservation means everyone is poorer. by Anonymous Coward · · Score: 0

    I like your style. Wealth and progress.

  22. Power Metrics? by porkrind · · Score: 1

    So, is there some kind of standard for data centers for computing power - some combination of raw processing and data throughput - per megawatt? I mean, is there some magic number that data centers should try to achieve? And if there is, how do you measure that?

    -John Mark

  23. Who is this article about? by icydog · · Score: 1

    According to the EPA, data centers -- not including Google et. al
    So data centers, excluding large data corporations, are going to use a lot of power? ???
    1. Re:Who is this article about? by belg4mit · · Score: 1

      More importantly, it's et al. , short for et alia (et alii, et aliae)

      --
      Were that I say, pancakes?
  24. 'dynamically' realocating servers by un1xl0ser · · Score: 1

    This was an idea that I had before at the last job, never got a chance to implement it. We had a lot of cluster nodes for computing various things that were business related. There was a queue of work, and they got assigned to nodes if it matched the right criteria. The issue is that we had a few things, like crunching financial data at month/year end which required us to have some database servers that were beefier than others.

    So the idea was to have systems re purposed automatically, using something like Oracle Rack 10G. You have a need for more database performance, you take unused nodes from your fluid dynamics processing pool, make them oracle nodes (took about 15-20 minutes to install RHEL 4) and go to town. This works a lot better with things like LDAP servers, web servers, et cetera, but we needed it for Oracle.

    So anyways, this would be one way of eliminating unused hardware, being nimble about how what application uses it. This also would be combined with shutting down cluster nodes that are not in use. Boot time is so cheap, why bother keeping them on.

    --
    v4sw6PU$hw6ln6pr4F$ck 4/6$ma3+6u7LNS$w2m4l7U$i2e4+7en6a2X h
    1. Re:'dynamically' realocating servers by Colin+Smith · · Score: 1

      Ah, a man after my own heart.

      Add VMware or Xen to the mix and you can pretty much get rid of the boot time as well as the install time. And if you have uniform hardware with LOM cards you can even automate the powering on/off of the base servers depending on the load of all the existing machines in the grid.

      --
      Deleted
    2. Re:'dynamically' realocating servers by un1xl0ser · · Score: 1

      Since the install time is so little, it didn't matter much to me. VMware and Xen add complexity that wasn't necessary in my ideal, plus those solutions were well not terribly well vetted in the enterprise when I came up with the idea 3 years ago.

      The real benefit to those is that if you have systems that don't need resources in large chunks (i.e. 2-4 dual core Opterons, 8-16 GiB of RAM). That chunk size seemed to be fine, and it was similar to what the DBAs were used to throwing into our V880s when they needed to get more power.

      The other aspect of this is you have to measure utilization very well. If someone isn't using their system, and doesn't have any specific hardware tie-ins, they get virtualized. If someone needs to grow a lot, very fast, they get new servers. Now that I think about it, security of the drives and data aside, this could work out well for a large managed datacenter. You don't need your servers, they are shut down. You also have your non-sold equipment on standby for surges in demand. Having that on tap in even 1/2 an hour would be fantastic.

      --
      v4sw6PU$hw6ln6pr4F$ck 4/6$ma3+6u7LNS$w2m4l7U$i2e4+7en6a2X h
  25. And that is why by drix · · Score: 1

    Sun Microsystems is going to make a killing over the next couple years. Say what you will about "red-shift" and all their other hokey marketese, they definitely got ahead of the curve when it comes to the idea that power, cooling and infrastructure costs and going to become limiting factors on business growth in the coming decades.

    --

    I think there is a world market for maybe five personal web logs.
    1. Re:And that is why by Anonymous Coward · · Score: 0

      Sun is okay, but Rackable Systems is really leading the way I think. They just won the LinuxWorld Best Product award for their great innovation in this field. They are a smaller player but have some big customers (Yahoo, Microsoft, Amazon, etc.) and if they can keep their innovation ahead of market, they are going to be making the killing, not Sun.

    2. Re:And that is why by Anonymous Coward · · Score: 0

      Yep - note the HP marketechture paper avoided actually testing any of the new Sun boxen. Fast/low power/small - MmMm good.. Saw one of the T2000's replace a whole rack full of HP blade servers as a web tier. Can't wait to run some tests on the new T2-based ones when they're out.

  26. How much would consumption decrease.... by r_jensen11 · · Score: 1

    ...if ./ and Digg went offline for a week?

    On a serious note, since I've got nothing else to do at work (as it's mind-numbingly boring), I was trying to figure out how many electrical plugs I would actually need to live a happy life (note, this is an extreme).

    My answer: 2

    One for the fridge, the other for a radio. Of course, it would be nice to have ceiling fans too, but those don't require power plugs, just direct wiring. Think about how many things you have plugged in to your house (e.g. those cell phone chargers you forget to unplug, how many clocks do you have in your house?, that entertainment system that never actually gets turned off but switched into standby) and compare that to what your parents went through when they were your age. And yet, somehow they managed without all of these doohickies that we have today.