Slashdot Mirror


Surveying the World of the Biggest Server Farms

1sockchuck writes "Rackspace said this week that it is managing more than 50,000 web servers, raising the question: who else has that many? Of companies that publicly discuss their server counts, there are only a handful that are near or above the 50,000 server mark, including 1&1 Internet, The Planet, and Akamai, as well as Rackspace. The larger totals are found among companies that don't discuss how many servers they're running. The leading suspects: Google, Microsoft, Amazon and eBay."

36 of 106 comments (clear)

  1. Shouldn't it be easy to figure out? by afidel · · Score: 2, Interesting

    Figure servers per sq ft and add up their total datacenter sq footage. Googles a bit harder due to changing strategy over their current server lines but a good guesstimate shouldn't be too hard.

    --
    There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    1. Re:Shouldn't it be easy to figure out? by confused+one · · Score: 2, Informative

      Doesn't always work. A lot of datacenters are not filled out because the power density of racks of modern servers is so high it can result in the datacenter exceeding the available electric service, if they filled out all of their space..

    2. Re:Shouldn't it be easy to figure out? by afidel · · Score: 2, Interesting

      Should still get you in the right ballpark, few companies are going to waste huge amounts of money building empty datacenters =)

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    3. Re:Shouldn't it be easy to figure out? by JWSmythe · · Score: 3, Insightful

          That's going to come up with some off numbers.

          There are oddities in there. Like, Google has some decent square footage with QTS in Atlanta.

          Is the number, the number of machines in house, or the number of machines managed by the company in house? Places like Equinix have huge facilities, but they don't manage them (except for helping hands support).

          Just because a company takes a bigger footprint, does that make them bigger? My old place had an Alexa rank in the top 300, but the main sites were served from maybe 24 machines set up for well over 10 million users/day, not including hot spares and ancillary equipment (DNS, mail, etc). That was unique users, not requests or page views. :)

          I know Quantcast has a huge footprint, but they're in someone elses facility. I think it was only one of a few DC's that they're in. I didn't know it was their equipment until I talked to one of their techs. It was a very nice setup. The conversation with my coworkers went "I want to set our stuff up like theirs. Too bad we have dissimilar equipment, it'll never look so good."

          Depending on who you're looking at, the footprint isn't the front end either. Places like Google, Quantcast, and many others have a LOT of non-public equipment for doing the fun stuff. Like Google has a slew of spiders crawling, analyzing, etc. It's perfectly likely the could (not necessarily do) get away with just a few dozen front end machines for Google.com, and pass the work off to back end machines.

          It's all in the strategy that they use. If you're a multibillion dollar operation, do you squeeze every bit of power you have out of a machine, or stay real low and spread it across many? For my old place, I set up to squeeze everything I could out of them, and then spread out across machines so we could lose machines (hardware failures usually) without hurting the site. We didn't have the budget (the bosses liked the profit), but if I had really wanted, I could have probably spread it across thousands of machines. It just makes for headaches and larger IT staffs to keep up with it.

      --
      Serious? Seriousness is well above my pay grade.
    4. Re:Shouldn't it be easy to figure out? by 0x15 · · Score: 2, Interesting

      Many of the production data centers I've worked on have been using VMs for web servers for quite some time (e.g. ESX 2.x). That includes one of the companies that is on the list of unknowns in the article. I'm surprised it wasn't addressed and that so many jumped to a physical server conclusion. However, even if the 50k servers are all VMs, that's a major management load. 50k of just about any configuration item takes some work.

    5. Re:Shouldn't it be easy to figure out? by Gorobei · · Score: 4, Interesting

      A lot of datacenters get built in repurposed buildings - the square-footage is often misleading (some are even 60's era compute farm housing - 90% of the space may be unused.)

      For low-latency datacenters, you build in the middle of cities. Then you find square-footage really doesn't cover it: the fire-marshall shows up and red-tags you because he doesn't want a six mega-watt dense power sink in the middle of his premium real-estate.

    6. Re:Shouldn't it be easy to figure out? by xp · · Score: 4, Insightful

      Also hopefully they are not counting virtual machines here.
      --
      Slow Poke

    7. Re:Shouldn't it be easy to figure out? by hardwarefreak · · Score: 2, Interesting

      Also hopefully they are not counting virtual machines here.
      --
      Slow Poke

      It's almost guaranteed that they are talking virtual servers as well as some bare metal. Rackspace is talking about the 50,000 web servers that *THEY* manage--i.e. managed services. This figure likely doesn't include the thousands of rented colo machines which their customers manage themselves. The latter was Rackspace's first business model, a pure colo. It wasn't until relatively recently that they started offering managed services. It may very very turn out that the 50,000 server figure is the entire sum total of physical servers on their network.

  2. 50,000 web servers, not physical servers by Amarok.Org · · Score: 3, Insightful

    They're using Netcraft to prove their server count - which reports on IP addresses. Just because there are 50,000 IP addresses responding to port 80, doesn't mean they have 50,000 boxes. The shared hosting arrangements can easily have dozens and dozens of "servers" operating on the same physical box.

    Yes, it's still impressive... but not as impressive as it would first appear.

    --
    -- "Other than that, how was the play Mrs. Lincoln?"
    1. Re:50,000 web servers, not physical servers by linhares · · Score: 3, Insightful

      They're using Netcraft to prove their server count - which reports on IP addresses. Just because there are 50,000 IP addresses responding to port 80, doesn't mean they have 50,000 boxes. The shared hosting arrangements can easily have dozens and dozens of "servers" operating on the same physical box.

      Not exactly what netcraft is saying right now from TFAL:

      Netcraft has developed a technique for identifying the number of computers (rather than IP addresses) acting as web servers on the internet, and attributes these computers to hosting locations through reverse DNS lookups.

      I suspect there's a power law in here, and that the estimates of google and amazon and others should be way beyond this, perhaps surpassing 500.000. That would be an interesting milestone.

      FTFA:

      There's a widely circulated estimate of 450,000 servers, but that number is at least three years old. If it was ever accurate, it certainly isn't anymore, given Google's data center building spree. Google's recently revealed container data center holds more than 45,000 servers, and that's a single facility built in 2005.

    2. Re:50,000 web servers, not physical servers by nedlohs · · Score: 4, Informative

      If you bother to RTFA you'll see it isn't a netcraft server count, but a mention in their SEC reported earnings something they are unlikely to just make up.

      The comparisons are with netcraft numbers. And those netcraft numbers are explicitly not IP address counts and have rackspace as way under 50,000 (which you would expect since many machines wouldn't be web servers (database backends, mail servers, etc) and many would be firewalled to not allow public access.

    3. Re:50,000 web servers, not physical servers by ZosX · · Score: 2, Interesting

      4 years ago. I wonder if all those boxes are still running right now? I wonder what google does when it retires servers.....it would be kind of cool to have a couple of bonafide google racks doing something cool at my house.

    4. Re:50,000 web servers, not physical servers by m.ducharme · · Score: 4, Informative

      If this is to be believed, they salvage and rebuild as much as they can, and send the rest to recycling.

      --
      Rule of Slashdot #0: You and people like you are not representative of the larger population. - A.C.
  3. Keyword: Evil by zerosumgame · · Score: 2, Insightful

    50,000 -- that's peanuts compared to the likes of Google or Yahoo etc... Here's a short article on the data center that Google is building (has built??) in Oregon.. http://harpers.org/media/slideshow/annot/2008-03/index.html

    1. Re:Keyword: Evil by wwwillem · · Score: 2, Insightful

      > By now could be the order of millons.

      Forget about "could be". According to IDC (feb 2009):

      "According to IDC's Worldwide Quarterly Server Tracker, [...] 2008 worldwide unit shipments grew 2.0% to 8.1 million units. [...]" [http://www.idc.com/getdoc.jsp?containerId=prUS21703309]

      With 8 million units cranked out in just a single year, there must be 30+ million servers "up and running" in current datacenters.

      Which is in sync with my experience with our customers. Ten years ago (which was the time of the dot-com :-) the number of servers decreased heavily caused by consolidation to big servers, etc., resulting in half empty DC rooms. The last few years, because of the pervasiveness of IT in our lives, but likely also because of the adoption of Windows as a server platform, the number of servers has sky-rocketed and everybody is now struggling with space, power and cooling problems in their datacenters.

      --
      Browsers shouldn't have a back button!! It's all about going forward...
  4. Yahoo? by neoform · · Score: 2, Insightful

    Seems to me the second largest search engine likely has 50k servers or more..

    --
    MABASPLOOM!
    1. Re:Yahoo? by eldorel · · Score: 4, Informative

      Ask, and ye shall receive...
      US_realm_list_by_datacenter

  5. What about porn? by Mishotaki · · Score: 3, Insightful

    I'm sure some big porn websites(those that regroup many websites together especially) have quite a lot of servers...

  6. From TFA by actionbastard · · Score: 2, Informative

    Netcraft has developed a technique for identifying the number of computers (rather than IP addresses) acting as web servers on the internet, and attributes these computers to hosting locations through reverse DNS lookups."

    Apparently it's not just the number of webservers (just IP addresses), but the number of physical boxes these guys are running. If Netcraft's technique is valid, then it could be helpful in determining the 'true' penetration of FOSS based server installs on the Internet. This could severely impact the ranking of sites that are hosted on certain proprietary OSes.

    --
    Sig this!
  7. Re:just look at the cooling equipment by afidel · · Score: 2, Informative

    Uh, MS doesn't do HTTPS on their update site because they do PKI signing on the update package themselves so they don't need to.

    --
    There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
  8. Re:analogy by Anonymous Coward · · Score: 5, Funny

    Are you trying to show who has the biggest dick in the IT world ? :3

    No, didn't you read the summary? We're trying to find out who has the biggest rack.

  9. Re:just look at the cooling equipment by Achromatic1978 · · Score: 4, Informative

    MS has approximately 160,000. When I was there a year ago, I did a tour of one of their test datacenters... (even regular staff don't get to tour production), and they remarked that they'd recently turned on their 150,000th.

  10. Re:Who else? ME. by Mad+Merlin · · Score: 2, Funny

    Thats right, me. In fact I just set up #50,001. You'll never guess what I'm doing with them either.

    Do you really need that much porn?

  11. Financial Services Industry by Chris+Snook · · Score: 5, Interesting

    Most of them won't go into detail, but Wall Street firms have immense server farms. Some of them are limited in size by the amount of electricity the New York City power grid can supply them. They also have huge data centers in less prime real estate, but microseconds are dollars in the financial markets, so they try to keep as many of their systems as close to the action as possible. There are entire floors of NYC skyscrapers full of racks modeling the financial markets in real time, conducting transactions, and crunching numbers for human analysts.

    --
    There's no failure quite as dissatisfying as a complete and total solution to the wrong problem.
    1. Re:Financial Services Industry by drinkypoo · · Score: 2, Insightful

      Using flawed algorithms and crap assumptions as input. See where that got us.

      These people don't have "maintaining a stable economy" as a goal. They have "making more money than the other guy". If the market drops more than they do, they consider themselves winners. They can trade into a position that will come up! Who cares what that fluctuation in the market means?

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  12. Re:Who else? ME. by linhares · · Score: 4, Funny

    he's got an 8-screen setup

  13. If you ask.. by MobileTatsu-NJG · · Score: 4, Funny

    ... Google how many servers it uses, does that mean it's self-aware?

    --

    "I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)

    1. Re:If you ask.. by drmofe · · Score: 2, Funny

      Wait until next week and then ask Wolfram Alpha.

  14. Two hosting providers ago... by Leomania · · Score: 3, Funny

    I'm pretty sure the hosting company I had a few years ago (aka "kiddie hosting") had that many customers on the server that I was on. Does that count?

    --
    You don't use science to show that you're right, you use science to become right.
  15. Google Data Center Tech by Anonymous Coward · · Score: 5, Informative

    Hello,
    I have worked supporting Google's servers in one of my former employers data center. What I can tell you about there deployments is as follows:
    1) 20,000 Servers in our data center; they occupied 8 other sites (~160,000 servers). Our site was one of the largest.
    2) Over 30 GigE connections feeding into dual Juniper M20 later upgraded to Juniper T-320
    3) Yes they run a custom version of RH

    Now for the record; they had approx 160,000 servers in our companies data centers. I have met techs from other data centers which had similar counts. At a minimum I can confirm approx 160,000 and potentially 320,000 and up for other data centers; providing they mirrored their servers. It wouldn't make sense to put all your eggs (servers) in one basket. The time frame for these numbers was back in the early 2002.

  16. Re:just look at the cooling equipment by CharredMetal · · Score: 4, Informative

    Microsoft would have a total pile but since they can't even do SSL on their update sites they are running cheap and probably have less than 300k even with hotmail

    The update protocol does winhttps (SSL). The actual file downloads are simple winhttp, since they are signed.

  17. Re:2 CPU servers are a joke by lintux · · Score: 2, Insightful

    And you seriously think that Google (or any other bigger company) is still using 2-CPU servers?

  18. Re:Who else? ME. by Joebert · · Score: 4, Funny

    There's an octo-mom joke in there somewhere.

    --
    Wanna fight ? Bend over, stick your head up your ass, and fight for air.
  19. Not Web Servers, but ... by rlp · · Score: 2, Funny

    I imagine there is a fairly large server farm somewhere near Ft. Meade, MD.

    --
    [Insert pithy quote here]
  20. Re:Who else? ME. by tuxicle · · Score: 2, Funny

    ...maybe if it said "eight-head" setup?

  21. Re:1&1 by rbrausse · · Score: 3, Informative

    sorry for the second reply but according to the homepage ( http://www.1und1.com/ [in German]) the company has over 7.2 million customers in Germany, UK, France, Spain, Austria and the USA running on more than 55k servers.