Surveying the World of the Biggest Server Farms

1sockchuck writes "Rackspace said this week that it is managing more than 50,000 web servers, raising the question: who else has that many? Of companies that publicly discuss their server counts, there are only a handful that are near or above the 50,000 server mark, including 1&1 Internet, The Planet, and Akamai, as well as Rackspace. The larger totals are found among companies that don't discuss how many servers they're running. The leading suspects: Google, Microsoft, Amazon and eBay."

Comments

50,000 web servers, not physical servers

[Amarok.Org]

They're using Netcraft to prove their server count - which reports on IP addresses. Just because there are 50,000 IP addresses responding to port 80, doesn't mean they have 50,000 boxes. The shared hosting arrangements can easily have dozens and dozens of "servers" operating on the same physical box.

Yes, it's still impressive... but not as impressive as it would first appear.

Re:50,000 web servers, not physical servers

[linhares]

They're using Netcraft to prove their server count - which reports on IP addresses. Just because there are 50,000 IP addresses responding to port 80, doesn't mean they have 50,000 boxes. The shared hosting arrangements can easily have dozens and dozens of "servers" operating on the same physical box.

Not exactly what netcraft is saying right now from TFAL:

Netcraft has developed a technique for identifying the number of computers (rather than IP addresses) acting as web servers on the internet, and attributes these computers to hosting locations through reverse DNS lookups.

I suspect there's a power law in here, and that the estimates of google and amazon and others should be way beyond this, perhaps surpassing 500.000. That would be an interesting milestone.

FTFA:

There's a widely circulated estimate of 450,000 servers, but that number is at least three years old. If it was ever accurate, it certainly isn't anymore, given Google's data center building spree. Google's recently revealed container data center holds more than 45,000 servers, and that's a single facility built in 2005.

Keyword: Evil

[zerosumgame]

50,000 -- that's peanuts compared to the likes of Google or Yahoo etc... Here's a short article on the data center that Google is building (has built??) in Oregon.. http://harpers.org/media/slideshow/annot/2008-03/index.html

Re:Keyword: Evil

[wwwillem]

> By now could be the order of millons.

Forget about "could be". According to IDC (feb 2009):

"According to IDC's Worldwide Quarterly Server Tracker, [...] 2008 worldwide unit shipments grew 2.0% to 8.1 million units. [...]" [http://www.idc.com/getdoc.jsp?containerId=prUS21703309]

With 8 million units cranked out in just a single year, there must be 30+ million servers "up and running" in current datacenters.

Which is in sync with my experience with our customers. Ten years ago (which was the time of the dot-com :-) the number of servers decreased heavily caused by consolidation to big servers, etc., resulting in half empty DC rooms. The last few years, because of the pervasiveness of IT in our lives, but likely also because of the adoption of Windows as a server platform, the number of servers has sky-rocketed and everybody is now struggling with space, power and cooling problems in their datacenters.

Yahoo?

[neoform]

Seems to me the second largest search engine likely has 50k servers or more..

just look at the cooling equipment

[Anonymous Coward]

Can't say for sure but looking at cooling equipment from aerial views, you can get a good idea. A datacenter with nearly as real estate in cooling as floor has some serious stacks in place or planned.

Googles been building out datacenter space like money is free.They sell blank space on web pages! I'd peg them at the top of the pile. Given their weird, packed like sardines servers they let the world know about they may be even topping a million if they finish up the new stuff in Europe and worldwide.

Yahoo would have to come in next and probably pretty close to Google. Microsoft would have a total pile but since they can't even do SSL on their update sites they are running cheap and probably have less than 300k even with hotmail. Amazon I can't even guess on. But I would put them behind Microsoft. Ebay? on par with Microsoft.

What about porn?

[Mishotaki]

I'm sure some big porn websites(those that regroup many websites together especially) have quite a lot of servers...

Re:Shouldn't it be easy to figure out?

[JWSmythe]

    That's going to come up with some off numbers.

    There are oddities in there. Like, Google has some decent square footage with QTS in Atlanta.

    Is the number, the number of machines in house, or the number of machines managed by the company in house? Places like Equinix have huge facilities, but they don't manage them (except for helping hands support).

    Just because a company takes a bigger footprint, does that make them bigger? My old place had an Alexa rank in the top 300, but the main sites were served from maybe 24 machines set up for well over 10 million users/day, not including hot spares and ancillary equipment (DNS, mail, etc). That was unique users, not requests or page views. :)

    I know Quantcast has a huge footprint, but they're in someone elses facility. I think it was only one of a few DC's that they're in. I didn't know it was their equipment until I talked to one of their techs. It was a very nice setup. The conversation with my coworkers went "I want to set our stuff up like theirs. Too bad we have dissimilar equipment, it'll never look so good."

    Depending on who you're looking at, the footprint isn't the front end either. Places like Google, Quantcast, and many others have a LOT of non-public equipment for doing the fun stuff. Like Google has a slew of spiders crawling, analyzing, etc. It's perfectly likely the could (not necessarily do) get away with just a few dozen front end machines for Google.com, and pass the work off to back end machines.

    It's all in the strategy that they use. If you're a multibillion dollar operation, do you squeeze every bit of power you have out of a machine, or stay real low and spread it across many? For my old place, I set up to squeeze everything I could out of them, and then spread out across machines so we could lose machines (hardware failures usually) without hurting the site. We didn't have the budget (the bosses liked the profit), but if I had really wanted, I could have probably spread it across thousands of machines. It just makes for headaches and larger IT staffs to keep up with it.

Re:2 CPU servers are a joke

[lintux]

And you seriously think that Google (or any other bigger company) is still using 2-CPU servers?

Re:Shouldn't it be easy to figure out?

[xp]

Also hopefully they are not counting virtual machines here.
--
Slow Poke

Re:Financial Services Industry

[drinkypoo]

Using flawed algorithms and crap assumptions as input. See where that got us.

These people don't have "maintaining a stable economy" as a goal. They have "making more money than the other guy". If the market drops more than they do, they consider themselves winners. They can trade into a position that will come up! Who cares what that fluctuation in the market means?