Slashdot Mirror
Surveying the World of the Biggest Server Farms
1sockchuck writes "Rackspace said this week that it is managing more than 50,000 web servers, raising the question: who else has that many? Of companies that publicly discuss their server counts, there are only a handful that are near or above the 50,000 server mark, including 1&1 Internet, The Planet, and Akamai, as well as Rackspace. The larger totals are found among companies that don't discuss how many servers they're running. The leading suspects: Google, Microsoft, Amazon and eBay."
Figure servers per sq ft and add up their total datacenter sq footage. Googles a bit harder due to changing strategy over their current server lines but a good guesstimate shouldn't be too hard.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
They're using Netcraft to prove their server count - which reports on IP addresses. Just because there are 50,000 IP addresses responding to port 80, doesn't mean they have 50,000 boxes. The shared hosting arrangements can easily have dozens and dozens of "servers" operating on the same physical box.
Yes, it's still impressive... but not as impressive as it would first appear.
-- "Other than that, how was the play Mrs. Lincoln?"
Are you trying to show who has the biggest dick in the IT world ? :3
Thats right, me. In fact I just set up #50,001. You'll never guess what I'm doing with them either.
50,000 -- that's peanuts compared to the likes of Google or Yahoo etc... Here's a short article on the data center that Google is building (has built??) in Oregon.. http://harpers.org/media/slideshow/annot/2008-03/index.html
Seems to me the second largest search engine likely has 50k servers or more..
MABASPLOOM!
Googles been building out datacenter space like money is free.They sell blank space on web pages! I'd peg them at the top of the pile. Given their weird, packed like sardines servers they let the world know about they may be even topping a million if they finish up the new stuff in Europe and worldwide.
Yahoo would have to come in next and probably pretty close to Google. Microsoft would have a total pile but since they can't even do SSL on their update sites they are running cheap and probably have less than 300k even with hotmail. Amazon I can't even guess on. But I would put them behind Microsoft. Ebay? on par with Microsoft.
I'm sure some big porn websites(those that regroup many websites together especially) have quite a lot of servers...
has 11k servers or so.
Porn.
ok, so "managing more than 50,000 web servers"...
pretty sure:
IBM
EDS
CSC
Fujitsu
all trump that. Or did you mean "manage in a single pool" or some such?
Netcraft has developed a technique for identifying the number of computers (rather than IP addresses) acting as web servers on the internet, and attributes these computers to hosting locations through reverse DNS lookups."
Apparently it's not just the number of webservers (just IP addresses), but the number of physical boxes these guys are running. If Netcraft's technique is valid, then it could be helpful in determining the 'true' penetration of FOSS based server installs on the Internet. This could severely impact the ranking of sites that are hosted on certain proprietary OSes.
Sig this!
I have a lot of old 486's and Pentium's buzzing along doing their intended jobs (web server,ftp,IRC) running NT 3.51/4/*BSD/Linux.
Obviously, if you can count them, then you don't have all that many.
"A million here, a million there and pretty soon you have a real server farm."
I didn't know you could grow them. Have some spare seeds?
The good, the evil and the vacuum tubes.
Most of them won't go into detail, but Wall Street firms have immense server farms. Some of them are limited in size by the amount of electricity the New York City power grid can supply them. They also have huge data centers in less prime real estate, but microseconds are dollars in the financial markets, so they try to keep as many of their systems as close to the action as possible. There are entire floors of NYC skyscrapers full of racks modeling the financial markets in real time, conducting transactions, and crunching numbers for human analysts.
There's no failure quite as dissatisfying as a complete and total solution to the wrong problem.
Coming from the Goog. I would say 50 is a fucking joke... Laughable really. Try 6 times that and climbing every single day.
... Google how many servers it uses, does that mean it's self-aware?
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
I'm pretty sure the hosting company I had a few years ago (aka "kiddie hosting") had that many customers on the server that I was on. Does that count?
You don't use science to show that you're right, you use science to become right.
Hello,
I have worked supporting Google's servers in one of my former employers data center. What I can tell you about there deployments is as follows:
1) 20,000 Servers in our data center; they occupied 8 other sites (~160,000 servers). Our site was one of the largest.
2) Over 30 GigE connections feeding into dual Juniper M20 later upgraded to Juniper T-320
3) Yes they run a custom version of RH
Now for the record; they had approx 160,000 servers in our companies data centers. I have met techs from other data centers which had similar counts. At a minimum I can confirm approx 160,000 and potentially 320,000 and up for other data centers; providing they mirrored their servers. It wouldn't make sense to put all your eggs (servers) in one basket. The time frame for these numbers was back in the early 2002.
Sean Gorman mapped out the US fiber-optic telco fiefdoms, and mb spotted less redundancy than was claimed. :)
Parts of his dissertation where "removed".
He showed the choke points and critical links.
This plot of the large server areas would be fun to map and then visit.
Spot the NSA tap points
Domestic spying is now "Benign Information Gathering"
I heard you like to farm servers, so I put a server farm in your farm so you can service your servers while serving your farm.
Companies that have "real" computers - 32-cpus and more and have 10,000 of those are more impressive to me. Having 100,000 servers, all the same, is impressive, but still will use about the same management as 10,000 servers would.
I've deployed (3) 48-way servers with 2 for HA Oracle DBs and the other for DR and testing. Oracle RAC was the best in class at the time - grid didn't exist. I've deployed hundreds of custom servers (diff OS with diff required patches) running specialized applications from many, many vendors. Getting an application from SAIC or Telcordia or Teradata to work inside your normal infrastructure is harder than it sounds. Even for huge customers, they barely bend without huge payments and there usually isn't any competitive alternative.
Some workloads aren't worth hunting/designing ways to split up. Get over it and buy the big servers.
MS runs 160,000 servers? If they converted to Linux, that could easily be reduced to 10,000. ;)
My single group at a large IC manufacturer had over 1000 compute servers for simulation. My guess is one of the federal labs(LBL, LLNL, Oak Ridge) has the largest.
What are the current server specifications necessary to support 50K virtual instances of an O/S? How many boxes does that need to map down to in order to maintain sufficient redundancy and power efficiency? Now, how big a datacentre do you need to support that number of servers and hence what space/power/air conditioning requirements would you actually need?
Google's container-based datacentre is about 1300 servers. If each of those is capable of running multiple instances, that would bump up the total.
According to Google: in the time it takes to do a Google search, your own personal computer will use more energy than Google uses to answer your query. Another article points out that Google datacenters are more efficient than anything anyone else has published for a colo/datacenter.
Typical scare articles forget to divide by the number of users. For example, 50MW used to serve 10M users is 5w/user. Some TVs use that when they are "off".
Hey, why isn't IBM on their list. I doubt why it would need so many servers though. Any ideas?
they had 40K at the last official count and their new datacenter has a 50K capacity and filling quick (+3 bays/day 7/7). Not surprising given they offer the cheapest dedicated one can find ($15/mnth no contract: Atom 1.6Ghz, 512MB ram/2GB flash for swap, 10GB iSCSI disk, unlimited bandwidth).
have you been defaced today?
Never head of this company.
http://cgi.ebay.co.uk/ws/eBayISAPI.dll?ViewItem&item=110388976453
Obviously they are talking physical number of servers, because why in the hell would they list number of virtual machines in a financial report? Who would care about that?
While I can't speak for the others, I know Microsoft are growing their online infrastructure hugely now; at approx rate of "one facebook worth of servers every month" as one guy told me.
Most of it's consumer Live stuff, but there are plans to expand corporate services too so I understand.
throw new NoSignatureException();
Just in the DC where I work, one of many of our buildings, we have 750-1000 racks of servers. Even if each rack had only 50 discrete systems in it, we'd be past 50,000 at just my location. I think we have several other DC's bigger than this one, so we easily have hundreds of thousands of servers.
And having seen all of softlayer's Dallas areas going to go with large.
I imagine there is a fairly large server farm somewhere near Ft. Meade, MD.
[Insert pithy quote here]
All those WoW and Diablo fanboys have to go somewhere....
Donte Alistair Anderson Roberts - hi son!
Karma: Chameleon
There are several companies and industry conventions devoted to efficiently building massive server farms. I went to an oil industry convention session on these last year (The energy industry is 6th largest sector of supercomputing for seismic exploration, oilfield simulation and credit transactions.) Server farms consume vast amounts of power in the CPUs and air conditioning.
Modular expansion is fad. Server companies now recycle shipping containers as a row of racks with built in service, power distribution, and air conditioning. You have a certain rating of petabytes and kilowatts per shipping container. You start filling up a warehouse with these on a "as-needed" basis. Google is reputed to be at the top of this game.
Keep in mind that companies which use mainframes have seen a fairly large reduction in machine size over the past 20 years or so. Old water-cooled machines and rows of tape drives are being replaced with CMOS machines and tape silos, each taking of a fraction of the floor space that their predecessors did.
When I started working for Northwest Airlines, for example, there were multiple IBM and Unisys mainframes sitting in the main computer center, all residing in a large temperature-controlled raised-floor room.
These days, most of that floor space is empty. The number of mainframes hasn't been reduced at all ... they're just a lot smaller.
Mainframe/UNIX Bit Twiddler and long time Windows/Linux Hobbyist.
The Theorem Theorem: If If, Then Then.
Servers farms supply the commodity information for commerce and recreation in the early 21st century. Depending on what source you accept, they, along with client computers and video screens of all sizes, consume 3% to 10% of the US's energy.
...
I shamelessly stole this idea from Peter Huber of the Manhattan Institutes recent book The Bottomless Well. The trend of human commerce over the past couple centuries was to use more energy in more refined ways: wood, coals, whale-oil, petroleum, electricity, solar, computing, optical
I say early 21st century, because maybe some new discovery in computing technology or energy will drastically cut the energy consumption of yottaflops of computing before the century is over.
I had a phone interview with google around '05, I think, and was *told* by the interviewer that they had over half a million (physical) servers. I don't remember the exact number, but it was between 500k and 600k servers.
mark
I know the company I work for GSI Commerce has one of the largest networks in the US, behind (if I recall correctly) Google, Amazon, Wal-Mart and Ebay. Now networks are not servers, but they have well over 100 partners that they represent with many's websites hosted by GSI. But still probably not anywhere near 50,000.
Like a city whose walls are broken down is a man who lacks self-control.
runs on ~300 servers. you don't need millions of servers to reach everyone and/or be useful.