Slashdot Mirror
Yahoo! Orders Wikipedia Hardware
Edit This Page writes "Jimmy Wales announced today that Yahoo! has ordered 23 HP servers for the Wikimedia Foundation. The three database servers are model DL 385, and will come with dual Athlons, 8GB of RAM, and 6x 146GB 15K RPM drives each. They will also provide rackspace and bandwidth. The announcement comes four months after Google's announcement of support, and two months after Yahoo's own. Google has not yet made their intentions clear. You can read more about the specifications of what will soon be a 100+ server cluster at the Wikimedia Servers wiki article."
As I write this, our developers are switiching the entire site over to Mediawiki 1.5 (from 1.4), and most of the changes will make it run faster. So we're lowering the per-transaction cost of the software and increasing the server capacity -- this is a good thing.
To make laws that man cannot, and will not obey, serves to bring all law into contempt.
--E.C. Stanton
Just think of all the links that get posted in slashdot to wikipedia and it doesn't falter under the load. That and it's not just static pages, between building, rebuilding, keeping reversion history, indexing for searches and constant slashdotting...
I am not in anyway affiliated with Max Cannon
Not Athlon
Actually, no, bandwidth (I'll assume here that you meant "throughput" ;-)) problems are not significant, it's much more the actual server hardware. Wikis are very database- and CPU-heavy.
James F.
'Cos Yahoo! offered to host them at their facility there, and our overall global reach has a bit of a paucity in Asia.
James F.
Well, first of all, everything grows. Number of user increases all the time - doubles every two or three months. Number of pageviews increases as well. And last but not the least, there are more and more, bigger and bigger articles with more and more of history. Wikipedia is growing and it is running on really low-budget hardware. And... every time we make a site running faster, more users come and use available resources. Therefore, we can do two things. Optimize our software platform and increase our hardware capacity. There are questions why are boxes added in Seul. We're trying to bring content as close to people as possible. Light speed means slow in information age. We already have donated cluster in Amsterdam, which serves all Europe, we want to have same or better capabilities in Asia. And sure, we're improving constantly our main cluster in USA. Why we really need that much cpus? Wiki means a website with a content that could be edited last second. It cann't be desynced, as editing outdated content isn't that sane. Also, it doesn't simply serve HTML content. In wiki all documents are related, links tracked, document quality observed, etc. Therefore, for a task, that might look quite simple, we need quite lots of servers. We could serve those poor 2500 requests / second (~1500 pageviews per second) with two or three web servers, but.. hey, EDIT THIS PAGE.
Also, in addition to HunterX11's comment, Wikipedia articles almost always have relevant links and sources listed. It's meant more as a starting point for research - it gives you a rather verbose summary of the information, and then points you in the right direction for more involved, serious research.
If you use it correctly, you won't find a better encyclopedia anywhere.
FWIW, they have Squid caches in front of the web farm, so there are cached static copies of busy pages.
Nothing to see here; Move along.
Up until recently when they moved to a new co-lo this data was out there, but it is unfortunately no longer available. I can say as a fact though that they are currently pushing out about 17 terabytes per month and growing strong. There's a bandwidth graph and instructions to read it on this page of my site.
python -c "x='python -c %sx=%s; print x%%(chr(34),repr(x),chr(34))%s'; print x%(chr(34),repr(x),chr(34))"
RTFA - it is the Wikipedia guys who are holding up Google's donation, not Google:
"Wikimedia's planned facilities in Amsterdam (The Netherlands), Belgium, and Asia are not online yet, so it would be premature at this juncture to ask Google for something specific when we don't yet have good technical knowledge of what we will need in the coming months following the introduction of these new facilities. Google are eager to help us, and Wikimedia are eager to accept their help, but the Board want to be good stewards of donor money, and this requires them to move carefully"
Both Yahoo and Google deserve approximately equal kudos for being helpful to the projects. Thanks!
Hello? Wikipedia keeps all versions of an article. Just refer to the specific version in that case.
No, you're thinking of the DL585, which is quad-Opteron (up to 8-cores). The DL385 is dual-processor (though you can install dual-core Opterons to get 4 total cores).
Here is their servers list.
Since I presumably have moderation to burn, I'll say frankly that I'm appalled. Wikipedia is enormously valuable as a resource in objective domains such as hard science and mathematics, but its articles in politically and culturally sensitive areas are abyssmal reflections of popular delusion and political correctness that do an enormous disservice to us all. The cockles of my heart not not warmed.
-I like my women like I like my tea: green-
That's a Wikipedia server admin that's speaking.
XML is like violence. If it doesn't solve the problem, use more.