Slashdot Mirror
Researching the Slashdot Effect?
leonia asks: "At Columbia University, we are investigating methods and software to automatically deal with sudden, unpredictable load spikes ala the 'Slashdot effect'. We are looking for web sites that have been the 'victim' of this effect who would be willing to share anonymized web logs with us. If you know of a site that went under after it was mentioned on Slashdot or similar site or are the proud owner of such a site, we'd love to talk to you."
title says it all?
Slashdot gets slashdotted regularly. They just know how to handle it.
Ever notice how sometimes, the front page doesn't have your login information? Or the post count doesn't get updated? You're being served a static page for one reason or another.
The simple fact of the matter is that most of the 'database driven' sites, which is where you get the 'mysql socket error' or 'too many sql server connections' or whatever problems, don't need to be database driven. You don't write a static article, put it in a database, then hit that database every bloody time somebody wants to read it. Put a copy in a variable, display that, refresh the variable every five minutes. Or even every minute. Or even ever ten seconds. You'll still never get more than six connections a minute, at that rate, and the information is just as current and fresh.
Vintage computer games and RPG books available. Email me if you're interested.
The "throttle" module you refer to is a mechanism to detect surges in traffic, and to allow other modules to thus intelligently and automatically react. For example, the "statistics" module will automatically stop some of its logging and statistical analysis during heavy loads. In any case, this is new to Drupal, to be released with 4.1 (otherwise currently available in the development version from CVS). I hope in time other modules will utilize this logic. (I.E., themes could automatically become simpler under heavy loads, fancy features requiring lots of CPU could be temporarily disabled or reduced, etc) For more information read the help page.
I have written another module, called "filecache" which does what you have described. That is, it simply caches pages to the filesystem. If Slashdot results in MySQL choking, the site will continue to function thanks to the static cache files, automatically resuming dynamic generation after the rush is past and MySQL is able to respond to queries again. The module and patches to Drupal can be viewed here. I'm adding some more features to this module, but the core is done and seems to work quite nicely. (Additionally, with pages cached to the file-system I'm seeing a significant performance boost)
Drupal is an excellent engine, very modular in design and easy to work with. More, it's backed by a team of friendly and helpful developers. I made the switch after performance issues with PHP-Nuke, and I've not looked back. And it just keeps getting better...
-Jeremy
Try some Google searches similar to this one that will locate websites that have archived their hit statistics including the referring page information. I'm sure if you go through enough of them you will find a bunch that were seriously slashdotted during the time period those statisitics cover. Plus when you find them you will be able to put them in context of how many hits they are getting from sites other than slashdot, and if they have the stats for other months/weeks so you will be able to really put them in context of what level of traffic they normally get.
Note: make sure you check Google's archive of the pages for those that aren't static.
Work for Change & GET PAID!