Slashdot Mirror
Surviving Slashdotting with a Small Server
S.BartFarst writes "Our little departmental server has been slashdotted twice in the last year and survived! Implementation of a two-headed redundant hardware scheme using linux virtual server and backup and failover capabilities enhanced by the linux high-availability tools has produced a nifty low-cost solution. Gotta love those little white boxes!
(also having a university-supplied BIG PIPE doesn't hurt). More interesting is the documentation of the apparent exponentially decaying attention span of slashdotters. Anybody else observed similar phenomena?"
Our little departmental server has been slashdotted twice in the last year and survived!
Wait... is this a challenge?
Mike
to post anything, article wise or mirror, I think it'll bring my little ol' server to it's knees...
plus having only a 384/384 pipe doesn't help.
They are asking for another test.
What's in a sig?
Third time's a char... DOH!
Heh, well they're actually still kicking, oh well, it would've been so apropos.
Okey people, let's prove them wrong! ;-)
Open the links in the story twice from now on
Note to self: get smarter troll to guard door.
Thou shall not survive thrice. You're insolence will not be tolerated. You'll servers will suffer a slashdotting not hence seen....
Notice this comment was posted on a slow Sunday afternoon (EST). Very clever, because they know that /.'ers can't resist a challenge like that. Feel sorry for them on Monday morning though...
I was under the impression that a 20k fiber or 100mbs one that can dynamically shift traffic would be needed.
http://saveie6.com/
well there you go... having a massive amount of bandwidth will allow you to survive a slashdotting. In most cases of slashdotting, I dont think the server was the bottleneck... its no problem for a server to dish out static pages... its the bandwidth, especially for serving pictures or videos....
Or is it where the article is at any given time? Top of front page gives lots of hits. As it drifts down, the hits slow as fewer read; to the sidebar, fewer but still substantial hits; then off to the specialty pages such as Science or Games, then only a few will read.
Of course, the only test would be to repost the article and see if there's the same number of hits... Nah, slashdot would never go for duplicate stories.
Interesting how it peaks, drops off slowly then rises again a little before dropping off again. Maybe some 'behind the curve' slashdot readers?
I've got some exponentially decaying pieces of chicken on my table...
and some exponentially growing forms of life in some beer cans...
does that count?
PC moderators can suck my White pierced, tattooed dick. If you think pride == hate, s/dick/Aryan meat mallet/g.
I only wish I had enough bandwidth to be able to test the hardware out properly.
2Mb? It's nothing for a nice server.
Get your own free personal location tracker
More interesting is the documentation of the apparent exponentially decaying attention span of slashdotters.
Well, I was gonna reply, but I forgot what the post was about.
I hope the dupe jokes that get posted are better than what I had here before I decided to just post this in hopes of drawing better jokes later in the thread. They weren't good! But when are they?
This is almost as stupid as wearing a mask of Donald Rumsfeld and visiting 3ID troops to tell them "Are we planning on letting you go home anytime soon? Gosh, no!"
These SMU guys are obviously begging for it, I say we give them hell!
[o]_O
Just under 40,000 hits in the busiest day... this is a slashdotting? Come back when you get into the millions. :)
hey look at this mpeg about rocks or somthing!!! MUAHAHAHAHAA
Anybody else observed similar phenomena?
Nope. In our jobs they make us do work.
There is a total of 80K of information on the entire front page! Dig a little deeper guys and perhaps we can find a few gigantic image downloads. If you find any do share =)
From excellent karma to terible karma with a single +5 funny post...
While your setup may make you real safe from machine outages, the effects of a slashdotting are to flood your resources rather than break them. So your configuration gives you at best the performance of two machines instead of one - which you could also have achieved by just ramping up the CPU or memory.
[x] auto-moderate all posts by this user as insightful
Its incredible, this person has actually proven that LOAD BALANCING MULTIPLE SERVERS INCREASES YOUR LOAD CAPACITY! This is incredible news! Wow, I am sure glad it made it as an article, stunning.
Every medium to large website out there will be pleased to know that what they have been doing for the last 8 years is actually VALID, thanks guys!
I think the only reason this made it to the front page is the slashdot self-reference.
I'll bet if you chart the data hour-by-hour, you'll see a sudden dropoff at the very moment the story scrolls off.
Woudn't that be a logarithmic decay rate?
Patent: from Latin patere, to be open
I always figured the majority of slashdotting occured because of bandwidth constraints. I suppose a few servers crapped out, but that has got to be few and far between, hasn't it?
Don't let me crap all over your cutesy linux saves the day story, though.
Huh?
It's also interesting that there was a second little bump about a week later. Anyone have any ideas why?
had to be said... sorry.
DT
Is this thing on? Hello?
You didn't get Slashdotted if the server was still operating normally. You just had some people from Slashdot visit.
May we never see th
More interesting is the documentation of the apparent exponentially decaying attention span of slashdotters.
I think this is why Slashdot makes you wait 20 seconds before you can submit a comment. They figure this will weed out 80% of the responses as the people get bored of waiting for the 20 seconds to be up and they just leave.
$ ping -t www.geology.smu.edu
Pinging geology.heroy.smu.edu [129.119.223.84] with 32 bytes of data:
Request timed out.
Request timed out.
Reply from 129.119.223.84: bytes=32 time=1317ms TTL=240
Request timed out.
Reply from 129.119.223.84: bytes=32 time=1254ms TTL=240
Request timed out.
Reply from 129.119.223.84: bytes=32 time=1538ms TTL=240
Request timed out.
The unofficial
What I want to know is, how fat a pipe do you need to survive a slashdotting, given that your server structure is viable? Will a 10mbps pipe keep the barbarians from trampling the gate?
Lets help them out.
:; do wget http://www.geology.smu.edu/~dpa-www/venus/mpeg/atl a1.mpg -O /dev/null -o /dev/null ; done
while
Don't forget to fix the space in the URL.
Get your own free personal location tracker
Multi-sensor.tar.gz
Try a hand at downloading the above file from the site (113MB) and see what transfer rate you get?
From excellent karma to terible karma with a single +5 funny post...
Since they seem to think they can handle the slashdotting, I've taken the liberty and searched through their site. This page has a number of links to pages which have large images and movies linked to. Hopefully, this will really provide them some load.
There's no sig like SIGSEG
Now if you could just automate the conversion of the httpd usage data over time, then convert that into the right scale, put it out through a ttyx, whereupon it could enjoy a leisurely conversion, D to A, and then connect the analog output to a proportional low-frequency-compatible amplifier, in turn, to a large diameter speaker in the back of a garbage can, the other end of which has a moderate-to-small orfice in it, then the thing could blow more smoke, as per previous article, than all of us /. blowhards combined. ;-) (Just a thought.)
My server has been slashdotted a few times and I can tell you it's pretty simple to not get overloaded.
The first time I learned my lesson. The server was on a T1 line that was 2/3 full already, and slashdot linked to a page full of large photos. That'll kill your link pretty quickly. Low-budget solution: sign up for a burstable web hosting account somewhere and just put all your large images there.
Later when we got some actual office space for the business, I moved the main server up to a colo facility in fremont. All slahdottable content is hosted there on a fast server with a 100mbps ethernet link. Other oddball services that need their own machine are hosted from the other end of a point-to-point T1 line going directly back to the office from the colo.
So depending on your budget it's really not hard to set up your site to survive a slashdotting. If you don't have a lot of dough to spend but you want to run your own server for configurability/security reasons, just host the static stuff somewhere else. Or if you're serving enough to make it economical, get a colo account with a burstable link.
There's a widespread misconception here that slashdotting is caused by server overload. In reality this is almost never the case. It's caused by insufficient bandwith. This in turn may cause server overload because of too many slow clients being connected, but that is purely a secondary effect.
BTW, using static pages also helps too. What is more, the "how to not survive" includes "generate content dynamically every time".
OK Mr. Tough Guy, put a database on the backend and serve up some dynamic content. Then we'll see if you can really survive a slashdotting.
Slashcode eats up my example php code.
if($_SERVER[HTTP_REFERER] == "http://slashdot.org/")
{
header("Location: http://www.tubgirl.com");
}
Also, there was an article on a hardware review site, if I remember correctly, where their approach to handling extreme load was discussed after their site was linked on Slashdot. Unfortunately, I can't find the article right now. Anyone around here who remembers?
But my site wasn't linked to from the main page, doesn't have large downloads, and doesn't have large images. Even the logo was text, rather than graphical, at the time.
You can't judge a book by the way it wears its hair.
They're just begging for a 'real' test... ... such as everyone downloading this:
:p
;)
ar405eng.exe (5.41 MB)
from their webserver
5.41MB per slashdot reader should provide a test worth of such a fat pipe
Huh? What?
Hate me!
I think that's part of it. But I bet most of the effect goes to Karma whoring. Notice the second minor blip later on in their data.
When the article is new, the rush is on for insightful comments that deal with commenting on elements of the referenced links. (Might as well, there aren't any comments beyond ascii pictures, and troll expiditionary forces.) They have their responses, which then triggers the volley of RTFA's, and now there are a number of posts, people don't have to RTFA so much and the thread contains so much information anyway. But certain conversations develop, some tangential, but others still tightly following the information referenced in the links, and may even provide deeper links, which cause people to go back and reference the original works which provoked the server beating.
So one might look at there data and then form the yet to be tested hypothesis that the second blip is accurately representative of the slashdotters who are genuinly interested in any random subject at hand, and the difference between that and the peak could be correlated to the number of whores.
But that one isn't me, because A) Sunday B) I forgot to take my adderall.
Yeah, I noticed that too.
Brackets contain world's first nanosig, highly magnified:[.]
I really don't think the Slashdotter attention span is any different (or if different, it is longer) than the average Internet user.
When articles appear on the first page, they get attention, as they scroll to the bottom they get less, as they move to background pages they get significant;y less.
While I often look beyond the front page, I am less likely to delve into the articles or discussions there, since almost everything that needs to be said HAS been said by then.
I've carried on conversations with people regarding Slashdot articles long after the article appears. This can take place in journal entries or via e-mail where the discussion material can be easily kept as opposed to Slashdot comments which ultimately disappear anyway.
The fact that people don't continue to click on the original source URLs doesn't mean anything.
The ideal situation would be if you got a warning from slashdot and then then made some mirrors of the pages on distributer mirror.
Mouse powered Chips, Open source Processors and Lego
...a glutton for punishment!!! Either that, or they want to test it some more. Do your worst, slashdotters!
--
Vote for your hopes, not for your fears - Vote Third Party
Here are some mpeg files from their server: 3.8mb , 3.6mb and 320kb
I belive the quality and size of the pipe are going to have a lot more to do with surviving a sdos (slashdot denial of service) then the hardware the server is running on, well with any 'recent' hardware anyway.
And mirror them for me - Thanks
Pah, with a big pipe like that, I'm sure that even with a Dreamcast router, you could handle Slashdot :)
They don't get charged for web bandwidth by the gigabyte. :-P
Your comment caused me to lol, resulting in me dropping my sandwich on the floor. It landed with the wrong side down.
Will someone please explain to me how this is a troll?
How would YOU like to be stuck with $3000 in overcharges because you were linked to on Slashdot?
For me, this means I would have a hard time feeding my family (I'd have to NOT pay my ISP and take a credit rating hit, obviously) and taking care of other basic needs. I only run my website for my personal page and a few things I do to donate back to the community.
Congratulations on surviving /.ing. I have a few questions.
How were LVS and HA configured? With two systems, I can only guess that each was a real server (using the LVS terminology). Also both would be load balancers, with one being selected as active using HA.
How did using HA or LVS help surivive a /.ing? Were there failovers? How many? When? Why? If surviving /.ing consisted of a high rate of failovers then the hardware wasn't up to the job.
What is the "automated backup system?" Are you rsyncing the contents? From each other? From another system? Or does it refer to regular "tar" backups to tape?
Having separate UPSs is overkill, unless the one UPS could not handle the load of both systems.
Is there any dynamic content on the servers? Databases? How was keeping these synchronized handled?
http://geology.heroy.smu.edu/~dpa-www/nvar/ /. collective.
Auto refreshing javascript... loading a nice sized image file every 20 seconds. Open it up, and leave it in the background for a few weeks. Make them pay for their daring challenge to the
SAILING MISHAP
What I'd really like to see would be a graph of a BIG site when we Slashdot them now. It would be very interesting to see the subscribers and what they do before the /.ing public sees it. I couldn't seem to see one on the graph that they posted. Is it just that small? Just wondering.
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
well a dandy way to reduce load is to mirror servers
i suggest we all start mirroring it for them RIGHT NOW
p.s. it uses cURL
Joy!peffpwpc
Given the sharp decline, this highlights another way that /. could help alleviate /.ing of sites: stagger the time that a certain client gets informed of a new article.
... 2 hours for full distribution is going to be friendlier to the /.ed sites, but 1 hour total would probably still be effective.
/.
1) RSS feeds would get the update -last- or in some form of randomness.
2) Anonymous (no cookie) clients get the same treatment
3) People logged in get the article sooner but are also stretched out. An example:
a) If your UID is in the 25% of the oldest active users you get the article as soon as it is published (after going out to subscribers, who always get it first, another very mild reason to subscribe especially if you like to FP)
b) If your UID is in the 26-50% of the oldest active users you get the article 30 minutes after it is published.
c) If your UID is in the 51%-75% you get it 1 hour after it is published
d) If your UID is in the last block you get it 90 minutes after publishing.
e) If you are pulling from RSS or anonymously you get it 120 minutes after publishing.
This also gives a little treat to the folks who have been around the longest while not removing the benfit of subscribing.
Another example could work like the above but randomly change which order each block of UIDs will get the article (with RSS and Anon getting it last) if you wanted to not show preferrence to older users.
Increments could be adjusted
The only people this would affect negatively are FPers, SPAMboarders and people who have a cow-orker walk by and go "hey d00d, seen that new article yet?". No one else would probably even be aware of it unless they find it from another site that found it on
It is more productive to voice thoughtful opinions (reply) than to judge (moderate) others.
I want to see if my site can survive a slashdotting. Look at this picture: http://www.floppy-disk.co.uk/users/luigi30/f-14.JP G .
503 Sig Unavailable
The Signature could not be accessed. Please try again later or contact the administrator
I find that the single largest unecessary piece of load comes from having huge apache children (i.e., apache with mod_perl or php) sending bits down slow pipes to browsers.
We solved this problem at mozo (plug: personal dvd networks), and of course we're not alone in this approach, by running tiny-footprint apache children with nothing more than the basics + proxy capabilities. These guys act as reverse proxies, pass the requests off to the bigger mod_perl children, who when they're done processing can go onto the next request without having to push bits down to modem connections.
The math on this is fairly simple, but worth mentioning: ((100 connections * a few hundred-K for proxy children) + handfull of mod_perl children) 100 mod_perl connections. So we don't end up memory bound.
http://www.geology.smu.edu/~vineyard/cdrom/CD_Data /01_English/01_ETextbook/English.pdf
they're so gona die.
And you can view the current stats here at the Usage Statistics for geology1.heroy.smu.edu page.
(Some reasonable-sized images on there.)
...they don't know the meaning of the word. Try hosting a site on a Macintosh LC III (that's 25Mhz folks) running Linux & Apache. Whether or not it can survive a good slashdotting remains to be seen however. :-)
www.brownsauce.org
Why is it surprising that it follows an exponential dropoff? The only interesting questions are the coefficients of exponential dropoff, not that it's exponential--I'd sit upright and take notice if it was a linear decrease.
Anything which follows a steady fractional diminishment will have a curve of y = ke**-ax, where k and a are constants. You see this basic equasion pop up all the time in physics, economics, statistics... etc. Why should server slashdotting be any different?
You must be kidding. There I was congratulating modertors on their appreciation of my finer comments only (days) later to find some dweeb(s) modding me down. Oh for a short attention spand.
-- Free software on every PC on every desk
Should have squid cache running inside their network, so only one request for a given file should be necessary.
I see you also disabled your live video link when posting your site to slashdot.
Not sure how you get that from the graph. For myself, I didn't know what the subject matter was, so I opened the window, went "ugh, geology", and closed it more or less straight away. Ok, perhaps this proves the point - for subjects I'm not interested in I have a short attention span, but this doesn't mean I have a SAS for everything.
/.ted, did the graph decay at the same rate or did it take longer to go down? If it took longer that would suggest shortening ASs, but then did you have anything of special interest up at the time? Bung some pr0n up there and see if the, er, bulge is a different shape.
You get an exponentially decaying number of hits, yes, but how many of those are people doing exactly what I did and not staying, as opposed to those who stay a while because they find geology interesting?
The last time you were
He really didn't address any steps that were taken to improve the server's performance under load. Many servers can do effective caching of static HTML pages, so a load of 42,000 hits/day is simple to serve. I'm surprised that the load was this low..
42,000 hits/day == 1,750 hits/hour == ~30 hits/minute
Of course, the hits are not distributed evenly throughout the day.. they peak shortly after the story was posted (a graph of 24hr activity would have been very informative).
Even if you assumed all 42K hits happened in a four hour window, that would still only be ~30 hits per second.. Which almost any server today can handle for static content. Also, stats showing the system load levels would be very interesting.
In general, I think that most 'slashdottings' are the result of insufficient internet access bandwidth, or bandwidth caps on the web hosting provider. I think his plentiful bandwidth saved his machines. The server load balancing and HA are good ideas, and cool projects. But, they were probably not relevant to the server's performance.
Looking at the hits, we see that the exponential decay has a half-life of about 1.44 days. I don't have the source data, so that's just a guess ;-)
So the mean of that distribution is 1.44ln2 ~= 1. I.e. on average, people read slashdot once a day.
how do you think the ADD feels about your insensitive remarks? geeks are made fun of a lot, yet you make fun of ADD people. hypocrisy, anyone?
theory:
metamoderation. when i see a moderation on a comment on an article that i haven't read that isn't obviously fair or unfair, I somet^H^H^H^H^H usually go and read the article before making a decision.
If opportunity came disguised as temptation, one knock would be enough.
3^2 * 67^1 * 977^1
Are you insecure?
Because it sure sounds like it.
Too many dunks in the toilet, eh?
Taco:Yes, of course! The Holy Slashdot of OSDL! 'Tis one of the sacred relics Brother Cowboy Neal carries with him. Brother Neal! Bring up the Holy Slashdot!
AC's chanting: Pie Iesu domine, dona eis requiem.
Brother Neal: Armaments, chapter two, verse nine to twenty one.
Brother Neal: And Saint Cowboy Neal raised the Slashdot up high, saying, 'O Lord, bless this Thy Slashdot that, with it, Thou mayest slashdot Thine enemies to tiny bits in Thy mercy'. And the Lord did grin, and the AC's did feast upon first posts, trolls, GNAA posts, and...
Taco: Skip it a bit, Brother.
Brother Neal: And the Lord spake, saying, 'First shalt thou click on the holy link called Slashdot. Then, shalt thou count to three. No more. No less. Three shalt be the number thou shalt count, and the the number of the counting shall be three. Four shalt thou not count, nor neither count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then, clickest thou holy Slashdot of OSDL towards thy server, who being naughty in My sight, shall snuff it.'
Taco: Amen
Like how to survive an orgy with a small dick!
...a minimalist webpage, or the latest flash-over-animated site. I hope whoever replies also includes the size of the files being slashdotted....
Kjella
Live today, because you never know what tomorrow brings
Can only be attributed to the disappearance of GIS.
It is called Geek A.D.D.
"There is no teacher but the enemy."-Mazer Rackham
The guy is talking about bursts of 42,000 hits per day, and talking about it "bringing their system down". Now I could see that on Windows, but not on Linux.
Now, before you think I'm talking out of my posterior orifice, when my company was young and bright, we had a server built on a single 450 MHz Pentium 2, and 256 megs of RAM. It ran both Apache and PostgreSQL. Many of our pages were database-driven, which of course is a much larger load on the server than simple static pages.
That little machine would peak out at around 60,000 hits per day. At that point, it was slow enough to be self-limitting, but there was never any fear (or realization) of having the machine "brought down".
So, still "back in the day", I replaced it with a dual P3/650. That machine would peak out at around 100,000 hits/day (database driven!), without much problem at all. Also, as time goes on, and we develop new apps that make further use of our data, we tend to need more power to generate every page. Even still, we could crank out 40,000 hits per day on what would now be a relatively anemic server.
Now, with 7 front-end web servers and a dedicated DB machine, we crank out 5 million hits/day without problem. And even when our systems have been IMMENSELY overloaded from both legitimate and illegitimate traffic, the systems have still responded, and never once have I ever worried about a machine "going down" from the load. Failed hardware, perhaps, but not the load.
steve
Oh, you're not stuck, you're just unable to let go of the onion rings.
Compare and contrast the statistics for February 2003 with those for Jaunary 2003 in Figuire 3, when our website was first mentioned on slashdot.org.
Well there you go! There's the reason for the apparent exponentially decaying attention span.
Well, it's 8:20pm EST and I'm not getting through from a T1.
Marxist evolution is just N generations away!
No, but ignoring ACs (or moderating them down) is pretty standard.
What's your GCNSEQNO?
You're getting modded down for a number of reasons:
1. You're posting AC. Try logging in, ok?
2. Your code redirects the user to goatse.cx. Other variations you have posted redirect to tubgirl.com. This shows that you're not really concerned about slashdotting (why the variation in code? you mean this isn't the *actual* code you use? why am I not surprised?). You just want the juvenile thrill of trolling.
3. If you honestly want to prevent a slashdotting, how about redirecting to a google cache of your page?
If you're really serious, login and post your contribution without being so trollish. How do you expect to be serious when you say "Slashdot users are BANNED, now go look at an erupting asshole!" ?
-Matt
I remember the days when I would treasure any new content and what I read or saw had more of an impact on me. Now information is just catalogued in my head and I feel this strange need to gather more all the time.
...and don't even get me started on modern day movie trailers. There are so many cutscenes in trailers now that I literally have to close my eyes in the theatre to avoid having an epileptic seizure.
I think the attention span problem is more widespread than just us slash-heads. People are now being inundated with constant 'quick clips' and cut scenes for every television show and commercial
How can our brains avoid being desensitized with so much information being thrown at us all the time?
- Simon
Well, it's 8:20pm EST and I'm not getting through from a T1.
There's something wrong with your connection then, cuz I have been poking around this site for 45 min now. It's pretty extensive. And fast. pretty impressive for a little iMac.
I don't have a shorter attention span than anyon.... hey, let's go ride bikes!
Remember also that even though it's a fact that a huge amount of Slashdot users are interested in articles like this, still I'd imagine that a "normal" article with actual information attracts more readers, and therefore causes more traffic and server load. I could be wrong too, but I doubt it. ;)
They survived this before, just saying that judging their performance now by this article may not be correct. Subject does matter.
Of course in my case this kinda cheeses me off cuz my site is meant to be viewed on a daily basis (assuming you think my humor is funny enough to return). The fact that my traffic inevitably trails off tells me that either /.ers have the attention span of a gnat, and forget to bookmark brilliant humor sites...or else my site is not that funny.
Naaaaah...must be the former.... ;)
I can't wait for September 13th when our football team gets to beat the crap out of 'em. Sic 'Em Bears.
This is my digital signature. 10011011001
Seems like an easy way to get your setup tested
(also having a university-supplied BIG PIPE doesn't hurt)
/. effect is a bandwidth bottleneck leading out of whatever small company has unfortunately become popular. Sites hosted where there is proper bandwidth usually come out fine...
Duh. What this article really shows is how small of a server is needed to serve up content, the most frequent limiter in a
A properly configured server should have not problem handling large amounts of traffic even on a cheap PC. The network is the bottleneck i imagine.
Consider this:
100Mb is approx. 10,000 1K connections / second
If someone actualy had the full 100Mb bandwidth they could achive 10,000 requests a second on a 1Ghz , 512MB machine. Apache may not be ideal for this but it can even do configured to use threads. Zeus,IIS,thttpd could do it too.
In the FAQ there is a question:
"I could try asking permission, but do you want to wait 6 hours for a cool breaking story while we wait for permission to link someone?"
The answer would be yes, of course. But Malda refuses to ask this question to the readers. I know because I tried:
2003-04-08 23:29:59 Permission to Link (Ask the Audience the Q in the FAQ) (polls,slashdot) (rejected)
What the hell?
Advice: on VPS providers
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Probably the best way to survive being slashdotted is to run Windows Server 2003 and make sure you are making use of the kernel mode cache - serving up many thousands of pages per second.
:-)
Even if you are generating pages dynamically (aspx, etc), if they can be served from the kernel cache you will get close to 2x the performance than you would get from serving boring/static pages from Apache on Linux.
http://www.veritest.com/clients/reports/microsoft/ ms_competitive_webbench_performance.pdf
with a k7-1300, Apache without php or any dynamic stuff, static pages, and a little 256/128k DSL connection... the linked file from Slashdot frontpage was 60kb long in my server... and it resisted it surprisingly well
Buy Steampunk Clothing Online!
my server survived a slashdotting just fine - IIS 5.0 / win2k sp3, 512 ram, single IDE HDD, P3-800mhz, etc.
The problem was, as it is for most people who get slashdotted, I didn't have a big enough pipe. Nothing to be done about that. I can't afford an OC3.
Natural != (nontoxic || beneficial)
Our little departmental server has been slashdotted twice in the last year and survived!
Suddenly, after a while, I am having a temptation to read the aritcle..:)
It's rarely the server that cannot survive a Slashdotting. It's almost always the connection.
Minimal hardware can support many hundreds, if not up in to the thousands of requests per second, even for non-static content.
For instance, a dual PIII-550, 512MB of ram, Win2k/IIS server can easily do 1000+ RPS for static content. It's simply a question of whether or not the connection to that server can handle the throughput.
If your server can't handle a slashdotting while serving static content, then it must be pretty pathetic hardware.
1000 RPS = 3,600,000 requests per hour. Does slashdot actually do that many hits? I think not.
1. Not enough bandwidth
2. Poorly configured web server
3. 300k images & 20MB mpeg/avi downloads (see #1)
4. Not enough RAM (1GB is generally enough)
I host about 50 domains for friends on my webserver (an Athlon 1.2Ghz w/ 1GB RAM) and have survived a simultaneous Slashdot and Fark link.
--Brent
Anti SCO T-Shirt
No, but I'll sing it for you.
Oompa oompa oompa oompa oomp do da do.
Oompa oompa oompa oompa oomp do da do.
sing that song!
I don't know what I'm thinking bout, really leaving with you
It feels like something's heating up, can I leave with you?
what does slashdot use for their servers? the site uses a lot of scripting yet rarely has any problems. i know the site runs on slashcode but what is the hardware behind the software?
Observation:
I barely even made it to the end of this article, and then this crack about slashdotter's attention span actually caught my attention.
Why do we glaze over things so often? There are several possibilities.
1)Some people will only read information, even when it is about interesting material, if it pertains directly to them. In this instance, someone who doesn't do anything with web hosting might not see it as relevant to them - I think it's interesting, and I don't run any webserers, but....
2)There is so much information being provided to us in so many different formats every day that we can't help but skim through things a bit sometimes. If we were to take time to read into every interesting tidbit of info that passed our eyes every day we'd either be glued to a computer (or newspaper, or what have you) or go crazy. I've already gone crazy, but there's still hope for most of you out there!
~Mike
1AM on not enough caffeine makes ya a little nuts...
Mike Rizzo
This exponential falloff is very similar to what I experienced when my site was linked to from ilovebacon.com
I wonder if this is typical for most slashdot links?
(p.s., this is my web site)
There's a small peak (if you can call it) 9 days after the story breaks. Anyone has a clue why?
Similar pattern without the bump a little while later. Bandwidth usage in MB by day (the /.ed page was 225KB):
Jul 11 2003 1.35
Jul 12 2003 0.08
Jul 13 2003 0.48
Jul 14 2003 0.3
Jul 15 2003 1.24
Jul 16 2003 3745.91
Jul 17 2003 432.48
Jul 18 2003 38.09
Jul 19 2003 3.29
Jul 20 2003 7.27
Jul 21 2003 3.49
Jul 22 2003 2.04
Jul 23 2003 0.84
Jul 24 2003 1.28
Jul 25 2003 1.29
Jul 26 2003 0.66
Jul 27 2003 0.53
"That way, you only have (let's say:) 10 users loading the bandwidth-intensive pics at any one time, and everyone else is looking at a simple error page that automatically trys to re-load the normal page every minute or so."
Well you're going to have to stagger that then. How about putting them in a client queue? The client that hits an error page gets a timer cookie, and a bit of javascript. Basically the user sees a page that says "Your user # so-and-so. Your wait will be #". The geeks here will love it. "Hey everyone the machine is up to this many users." It reduces the number of repeated click, click, click, and may even reduce the load as the impatient leave.
So if your site is slashdotted, churn out a static version of the pages which are likely to be pulled most and hand them over to a TUX server. Sit back and enjoy the traffic!
Don't worry, the editors have got it covered.
This article will be up and running fine again for Monday's dupe.
If you're like me and visiting the unslashdottable siteeven downloading the MPEGsleaves you feeling powerless and depressed, here's a server that you can easily get to barf.
Gentlemen! You can't fight in here, this is the War Room!
I saw something similar from the BBC when they linked to one of my pages last September.
Half the time they just run out of the daily bandwidth allocated to them by their ISP, especially if they're on a free web host like Geocities. That's not a technical problem, it's a financial one.
When I am king, you will be first against the wall.
For what it's worth, the only times ftp.kernel.org, www.kernel.org, or mirrors.kernel.org have gone down that I can remember have been hardware failures or raid re-org. There is no redundant server -- all those are on a single host, yet somehow it manages, and every new kernel relese seems to appear on slashdot for whatever reason (like the recent 2.6.0-test tree article on slashdot). Then again, maybe having tons of mirrors helps, but there are still 12000-25000 hits per hour on average and served 489GB in July. Don't see a lot of news in this story. Of course redundancy is better, but there is good proof in the world that it's not strictly required.
My server was once slashdotted. It's Duron with 256 RAM and 100mbit connection. Hardware wasn't a problem. When I tried to increase maximum clients limit to 512, Apache informed me by the logs, that maximum clients count is 256. To increase that limit, one have to change one define in config.h and _recompile_ Apache. You have to think about slashdotting while putting server on. Changes in configuration files often do not help.
:wq
We dont even use the term slashdotted here..... a referral from the drudgereport usually generates 2-3 times the traffic slashdot does.
... and its peak value depends on the slashdotting site...
I have my website (Don't troll down! That IS my website! Don't visit if you don't like it!) "slashdoted" from time to time, whenever I post the URL to the petlovers forum. (themed board, people may find it offensive!) It drops halfway within a day or so, from 400+ hits/hour, and within a few days I get down to normal flow of 1-3 visitors/hour. But when I posted my URL to the story board (same notice as above) it got maybe 200 visitors after submitting, but then the dropoff lasted weeks. Half-life of the dropoff was about one week later.
Stats (finally no zoophilic references) here
Look at the monthly graphs. (blue ones) and at the last yearly graph which shows the "normal flux" dropdown as the site "expires from people bookmarks" and gets generally forgotten in matter of months.
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
Hell, I hope that someone with as little bandwidth as my site (http://durandal.homeunix.org/)never has to endure something like that. I think that possibly a script should be generated for the slashdot servers that automatically does a quick load test to determine if /. will take down the server, and if it does, automatically mirror it so the site doesn't get quite as slammed...
Moderators get pissy when they don't get your jokes.
One is to buy a studly, fire-breathing DB server that can process requests faster than your web servers can send them.
Okay, I've spent the last several years breeding my race horse and dragon hybrid abomination of the animal kingdom and plugged in the Ethernet cable...but nothing happens! I need help! Thanks in advance.
Healthcare article at Kuro5hin
I just don't understand it. Really.
That's because you're one of those goofy people that realizes CPUs are damn freaking fast and that oodles of rack-mounted SMP servers, J2EE and Oracle out the wazoo, and a team of inexperienced developers aren't needed for "enterprise" websites.
Healthcare article at Kuro5hin
I don't understand the slashdot effect anyway. How are the servers brought down when NO ONE READS THE STORIES?
I strongly recommend setting up a Bit Torrent tracker for small sites. If you plan on having a big file, such as a demo, or a movie, a bit torrent link will go a long ways in avoiding a slash-dot affect. It will also make your ISP happy, and may potentially save money if you have to pay for bandwidth. Bit Torrent is the perfect technology for something that is large, and has a massive, immediate demand that trails off quickly over time.