Slashdot Mirror
Are Internet News Sites Ready for Major World News?
An anonymous reader asks: "Heading says it all really - are Internet news websites ready for the next big world event? news.bbc.co.uk already switches format under heavy load (not sure if this is automatic or not) and i'm sure some other sites do the same. But should a major world event take place in the coming months/years, the Internet is going to be the primary news source for many millions of people, particularly those without access to a quality television news service. How will / can it cope?"
Hopefully news.bbc.co.uk can cope with having it's link on Slashdot's homepage...
Certainly when the events of September 11th took place, for those of us at work in the UK without a television at hand the only way to keep up with events was via the web.
News sites failed to cope with the load - millions of people trying to access the same sites meant that no amount of bandwidth could cope with demand.
For this reason, I don't think that the web is going to replace television as a source of live news coverage anytime soon.
At least, not as of 9-11-2001. When I heard a plane crashed into the WTC, it was confirmed by the fact that I couldn't reach any of the major news sites. Which didn't help an awful lot.
As long as major sites get slashdotted, they won't be ready for the traffic associated with a major event.
How many people have internet access, but no access to TV, radio, or other broadcast recievers? For major news stories boardcast medium will always be the main method of disseminating information to the masses, client-server systems aren't really designed for this purpose.
On sep. 11 last year I watched the first hour of it live from BBC's homepage without to many problems.
Seemed fine to me (not that I was thinking much about the quality). Was it really that bad?
TC - My Photos..
I mean ok performance degraded on 11/9 but that was the only event I can think of that made the news sites shudder.
;)
Do they have any obligation to serve under high load?
Do we even care? Maybe the radio is a better source of news sometimes, hell try CNN
Why? Are you planning one?
Al.The Daily ACK - Eclectic posts by yet another hacker
Sites can do their best to anticpate heavy load, but off-the-map events like 9/11 tend to reveal weaknesses in systems (which potentially can be elsewhere in the network). Also, it's pretty expensive to engineer to contantly be ready for such rare occurances.
Here's what I do: Bitty Browser & Andromeda
It somehow seems unlikely to me that someone will have access to the internet but not televsion so I believe the statement above is somewhat of a canard. Now that being said, it is possible that people may not have access to unbiased television news (eg: China, Middle East, etc ) though it could be argued that most of us dont.
I do however believe that it can be used effectively as a means for distibuting important news information and really is strength will be that it can provide a broader spectrum of news commentary and analysis.
When i was at work for september 11, all the news sites went down so we had to go watch good ol' television. In fact, the entire internet pretty much slowed down to modem speeds. However, I had no problem accessing cnn thru http://asia.cnn.com to get the latest info, so maybe they should have redirected people automatically...
This is how.
1. Resistance to large amounts of sudden traffic.
2. Meta-news from other sites.
Simple really.
-----
fat chicks need love too
err... given that google's news service uses slashdot as a direct source, I'm not sure that it matters. ;)
No they can't cope. It's been proven already. Even giants like the BBC and CNN had several moments where they could not handle the load on September 11th.
I'm sure that they have taken steps to improve things in the future but, there is only so much that you can do, or at least do cost effectively. There is no substitute for hardware and bandwidth but, maintaining enough to support the entire planet at one critical moment in time, that may or may not come, is not cost effective.
When the time comes, the news sites will buckle under the load, just as the telephone system does. The best source for news, during times of disaster are television and more so, radio. Even in the most remote places, you can still get radio and with new satellite radio, you can get it anywhere.
What type of backend is running most of the news sites? Are some of them distrubuted? (I know some are, but to what extent and how? )
If you mean a major bandwith spike, then where is the weakest link? Will the pipe fill up before the processing power is toped out?
I know that some ISP's had their bandwidth bursting at the seams during 911, so even if there was nothing wrong with the news/internet/network - the ISP was fragile.
Not really a post - in that I am not giving much in the way of answers, but just trying to ask the right questions. There is so much to consider in such a situation, rather than looking (drooling?) at their massive server farm(s), don't forget about the pipe that feeds it(them).
Create music
The BBC coped because of two main things. The first is because they switched to a low-graphics version. The other reason is that the BBC's servers are geographically spread out. They have servers on several European backbones, and also have seperate servers in New York Telehouse which serves all the content for the people on the other side of the Atlantic.
Thats how they coped, my old mucker.
Cheers,
Ian
Keynote have oublished a reporton the performance of major web sites on September 11th, 2001.
Of course, there's a lot of dark fibre around, so the capacity is there if it's really needed. Once the current recession is over, we can expect to go back to the days of massive overprovision and redundancy as content and bandwidth providers seek to build in capacity to handle peaks. What will really help is multicasting for video streams, and well-designed caches at ISPs.
They can do it the same way I cope when my power goes off... A cheep battery operated shortwave radio tuned to the BBC or other quality station. IMHO, I'm pretty sure if they can't get access to a TV then what chances do they have at getting the internet?
------
Never underestimate the power of stupid people in large groups.
That was my first thought. On that day Slashdot, Kuro5hin, and other places became "rip'n'read" sites and held up quite well under the load.
Best Slashdot Co
A lot of news sites got a taste of what covering big events on the internet is like. Some did okay, most didn't. Even Slashdot learned a few things about handling loads.
Also, it in part led to Google News. I'm actually kinda comfortable with Google handling news, as I think if such an event happens again, Google can just cache the important news.
TV and radio, though, will likely always have the advantage that viewer load doesn't affect them. So, even if someday we move beyond traditional TV/Radio broadcasting, emergency radio broadcasting should be kept in some form.
Drop the images, cut out server side operations... if you would have asked this question 2 years ago, I'd say no, we weren't ready. It takes hindsight to gain foresight; we're ready now, only because we've learned our lesson once.
"...particularly those without access to a quality television news service."
Gee, that's pretty much everyone.
On September 11th, major news sites like Yahoo, BBC, CNN were entirely flooded with traffic much like the phone system was, going as far as taking down some fairly large servers altogether. What ended up happening was that a bunch of IRC channels (specifically on SlashNET) cropped up with people giving live webcam shots, rumours and snippets of information, mirrors. Then the CNN closed captioning bots started relaying to IRC for those without the cable service. It was interesting as it showed the Internet both failing at and succeeding in its primary designed function, as a communications and information network that could survive a major catastrophe.
s200.org - visit it (me), love it (me).
With the prevalence of the internet as a means for distribution of all forms of data, new ways of meeting these needs are needed. No longer can one use traditional methods of increasing pipe size or basic colocation and assume that you're back will be covered. We're seeing increasing occurrences of sites being hammered (for whatever reason) and not just the small ones. While the internet may be a massively distributed thing, it still has some major Hopefully this is an area which the methodology of P2P systems and on-the-fly mirroring can help with. If something is in high demand, it should be made _easier_ to get hold off, not harder.
When major news events occur, CNN will shut off banner ads on its home page in order to speed up page loading times.
--Sam
Huh? I haven't seen any quality televsion news in years. During a crisis event it's even worse than normal. What brainwashed planet do you live on?
My observation for 9/11 waw that major news site crawled under the load. However, less often visited news site were responsive all day and gave the same news with the same level of coverage than the big news sites.
So I must say, find some smaller news site and bookmark them. When your big-shot news site will crawl under load, just go to the small one and you will get your news.
BTW if you just want nice video, the Intenet is not the place to go, turn on your TV, you'll get far better image quality and you don't have to wait until the video is buffered.
Being a Brit, the BBC was the first place I turned to for news and basically the whole thing ground to a halt and that was despite the BBC News outfit having upgraded systems substantially to cope with the 2001 UK General Election. Both the UK and US mirror were swamped and basically stopped working. Interestingly the US Mirror site was in New York, not far from the WTC, and despite the fact the power was lost in the entire area, the servers kept going for several days on backup generators until those generators died due to the dust.
It tended to be the second-tier news service like Ananova that could cope, simply because in times of crisis people will always turn to familiar names first.. the BBC, NBC, CBS, CNN etc.
I seem to remember that the low-graphics option came after 9/11, but it's only a partial solution to the problem.. several times since then the BBC have switched to low-graphics but there haven't been any events of the magnitude of 9/11 since then.
Look at it this way.. lets say the US has 50 million office workers with access to the Internet (a pure guesstimate) and they all try to access the same news sites within a window of 30 minutes. On 9/11 people were trying to download videos of the attacks so they could understand what was going on - don't forget that those now familiar images we all know now were completely unthinkable. This combination of huge numbers of users and very high demand for streaming video is almost impossible to keep up with.
In short, on 9/11 the web let us down and the only people who knew what was going on were those with access to televisions. The world has not moved on that much in the past 12 months, so basically the same thing will happen all over again if (God forbid) the same thing happens all over again..
Never email donotemail@WeAreSpammers.com
or at least this is what I think:
http://robots.cnn.com
however, I still think that the best medium for broadcast is not an interactive media like the Internet, but a one-way media like radio or TV;
Anyway, I would rather prefer a text-only information source like during the Gulf War the BBC did on IRC. But I may be wrong on that.
-- There are two kind of sysadmins: Paranoids and Losers. (adapted from D. Bach)
Even more scary... We won't know about it, because we won't have any news to tell us!
Nevrar
Needless to say, cnn.com really had to get more servers into production quickly. They worked with Sun to get several hundred servers on site and running.
I don't know why cnn.com had such an upgrade strategy, but it is what happened....
Get yourself a battery powered portable radio. Make sure that it is the type that can receive shortwave frequencies and you will never be without a BBC broadcast. The are lots of small cheap portable radios on the market that receive AM/FM/SW/TV and I'd also expect to see satellite portable radios soon but, I can't imagine paying the subcription for such a thing, especially when SW is availalble.
This is where proxies come in handy. If there are 1000 people in a large corporation trying to access the web at once on such a day, then a proxy would reduce the number of duplicate requests being made to the web site involved.
At the same time maybe the HTTP procotol needs a version that is capable of UDP broadcasts in special cases?
Jumpstart the tartan drive.
I live in the U.S., subscribe to cable, get all the cable news channels -- but I surely do NOT have quality tv news service -- it does not exist.
news.google.com should hold up under even the heaviest loads, and while you might not get the actual site it links to, you should at least be able to get the idea of what's going on based on the headlines.
In a time of crisis, is it really necessary to know the details of a major world event immediately? If a nuke goes off somewhere, I'm not too concerned about who did it--I'm driving to some remote place, THEN I'll start asking the questions.
On Sept. 11th, what did we know for certain:
*4 planes were hijacked
*Two towers fell
*The Pentagon was hit
*A plane went down in PA
everything else was mere specualtion at the time, and everything above could be read by headlines alone.
Just a thought,
sig--we don't need no goddamn sig
The answer to scalability has been there for years, and it's multicast. Multicast is a protocol that implement a one-to-many distribution of the information, allowing very efficient distribution of contents on the internet (the target is that the information should not pass more than once on any given physical line), and dynamic group joining and leaving.
However, ISP and users are confronted to a chicken-and-egg problem: ISP pretend there is no demand for multicast, so that can't justify the investment in increased NOC knowledge, users don't know what it is, and content providers have no support from ISP or user.
Multicast is however the scalable answer for live broadcast and scheduled replay, it's been there for years and I do not loose hope that it will be better used one day.
Surely you jest!
--jdp Maintainer of VisEmacs
Seems like the best answer would be automatic load balancing between disparate servers. But how would we get the services to cooperate? E.g. rushlimbaugh.com not be too keen on sharing resources with cnn.com. :) And that begs the question, would the "rescuing" site be entitled for a fee for their failover support?
3. Profit!
2. ???
1. On Soviet Slashdot, a Beowulf cluster of alien Natalie Portman overlords welcomes YOU!
The internet is not, and probably never will be the primary news source. Radio and TV are *live* mediums, available virtually anywhere at any time. Major events appear in real-time with audio and video. And unlike the internet, TV and Radio short of EMP are hardened to failures in power and other events. Take out a few key routers and where does that leave your net access?
It was Larry Niven who predicted the idea of "flash crowds". Of course, he was envisioning physical crowds via teleportation, but the basic idea still holds. It's only going to get worse as more and more people use the net.
Look at it this way: in a primative society, a clan or village would usually have a storyteller or sage who gathered the news of the world in story form and re-told as appropriate. We should not be supprised that it takes millions, perhaps even hundreds of millions of people to be the story-tellers to 6 billion (that's a US billion).
If the Internet had a higher percentage of useful sites for news (not just talking jpeg-heads, but innovative ways of conveying the STORIES that the news represents), then no one of them would be loaded down and the backbones would be the only bottleneck. Notice that so many of us flocked to Slashdot when the towers fell? Wonder why? Because Slashdot, for good or ill, is our community's storyteller, and we instinctively come here to understand how our community is reacting.
Folks,
If you don't have a portable AM/FM radio, or even better a shortwave receiver, then get one TODAY. Get some spare battteries for it as well.
The simple fact is if you want to hear what's going on during a "major event" radio is the best way to do it. And you have evacuate in a hurry, you sure as hell aren't going to be taking your 60" flat screen TV with you. You want pictures, wait for the evening news, if you want to know what's going on NOW, get a radio.
Even better, get yourself licensed as a ham radio operator so you can be part of the communication solution if needed (yes, amateur radio is still important, even today).
You know you're a geek if you've ever replied to a tagline.
particularly those without access to a quality television news service.
Isn't that a oxymoron?
Higher Logics: where programming meets science.
I mean ok performance degraded on 11/9 but that was the only event I can think of that made the news sites shudder.
..Err, I believe that was the author's intent when he said the next "major world news event". Obviously, that doesn't have to be "the next time that planes crash into buildings in the US, but when in recent times has newsworthy incident happened? Or at least one that draws that level of coverage? The question was not "Can major news sites cover the England vs. Brazil soccer game this weekend?", it was "If a major world event were to happen in the near future, will the current news sites be able to handle it?" The only point of asking at all is the fact that as you even noted, the last time no, they unarguably did not.
;)
Do they have any obligation to serve under high load?
No. If you walk into my store, I have no obligation to sell to you. This becomes a matter of self-appointed corporate responsibility. When it really comes down to the wire, are you about providing the public with vital, up to date information, or are you about providing content to generate revenue? If many of the advertisers' links were slowing up (as was already posted somewhere above), you're not generating all that many more hits, and if they have to click the ad, forget it.
Do we even care? Maybe the radio is a better source of news sometimes, hell try CNN
Do you get cable at work? I don't. I don't have a radio either. This happened when most people were at work, getting ready for work, or on their way to work, most of them probably have internet access, but relatively few have access to cable. Radio is a possibility, but on average probably less ubiquitous in the work place than internet access.
--- What
During the horror of the attacks last year, I was surprised and thankful for CNN's approach which allowed them to withstand the barrage of hits:
They switched to an old-school, how-the-web-used-to-be, no-nonsense design. It was basic HTML, with some embedded pictures that contribued to the information. No frills, no ads, no sidebars about the latest crap-news, just the information we were looking for. Needless to say, it also ate a lot less bandwidth.
Of course, they were down part of the morning, but when they came back in the altered format, I thought it was a great move. A few other sites were doing the same thing, and I think they'll remember the technique for the next time something big goes down (hopefully something pleasant next time? I can hope...)
To state the obvious, the major news sites would have to have not only leaner pages, but also have the infrastructure to withstand a slashdotting-with-hair-on-it. Leaner, lower bandwidth web pages benefit every one, every day, but for daily needs the infrastructure is going to be expensive overkill.
In contrast, more of the tech sites were already used to heavy loads and I would guess that his brought in a larger than normal number of new and infrequent visitors. Maybe it was my imagination, but it seemed that after that many mainstream newspapers, magazines, and radio magazines started to carry more cutting edge tech info and topics and providing in a much more timely manner - days instead of weeks or months.
It would be interesting to map how much the coverage and timeliness of tech issues by the mainstream press changed, when it changed, and how much was related to being able stay on line.
Beta is broken and the link to classic doesn't work. Stop wasting our time or there won't be anybody left here.
I have web access here at work without ready access to cable news services on TV. I watched/read about the events of September 11th on the web.
Hello! Doesn't anyone get it--it's funny!!
I propose a the /.-meter as the official measurement parameter. /.ing? Well, probably it is ready for Global Thermonuclear War.
Does a site resist under heavy
Signatures are for stupids.
The worst terrorist attack in recorded history occurred over a year ago, followed by a Holy War against Islam, and now Israel and the Palestinians as well as India and Pakistan are teetering on the brink of their own war, Argentina is in the midst of a financial crisis, America is considering launching attacks against Somalia and Iraq, and you people have the gall to be discussing Are Internet News Sites' Ready for Major World News???? My *god*, people, GET SOME PRIORITIES!
The bodies of the thousands of innocent civilians who died (and will die) in these unprecedented events could give a good god damn about Major World News, Internet News Sites, your childish Lego models, your nerf toy guns and whining about the lack of a "fun" workplace, your Everquest/Diablo/D&D fixation, the latest Cowboy Bebop rerun, or any of the other ways you are "getting on with your life" (here's a hint: watching Cowboy Bebop in your jammies and eating a bowl of Shreddies is *not* "getting on with your life"). The souls of the victims are watching in horror as you people squander your finite, precious time on this earth playing video games!
You people disgust me!
It isn't just the news sites we have to think about. We should also be asking, when the next big event does happen, will people even be able to get online to access the news sites?
I'm not talking about some sort of damage to the communications network. I'm talking about ISPs that enforce strict rules on how many of their customers can get online simultaneously. They are the real threat to the Net as a primary source of urgent information, and it's all about money. They take on millions of customers but total capacity is measured in tens of thousands.
For example, on September 11th there were a few hours when tall buildings in London and other British cities were being evacuated, but many people over here couldn't get online to access vital information because our ISPs have notoriously low capacity and only allow a small percentage of their customers online at any one time.
Obviously this is a greater threat in rural areas because the only available connection method is dial-up.
/. is enough
on 9/11 I got news. It works every time. It had enough speculation. I got everything what I need.
Others? Others can't be surrive. Becauese they are not designed for nerds.
[My english is better than most other people's Turkish, so please point out mistakes politely. Thank you.]
I am subscribed to a couple of worldwide mailing lists and I have found that email simply rocks in high 'net traffic situations.
During the New York tragedy, much of the traffic on those lists was along the lines of "I can't get to the major sites because the web is clagged solid - can anyone tell me the latest?". And thankfully for a couple of days, the rules about straying from the topic of the mailing list were ignored.
Granted, many of the complaints were actually related to individual corporate firewalls, http gateways and proxy servers, rather than the sites themselves, but the situation stands: for whatever reason, you can't get to the site. Our web proxy fell over under the load, but our SMTP gateway just kept on going. And so did most others around the world. And I imagine that NNTP stuff worked just as well the SMTP stuff.
Remember folks, the Internet is a lot more than the Web!
Ahh - My eye!
The doctor said I'm not supposed to get Slashdot in it!
Multicast news services worked well during 9/11 and there is no reason to think that they won't the next time. Multicast is specifically designed not to "melt down" under extreme changes in audience.
The trouble is that not everyone is multicast enabled, but this shows real promise in handling news and emergency information over the Internet.
one of the passengers on one of the planes that came down on 9/11 (it was the one that crashed in the field, IIRC) was a founder of Akamai Networks, one of the load sharing/distribution companies that allow bandwidth to scale according to demand. As his plane came down, his company was entering one of the most demanding days in its history, as more people were targeting news sites at once than ever before.
It's organisations like that which will assist in the next big news item.
-- james
bbcnews.com uses cookies (set during your initial visit) to direct you to WORLD-centered edition of UK-centered edition. (You're given a choice on your first visit)
Slashdot seems to be the Gray Hat QA engineer in testing concurrent site capacity. Maybe it should get a salary and benefits....
moto411.com
CNN uses Akamai as a CDN (Content Delivery Network), to put all large static files (e.g. Images, Video) at the 'edge' of the network (e.g. distributed to over 10,000 servers worldwide). Even with this technology, CNN's site still failed miserably when it counted (9-10am on 9/11/01). Why would they pay tens of thousands of $$$ per month for this service when it still cannot handle the load?
Ever since 9/11, I've noticed that the heavily-trafficked sites cope with sudden floods of hits by switching over to static pages with minimal graphics. The NY Times, for instance, did this when the AA flight went down in Queens last November. CNN's done it a couple of times as well.
When we're looking at scale, though, it's useful for us to remember that these sites can handle way more traffic than even the typical slashdotting can deliver. Most breaking major news can be handled by them with only a little bit of slowdown. It's only the 9/11-scale events that can really bring the news sites to their knees - so lets hope that we don't have to see anything that brings on a overload scenario for the big news sites.
The other thing to consider is that most of the news providers are still investing some money in their infrastructure - just less than before. It's very well possible that a 9/11-scale event might not hammer the servers the way they were hammered last year. A lot of web sysadmins learned valuable lessons that day that I'm sure have been applied since then.
-- Josh Turiel
"2. Do not eat iPod Shuffle."
... the headline reads, "Internet knocked out by multi-city EMP attack"
Besides the question of whether major news sites are ready for a catastrophic world event, I wonder how the amount of junk mail/SPAM will impact network capacity and overall cost of internet services. I can't speak to how this internet chaff might impact news services, but I'm certain it has at least a small effect on overall "net speed."
I mention this because this post brings back a strong recollection of emails that swarmed into my inbox after 9/11 --Taliban Singles, the fake face in the WTC smoke, various flash cartoons depicting the bombing of Afghanistan and Bin Laden, soft-hearted text descriptions (probably fake) about survivors and their families, and so on.
It was Princess Di's death. I was on shift the night it happened and it pretty much brought all news websites to their knees. That was the first time I noticed the low bandwidth version of CNN. At first I thought the site was choking because it looked like some graphics were not loading.
Still, I'll give it to Slashdot and to IRC. I spent most of 9/11 on IRC transcribing what was being reported on CNN, since for a while the site was pretty much useless. A bunch of us where also taking screen captures and posting them online so people could see the horror. I still have captures of the first flyover of the Pentagon, which is less than 10 miles from my office.
Pedro
----
The Insomniac Coder
the Internet is going to be the primary news source for many millions of people, particularly those without access to a quality television news service.
I dont got no TV.
No radio.
But I have internet access.
When there are big world events, the amount of net traffic does increase overall, but not hugely, as instead of wasting time reading/working, we all go and look at news sites instead.
One way around this problem is bandwidth insurance. What is this? Large groups of averagely popular websites all get their bandwidth from certain sources. When there's a sudden move in traffic, those really big providers can simply deallocate the bandwidth from gardening.com and reallocate it to the BBC .
I might be talking out of my ass here, as the BBC already has peering agreements with Telehouse etc it's so big. Alternatively ISPs could implement decent caching systems. Otherwise, FreeNet released 0.5rc1 earlier :)
When 9/11 hit, the first thing I did was wget about two dozen news sites and thousands of blogs immediately. CNN, in particular, got blacked out really, really hard, and was reduced to one image on the front. I wish I had my archives available to post but they're rather deeply gzipped ...somewhere. =)
Akamai had their work cut out for them that day, I can tell you. I was lucky. I called out sick.
But none of this really answers the question -- how do you cover your butt and insure that you keep getting a news feed when/if you need it? I noticed that when I go to www.php.com, it's quite slow. So I started using uk.php.net and it zips right along. The moral of this story is that you might want to find 3-5 news sites that you consider good (and a factor in this probably should be how fast news gets to their site), then find some printer-friendly version/low bandwidth links to their front pages. Those are far less likely to be used when things get crazy. Drop some admins an email, perhaps certain versions of their site is located on entirely seperate servers and might go unscathed during a 9/11-ish rerun.
My
Limekiller
With as much money as these big news corporations have, they can afford to pay for load-testing and geographic based load-balancing. Just look at the statistics of the last site IBM did for the Olympics. It had insane hit-rates and never experienced 1 sec of downtime or missing requests.
The only thing that can save us now is midget wire-fu.
quality AND television? where do i sign up?
Check out their quite impressive network infrastructure at this page. Lots of info on their network and links. And, suprise suprise, it spans quite some more countries than only the UK.
like Akamai, Mirror-Image, Digital Island, Speedera. It's cheap and any of them can handle the load...
for distribution of widely needed information such as news?
I've read nearly 100 comments so far and while many mentioned the inherent flaws in the "massive webserver" model we currently use for virtually all web news traffic, no one mentioned the alternatives. For things like news distribution television and radio have an incredible advantage (they don't have to double any output to reach twice as many people as normal) but peer to peer systems would also work wonders in situations like this.
Maybe that is the rationale behind project IRIS, the recent US government backed research into a complete network founded on the peer to peer methodology.
www.newscientist.com/news/news.jsp?id=ns99992861
iris.lcs.mit.edu/
Hopefully, the trial and exile of
The Chump-In-Charge
Thanks.
TV has a big advantage over the internet, because it doesn't matter how many people tune in to a TV station, its not going to prevent more people from watching.
The more people try to connect to a website, the more it will slow it down for everyone else. Bandwidth is a problem for the internet, not for broadcast (or cable) TV.
Yahoo news took the load. Yes, there are some sites that you can't "slashdot". Yahoo can bring up a crazy number of extra servers when need be, they use bi-coastal servers, Akamai'd images, and their formatting has alwasys been light. Apart from the DOS attack they suffered a couple of years ago, I don't remember a time when I could not get Yahoo news to respond.
That was a joke right?
Maybe it's not ready yet. For about 1/2 hr I've not been able to load this site, all the other sites are ok
the Internet is going to be the primary news source for many millions of people, particularly those without access to a quality television news service. How will / can it cope?
Huh?? I'm not sure what you're talking about but I'm pretty sure I don't recognize it. Quality television service is much more widespread than the Internet. I'm N. American, but I've lived for years in Africa and Asia. I can assure you that in "None of the above" has the web surpassed broadcast media as a source of news for any but an elite few. And the comment is irrelevant for the elite since they have access to "all of the above"-plus.
Seriously, even in the smallest, poorest villages around the world several people will have radios and access to VOA, BBC, a national broadcast network and one or two regional stations. In addition most villages will have at least one television.
The internet is a bit player if it's a player at all
Oh, how funny it would be if it were true.
- Cath
Yes.
I contacted them all and they said they're ready.
On Sept 11th (and you all know what happened there, save the ribbons for a different soapbox), I used the internet as my primary source for what was happening. Somebody here had a radio, and the news channels were spouting lie after lie, rumours on air, digging up unchecked sources, because that's what the mainstream media does.
I, instead, got my news from "switchboard" type sites (/., drudge and a few forum sites), keeping an eye on who was up, mirroring important pages, and basically exchanging as much info as possible. It lagged a bit...I was 10 minutes out of the loop when the tower fell, for example...but I also wasn't supplied rumours like "there are nukes in the air" or "A fifth plane is on its way to chicago."
By the way, BBC had amazing realtime coverage plus rm video that stayed online pretty well. NYTimes was slow as hell. CNN got swamped, as did MSNBC.
Hey freaks: now you're ju
A spokesman for CNN.com said "after talking to several vendors including Sun, IBM and Microsoft, hotnakedteens.com won the business by showing they routinely handle traffic 10 times the traffic we received around Sept 11."
Read reviews of shopping cart software
CNN and others provide email alerts for breaking news (which notified me of 911), the web then provides initial reports, then we switch the TV on and get realtime news as the web grinds to a halt.
Though if the next major event happens on the same day as a game demo or a new Matrix trailer are released, we're truly stuffed...
How about Google news?
"The best argument against democracy is a five minute chat with the average voter."
--Winston Churchill
...that a website (/.) that itself is well known for being down, is speculating about what OTHER (far more heavily trafficked) sites should do ? I have to wonder how /. would do if IT were under the same sort of traffic that CNN's or BBC's website experienced on 9/11 ??
So we have questions about bandwidth, okay -- but we also have questions about how and whether television and newspaper editorial process might break down in trying to get "instant" stories up on a Web site. A process set up to approve stories for tomorrow's paper doesn't necessarily apply to stories that need to go up now. (My two local dailies have really felt their way with that, too.)
particularly those without access to a quality television news service.
Okay, I'll bite... What quality television news service? Gotta get me some of that action. You must not be viewing the local sludge we get here, with the jocular anchors' repartee and all...
I've seen one U.S. "news" program -- Dateline, maybe? -- ask a scant few questions about the preparedness of New York's emergency Fire and Police responses, mentioning specifically the failure to improve the same communications gear that had failed in the earlier WTC attacks. The show mentioning those problems in passing, almost rhetorically -- "Some people wonder..." was the tone. (Apparently the TV network didn't wonder itself. Only some vague "critics" -- that's the tone I mean.) The New York Times published an article about those same problems, around a full year later if I remember right -- and the article's theme was "Why isn't anyone asking these questions?"
If we had quality "news" on TV, the shows would be investigating controversial events, not just... what, commemorating momentous ones? Journalism is about intelligent enquiry. If you had to choose between "intelligent enquiry" and "advocacy" in describing the Fox "News" Network, which would you choose? That network is about reinforcing people's political leanings, not reporting the news. No thanks.
"Fundamentalism" isn't about divine morality. It's about human authority.
But should a major world event take place in the coming months/years, the Internet is going to be the primary news source for many millions of people, particularly those without access to a quality television news service.
Please be advised that your set needs adjusting... It's pretty clear from the evidence (and from a phenomenological point of view if you observe your own reactions) that the experience of watching a major event on television as it unfolds barely qualifies as useful information, due in part to the nature of the medium, but largely due to the nature of media filters and techniques. When you see something like 9/11 going on, it's much closer to entertainment, unfortunately, than providing one with reconnaisance leading to rational behaviour. The drama of the moment helps you develop powerful emotions in relation to the event, but what kind of info do you really get?
When it comes to war, TV obscures. For instance, see this study on media and the gulf war. [Remember that? Oh wait, it's still happening.] A salient quote:
In other words, you'd actually be better off combing through usenet than sucking on the immediacy of the glass teat.
Qualifier: I've worked in media-democracy-oriented film/video for years, I'm involved and devoted to the medium!
Damn those pesky terrorists
After September 11th last year, many news sites found they could not cope. CNN is now hosted by Akamai's Edgesuite system. The BBC still serves all of it's own content. The switching between light and heavy pages is not automatic and both are kept up to date so it is easy to switch between them. Also due to the closeness of Telehouse New York to the WTC, half of the BBC's servers were out of action when NY lost power and the backup generators could not cope with all the dust in the air. This actually meant that the traffic was sent all the way to the UK. In light of this, a lot of effort was put into increasing the serving architecture to be ready for another such event...Although it has yet to be tested.
I work at a large Air Force base... Intel has a TV on all day long tuned to CNN... Guess that's who does our foot work these days.
If multicast were ubiquitous then things could have been much better. If people could received the html only web page and turn to the mbone or some other multicast network for the streaming video then the net could probably shrugged almost any event off. Since porn is one of the few things that makes money on the net I am suprised that multicast for streaming smut hasn't become more prevelant.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
William LeFebvre (of CNN.com) has an excellent talk that he's given at least twice at Usenix events (I saw it at Usenix '02, he also gave it at LISA '01), that gives a lot of detail as to the things that had to be coped with to keep CNN.com running on September 11. I can't find the full-text of the speech anywhere online, but there's some information at this site that at least gives you an idea. Interesting stuff!
A local SAGE chapter had the senior sysadmin for CNN come in to give a presentation on managing a large webserver farm. I remember the admin said their weekly staff meetings frequently discussed the answer to the question "What if the president declares war this week?" and the servers' readiness for the load, projected from the traffic they received during Desert Storm. In general, I seem to recall their main strategy revolved around scalable or easily-expanded network connections to the data center, and a large pool of servers used as a testbed and development set that could be switched over to production use (I believe they were using a round-robin DNS strategy similar to Netscape's ftp server system in Netscape's early days.
I atttended this presentation, so while the description above is first-hand, my memory of the details may well have dimmed with time.
I guess I should do a search on this, but as far as I noticed, on and after 9/11 the internet was running and reporting reasonably, considering the traffic impact ?
There was a noticable slowdown and I'm sure there was a fair share of server overload resulting in crashes - but the internet held up due to it's very nature.
What is probably a more worrying fact - what happens when an ISP carrying the bulk of the Worlds internet traffic crashes / goes out of business ?
A slashdotting - you get the stick first and then the carrot !
So tell me again what possible motivation the news industry has for upgrading their online capabilities?
Are any of them even making a penny on their websites? So why pour more money into upgrades? What's the reward? So they can pay more for bandwidth and lose more money?
I was at work, away from a television on September 11th I heard vague news of a plane crash on the radio. I logged in for details:
msnbc.com - down
cnn.com - down
cbsnews.com - down
abcnews.com - down
drudgereport.com - down
I turned the radio back on. Yep. Still works.
Why? Because radio can charge enough for ad space to pay for a working transmitter and a studio and a full-time staff. Cable news makes enough money to support their operation as well.
But online news, for the most part, loses money, and thus can exist only as an offshoot from an offline operation like a TV news broadcast or a newspaper. Therefore it winds up acting only as 1) a supplement and a promotional tool for the broadcast or publication 2) a reader feedback time-waster.
It's always this way; follow the money and you get your answer. And right now the answer is none of the online operations have the desire or motivation to be "the" online news source when the next 9/11 breaks. Let the site go down. Who cares?
Phallic Symbols in LOTR
A far cry from the days of Walter Cronkite.
You mean back when we had exactly 22 minutes of world news for the entire day?
You mean before the days of 24 hour news channels? And 24 hour Headline News channels?
You mean before the days of live congressional coverage via C-Span?
And are you aware that virtually every TV network went commercial-free during 9/11 coverage?
TV news deserves its criticism, for sure... but be fair. And don't pretend there was this golden age of news when reporters and newscasters worked for free because of an altruistic love of the truth. They've always been under pressure to make the news presentable, entertaining, to package it for consumption. If they don't we stop watching. But there are a HELL of a lot more TV news resources now than there were then.
Phallic Symbols in LOTR
CNN HEADLINE NEWS, baby.
From the AP:
CNN Looks to Get Hip, Think Young
Wed Oct 2, 5:02 PM ET
NEW YORK (AP) - Is CNN Headline News down with it?
The cable network is trying, judging from an effort emanating from its executive suite to think young.
CNN Headline News general manager Rolando Santos told the San Francisco Chronicle this week that he's looking to mix 'the lingo of our people' -- words like 'whack' and 'ill' -- into newscasts to attract young people.
And the New York Daily News on Wednesday quoted from an e-mail sent by a network manager to his headline writers, sending them a copy of a slang dictionary so they can be 'as cutting edge' as possible.
'Please use this guide to help all you homeys and honeys add a new flava to your tickers and dekkos,' the message said, referring to graphics on the Headline News screen.
The list of phrases included 'fly,' meaning sexually attractive.
Santos said Thursday that the e-mail was designed to point out resources that might help headline writers.
'The e-mail was informational, not a policy or directive from me,' Santos said. 'With that said, I should point out that I want the language used in our tickers and dekkos to be real, current and relevant to the people who watch us.'
CNN underwent a makeover a year ago to add busy graphics to make its screen look like a computer screen. Its ratings have been improving among young viewers.
--------------------
Eh, maybe that wasn't such a great example. "Yo, that suicide bombing is wack!"
Phallic Symbols in LOTR
I've since built some even larger systems; I've no doubt that it's possible to scale Internet streaming media distribution to millions or even tens of millions of simultaneous viewers using today's technology and protocols.
ellbee
You can't fight in here - this is the war room!
Actually I remember only MSNBC being able to cope with 9/11 quickly. CNN was so hard to get to for a long time. Eventually they had to manuall create a stripped down version of the site.
Anyone know how much bandwidth Google actually has at its disposal?
If I had to design a site that had to stay up during a world event, I'd try to talk to the people that were able to keep there site up(mostly) during the last world event. At least that would give me some ideas to work with.
Ha, I did the whole post about world events and didn't mention 9/11 once!. . . D'oh
The Kruger Dunning explains most post on
343 million flood to Google cache as Internet sites drop like flies around the world.
Powered by onion juice.
We heard it on the radio, in the UK, and instinctively went to cnn.com only to get a brief description of what was happening, simply the cnn logo, and a picture of the statue of liberty with the twin towers in the background, whilst it wasn't enough to make a shock about, there surely was nothing else to read about online that day?
even the bbc site went to a series of photos, but TV (and the american way) won out that day.
On Sept 11 I woke up to NPR's description of what happened. (8:00 PST, so after everything was over) Then, I turned on the TV to CNN. Soon, I had to go to work. I quickly found out that ALL the news sites had crumbled under the load (except slashdot). So, I had to turn to an alternative source. I turned to ShoutCast. There were dozens of broadcasts that had switched from music to a feed from CNN, I was really impressed.
I heard a stat that the internet traffic had quadrupled that day, but that hits had stayed the same. Many people complained that the internet had "failed", but we all know that only a few sites had failed, the internet as a whole behaved beautifully.
Travis
P.S. MSNBC also has automatic triggers that remove the graphics from the site when the load gets high.
I don't know if 2.0 is much better, but Apache's design adapts poorly to massive load.
It's a tradeoff; it makes it much easier to run the Perl/PHP/SSI/etc stuff that makes CNN.com and friends so purty, but on the other hand, it's very 'digital' when it comes to load- either you get a server process spawned, get your page served... or you don't, and your client times out, forcing you to hit refresh, which further loads the server.
Bandwidth throttling and other tricks- as seen in thttpd, among others- make it much easier to handle a rapid spike in load, even if your users have to wait for their page to appear at 300bps.
Damn, all of us in the US are screwed!
Read the EFF's Fair Use FAQ
Let's just say, they are well aware of the issues, and a lot of thinking and planning has gone into how they handle the load of major news events.
1) My trusty battery-powered AM radio
2) SkyTel 2-Way Pager
Everything else at my disposal was useless.
The event that is the inevitable point of discussion here is, of course, September 11. We all remember sites like CNN that were too choked to serve, but at the same time, I'm sure many of us remember unconventional news sources -- online journals, for example, and sites like Slashdot -- that proved invaluable for their insight and even for their inconsistency and tendency toward rumor. I remember locating friends in Manhattan through Livejournal, because while the phone lines were all down, many of them had broadband and could still get online. Their accounts of what Manhattan was like at the time considerably enhanced my understanding of the event. Since there is the increasing possibility that witnesses to a major news event will also be participants in an online community of some sort, there is also the increasing possibility that internet will prove indispensible in providing multiple livetime views of what's happening.
I put the "wry" in "riot."
http://news.google.com/
Who doesn't visit Google already anyway? So the load should not increase significantly.
Last year at the USENIX LISA 2001 conference I had the great pleasure to hear William LeFebvre of CNN talk about why the CNN website came down on 9/11 and what has been done to improve their ability to respond to a crisis of this scale. His discussion included details about how users repeatedly hitting "reload" caused basically a DDoS attack and how slashdot relieved some of the load. Also, after the site was back up, he and his co-workers had to swing some servers over to the cartoon network because of all the children who were sent home from school. Here is a link that includes some of the details from that talk. http://www.tcsa.org/lisa2001/cnn.txt
The only option is to use load balancing and PHP, as that is the only way that efficiency can be effectively offered to the typical user, which most sites are not using.
*really dark*
Actually, thinking about all that dark fiber around Africa's cost, one needs to wonder just how much of that dark fiber really exists. Once this current recession is over, we may find that the dark fiber just isn't there.
**really, really dark**
However, I'm not going to guess when this current recession will be over. When the tax load is low, war mobilizes the workforce, temporarily redistributes a lot of money (at a cost of war bonds: having to unredistribute it later, with a bite), and thus spurs the economy.
When the tax load is already high, war drives the tax rate up past the breaking point, and simply results in permanentizing the recession by pushing it all into unsupportable debt. That in turn means that the government will have to nationalize resources through taxes, confiscation, military raids on other countries, and whatnot. In CATO Institute's terminology, I rather suspect that this is equivalent to the government not being held by the rule of law (economically).
Nor will America's recent history hold them back from that.
Since the world economy is, by force (pun intended), tied to the IMF which is in turn tied to America's economy, the CATO Institute's rule for the Argentinan crash applies: A free currency that is not governed by the rule of law results in a destroyed economy. Worldwide. The exceptions will be those few countries *not* tied into the IMF economy.
Nope, I'm not expecting us to pull out of this one.
********
Correct Horse Battery Staple: 72 bits of entropy. Enter "Correct H" into google. When it generates the phrase, that's
-me
Love many, trust a few, do harm to none.
is because the general public wasn't using it for their news source.
Seems fairly obvious to me.
I recall on September 11, google began mirroring news sites from around the globe. Since then they have created a whole news section of their site.
They can handle the traffic. They regularly get millions of hits a day and they never seem to slow a bit and I don't believe they did on September 11 either.
Perhaps a more pertinent question would be, do we actually -need- global news to within 15-20 minutes? And is there actually any kind of accuracy to the stories when they're delivered that fast?
For all we like to read what's going on (I as much as anyone), did knowing minute-by-minute what happened in NYC actually make any significant difference to the outcome? IMHO, no. Planes crashed, people died. Generally speaking, we as individuals have little or no control (in the 6-hours-or-less timeframe, at least) of World news; is it really -that- important that we know so fast?
What percentage of the US population's lives would have been seriously disrupted had they not found out about 9/11 til the following morning, or even the following week (as would have been the case up til around the time of WW2)?
Being able to read about major events and email your friends there to ask "are you OK?" rightaway is, when it comes down to it, a luxury.
Seriously though, a major distributed system could handle a massive load. Maybe that's what news.google.com is about. If the news sites all mirrored the same content (which they pretty much did on sept 11th anyway and do in most major events) they could probably handle the traffic increase between all of them.
As for people using the net versus TV, it happens because TV doesn't provide as much information as people want sometimes. Websites often link to additional info that TV won't cover as it's time to repeat the same report in 5 minutes.
Oh and I thought the net coped pretty well with the last event. Phones were down all day but my b/f in NYC was able to call me in San Francisco using dialpad and keep a connection long enough to wake me up and let me know what was going on.
Introducing the new Occam Fusion! Now with sqrt(-1) fewer blades!
I think this could end up being a future legitmate application for peer-to-peer networking.
Imagine firing up Kazaa (Lite, of course) to get the latest emergency news.
Theoretically the capactity would be quite high. (Although overhead would be high too.)
MPEG at eleven.
Im pretty sure there are more TV's than Internet terminals. Plus lots of people gather round one tv where as they're less likely to do that with the internet. TV news is probably better for big events like september 11th because they interrupt normal programs. http://www.televisionarchive.org has all the major world tv channels on that morning - most of them had no idea what was going on, they just sat there repeating what they knew, but they certainly had it before the internet sites. (although some channels cough bbc didnt even pick it up until 20 mins later). TV is a much more logical way of delivering that sort of news - you can have as many people tuning in as you want with no degrading of the system.
This comment does not represent the views or opinions of the user.
Does anyone else, upon reading the majority of Ask Slashots, think that they are the stupidest questions asked and require nothing more than common sense to figure out?
Since 9/11, I have been doing a lot for different companies to prevent the load problems like the ones caused by the event.
Stuff like load balancing and caching appliance boxes had helped a lot and had proven to handle loads 4 times what came at 9/11 without even slowing down.
my sig
To put in a plug (though I no longer have any ownership interest in it) one of the things about
ClariNet is that it can handle any load.
Because it feeds out news in USENET format from the major wire services, the load is placed on the local server. ClariNet's servers never even feel increased demand. Even highly saturated internet pipes would only slow things slightly, USENET doesn't care about the latency of the pipe.
And all this using 20 year old technology, oddly enough. People always talk about the news sites failing during things like the Olympics, Sept. 11 etc. but the distributed technology never has that problem.
On top of that, USENET is designed for serial news, so that it shows you what's new. You don't have to sit there constantly refereshing a page to see if there is new material, you only see the new material. We even had a system so that urgent stories could be fed directly to your screen, and it's not a polling style of "push" like PointCast was.
Generally the newsreader is, surprise surprise, a great way to read news. What surprises me is that all these years later -- ClariNet was the first of the dot-com companies -- nobody has done the same. I sold it 5 years ago, but it's still running, if a bit shrunk from the economy.
Has it been over a year since you last donated to the Electronic Frontier Foundation
Unless you make realtime financial decisions, news is just another form of entertainment.
Hmm searched Google the other day do find some fancy ways of getting Apache statistics into MRTG, and I ran into some BBC mrtg stats right on top.
hmmm
my sig
And yet, even in the throes of 9/11, Slashdot itself fared better than CNN, MSNBC, Foxnews, and others.
The great thing about the internet generally is the ease with which content can be ripped off. If I get into a news site, I can easily mirror / cut and paste into a Yahoo chat room, onto /. etc...
/. was a damn good source of news when all the news sites were knackered. If slash-knitting, slash-boarder and slash-hump all join in on the act that takes a decent chunk of the population away from the big news sources - freeing them up for others.
As the use of the internet develops more granularity - i.e. people spend more time in smaller groups, not all huddling around google, bbc and yahoo, this will become a viable route for this 'big news' to get through.
About a year ago
Then your just waiting for the whole net to crumple under the load of a hundred million people IMing each other with "do you have any new news???"
As long as /. and new.google.com is up I'm sure we'll be ok. Or at least I'll still get my nerd news!
Karma: Bizzare (mostly affected by varying internal caffeine levels.)
But we're talking about when major world news stories like 9/11 hit. On 9/11, most TV stations were showing constant news coverage, preempting regular programming and often commercials too. If the news is important enough, they make time for it.
Is your question *really*, will the internet handle streaming broadband from a single source to millions of viewers. The answer is no, that's not what it's made for. Broadcast is much more suited to this.
We had this discussion about 4 years ago. It turned out that the "reliable news source" was really just a bugaboo put up by the existing media conglomerates who didn't like the fact that people might be able to form opinions or provide content on their own. Their fears turned out to be unfounded, but the illusion of "independent media" has been a terrific staging ground for them to build hype, test new ideas, and espouse views they don't dare make directly.
But if you mean "can new spread on the internet?" then the answer is a big "duh!" The real problem is controlling information, but as I alluded to above, they've got that pretty well under wraps, not as well as they'd like, but they're getting closer to it all the time.
I couldn't hit any news at 8:45 AM on 9/11. I had
to pull a crappy rabbit-ear set out of the closet
to know WTF was going on and if I was dressed for
WWIII or not. The web is NOT ready to cover events
of this (or, god help us, greater) magnitude.
as in, those with CNN will definitely want to go to the web ;-)
Those few countries not tied to the IMF wouldn't happen to include the USA, Japan, Canada, Great Britain, Germany, and a couple others that make up 95% of the world's enconomy would they?
My point was that, while the IMF may seem a big deal to countries like Thailand or Sierra Leon, and have the potential to affect others, like Argentina; it isn't really that big of an organization. It's hardly the spectre you think it is, and frankly, doesn't impact your life at all, except to the extent your heartstrings are tugged by Ticketmaster forcing you to attend the next U2 concert.
So ... internet news is better than television news because you were able to use the internet to talk about what the television was showing you?
" particularly those without access to a quality television news service."
Oh really?
Since when is it possible to use "quality" and
"television" in the same phrase except to point
out it has none?
There are a couple of thousand news sites around the country that are operated by local newspapers. On Sept. 11, most of them reported DROPS in traffic because their regular users were, in general, off watching the whole scene unfold on television.
Meanwhile, a handful of national sites (and the Associated Press wire.ap.org site) were either swamped or nearly swamped with traffic, mostly by people in offices with no access to TV. Many of those people aren't regular users of their local newspaper sites and may not even know they exist, but they've heard of CNN.com.
There's plenty of capacity. That's not the problem. The problem is overreliance on a small number of centralized distribution points. It's a bit like the problem of overreliance on a small number of centralized viewpoints, but not quite.
I've been doing online news since early in 1994, and I now work for a company with some 30 newspapers across the country, so I know something about traffic patterns under stress. 9/11 wasn't the first major Internet news event, you know.
In the next few weeks, our company will be migrating from the use of wire.ap.org (central point of failure) and instead will be distributing wire coverage from our own servers, which performed admirably on 9/11.
The BBC took the load just fine, and now take a larger load daily!
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
Akamai (and its competitors, AT&T and Speedera) have a business model that says they put lots of caching servers out in the network and sell caching to the web content providers who want to get their content out. By contrast, the original web caching was a pull model - businesses with firewalls and some ISPs use either transparent or explicit-proxy-based caching to cache *incoming* content at their gateways or other concentration points (e.g. cable modem network head ends), and they cover the cost of the caching equipment by reducing their bandwidth needs as well as by giving users service that's perceived to be better. Flooding networks like Usenet are good for non-realtime multicast-like behaviour, and multicast is good for streaming but could also be integrated with caching systems. Back during the Internet boom, there were several companies such as I-Beam that used satellite broadcasting to push content out to caching servers, but alas, Chapter 11 has eaten most of them.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
I probably shouldn't be starting a new thread this late, but I can't see an appropriate place to post this.
I was a sys-admin for a non-US news website during September 11. Basically we were hit hard, mainly because of the time-zones. Because the organization I worked for (I don't name them, but you should be able to figure it out if you know me) mainly produces old-media news, and re-purposes most of that for the Internet. Because of the time-zone difference, most of the action was over by the time I got into work.
Apparently, the late TV news shift had just finished the last broadcast for the night and was heading home when the news broke. They turned around and stayed on deck until the morning shift came in to take over.
We received a weeks worth of hits in less then 24 hours. Our load-balanced redundant web servers were purring along, not quite maxed out but very little room to spare.
The biggest killer was bandwidth. Looking at the bandwidth stats our international PVC (about 1/4 of our capacity), maxed out early in the morning, and was taken down briefly twice to increase its share of the total bandwidth.
In the end, we reached over 80% utilisation of our total pipe. This may not sound much, but at the time we had never used more then 40% of what we had available that day. I think the only reason we didn't go any higher was that something upstream was maxed out. My guess is the US link out of my country.
Several steps were taken to improve the performance of the website during the day. The main page was replaced by a news summary with a link to the old main-page. Most people only wanted the latest news on New York, so they could get that without hitting the rest of the content. We had to fine-tune the web servers a few time, and I've already mentioned the tuning to increase the share of international traffic.
HTTP was not the bandwidth killer. Because of the extended news coverage, the video from the news studio was streamed directly onto the Internet. Usually we use static video files or live stream specific shows, but the video stream was on for something like 12 hours, and that killed our pipe. I presume the streamed radio stations were also popular, but I haven't seen the statistics on that.
We survived, just. The biggest problem was that we were not ready, and that we had to react. If we were fully ready, or we could have reacted more quickly, then we would have done much better. The trouble was that these events happened during the night our time, and the staff on at that stage didn't know that there were things that the day staff could have done to help the load problems.
I don't think that you can expect a news site to be able to fully deal with an event like September 11. You can't justify having 10 times the bandwidth you normally use, just for a once in a decade event. You have to aim for the once-a year event, and try to deal with the other cases as best you can.
Democracy isn't about no one telling you what to do. It's about everyone telling you what to do.
...particularly those without access to a quality television news service.
Quality television news service? I'd kind of like to get some of that myself. None of the 417 channels I get has it.
It's simple: I demand prosecution for torture.
Way back during the Gulf War, I was playing on a MUD (Dartmud) and was getting scud raid news from a guild member based in Israel before the news about it came on (usually just before or after he headed to or came back from the bomb shelter). I also usually knew more about what was going on than my roommates, who were watching the war on TV.
So yes, sometimes the internet is better than TV, and no, I don't need to talk about what's on TV, as they sometimes don't really know what's going on, either.
Totally agree.
So there I was on 9/11, a British guy on business in Boston, wondering (amongst many other things) how badly my parents would worry when they heard the news. I was pretty sure the international phone circuits would be clogged, so didn't even try; I jumped into IRC and eventually reached a friend who knew my parents in real life and gave them a call.
what if it's an electromagnetic bomb and the power gets fried everywhere? that almost scares me more than a biological weapon. can you imagine a major city with all the power fried?
J-Log: Journalism News, Media Views
As the use of the internet develops more granularity - i.e. people spend more time in smaller groups, not all huddling around google, bbc and yahoo, this will become a viable route for this 'big news' to get through.
That's assuming the Internet doesn't get legislated down to "Television Plus" thanks to Disney et. al.
deus does not exist but if he does