Slashdot Mirror
Are Internet News Sites Ready for Major World News?
An anonymous reader asks: "Heading says it all really - are Internet news websites ready for the next big world event? news.bbc.co.uk already switches format under heavy load (not sure if this is automatic or not) and i'm sure some other sites do the same. But should a major world event take place in the coming months/years, the Internet is going to be the primary news source for many millions of people, particularly those without access to a quality television news service. How will / can it cope?"
Hopefully news.bbc.co.uk can cope with having it's link on Slashdot's homepage...
How many people have internet access, but no access to TV, radio, or other broadcast recievers? For major news stories boardcast medium will always be the main method of disseminating information to the masses, client-server systems aren't really designed for this purpose.
Why? Are you planning one?
Al.The Daily ACK - Eclectic posts by yet another hacker
personally, I didn't like watching TV on 9/11/02. They were repeating the same garbage all day long. The reporters were rather boring and the news coverage the same.
/. (as most other news sites were unreachable due to traffic) b/c of not only the news but also discussion w/others. It was interesting to read what other people were feeling, especially those that were not in the US.
I preferred to read
No they can't cope. It's been proven already. Even giants like the BBC and CNN had several moments where they could not handle the load on September 11th.
I'm sure that they have taken steps to improve things in the future but, there is only so much that you can do, or at least do cost effectively. There is no substitute for hardware and bandwidth but, maintaining enough to support the entire planet at one critical moment in time, that may or may not come, is not cost effective.
When the time comes, the news sites will buckle under the load, just as the telephone system does. The best source for news, during times of disaster are television and more so, radio. Even in the most remote places, you can still get radio and with new satellite radio, you can get it anywhere.
Keynote have oublished a reporton the performance of major web sites on September 11th, 2001.
Of course, there's a lot of dark fibre around, so the capacity is there if it's really needed. Once the current recession is over, we can expect to go back to the days of massive overprovision and redundancy as content and bandwidth providers seek to build in capacity to handle peaks. What will really help is multicasting for video streams, and well-designed caches at ISPs.
"...particularly those without access to a quality television news service."
Gee, that's pretty much everyone.
On September 11th, major news sites like Yahoo, BBC, CNN were entirely flooded with traffic much like the phone system was, going as far as taking down some fairly large servers altogether. What ended up happening was that a bunch of IRC channels (specifically on SlashNET) cropped up with people giving live webcam shots, rumours and snippets of information, mirrors. Then the CNN closed captioning bots started relaying to IRC for those without the cable service. It was interesting as it showed the Internet both failing at and succeeding in its primary designed function, as a communications and information network that could survive a major catastrophe.
s200.org - visit it (me), love it (me).
My observation for 9/11 waw that major news site crawled under the load. However, less often visited news site were responsive all day and gave the same news with the same level of coverage than the big news sites.
So I must say, find some smaller news site and bookmark them. When your big-shot news site will crawl under load, just go to the small one and you will get your news.
BTW if you just want nice video, the Intenet is not the place to go, turn on your TV, you'll get far better image quality and you don't have to wait until the video is buffered.
news.google.com should hold up under even the heaviest loads, and while you might not get the actual site it links to, you should at least be able to get the idea of what's going on based on the headlines.
In a time of crisis, is it really necessary to know the details of a major world event immediately? If a nuke goes off somewhere, I'm not too concerned about who did it--I'm driving to some remote place, THEN I'll start asking the questions.
On Sept. 11th, what did we know for certain:
*4 planes were hijacked
*Two towers fell
*The Pentagon was hit
*A plane went down in PA
everything else was mere specualtion at the time, and everything above could be read by headlines alone.
Just a thought,
sig--we don't need no goddamn sig
The answer to scalability has been there for years, and it's multicast. Multicast is a protocol that implement a one-to-many distribution of the information, allowing very efficient distribution of contents on the internet (the target is that the information should not pass more than once on any given physical line), and dynamic group joining and leaving.
However, ISP and users are confronted to a chicken-and-egg problem: ISP pretend there is no demand for multicast, so that can't justify the investment in increased NOC knowledge, users don't know what it is, and content providers have no support from ISP or user.
Multicast is however the scalable answer for live broadcast and scheduled replay, it's been there for years and I do not loose hope that it will be better used one day.
It was Larry Niven who predicted the idea of "flash crowds". Of course, he was envisioning physical crowds via teleportation, but the basic idea still holds. It's only going to get worse as more and more people use the net.
Look at it this way: in a primative society, a clan or village would usually have a storyteller or sage who gathered the news of the world in story form and re-told as appropriate. We should not be supprised that it takes millions, perhaps even hundreds of millions of people to be the story-tellers to 6 billion (that's a US billion).
If the Internet had a higher percentage of useful sites for news (not just talking jpeg-heads, but innovative ways of conveying the STORIES that the news represents), then no one of them would be loaded down and the backbones would be the only bottleneck. Notice that so many of us flocked to Slashdot when the towers fell? Wonder why? Because Slashdot, for good or ill, is our community's storyteller, and we instinctively come here to understand how our community is reacting.
Folks,
If you don't have a portable AM/FM radio, or even better a shortwave receiver, then get one TODAY. Get some spare battteries for it as well.
The simple fact is if you want to hear what's going on during a "major event" radio is the best way to do it. And you have evacuate in a hurry, you sure as hell aren't going to be taking your 60" flat screen TV with you. You want pictures, wait for the evening news, if you want to know what's going on NOW, get a radio.
Even better, get yourself licensed as a ham radio operator so you can be part of the communication solution if needed (yes, amateur radio is still important, even today).
You know you're a geek if you've ever replied to a tagline.
During the horror of the attacks last year, I was surprised and thankful for CNN's approach which allowed them to withstand the barrage of hits:
They switched to an old-school, how-the-web-used-to-be, no-nonsense design. It was basic HTML, with some embedded pictures that contribued to the information. No frills, no ads, no sidebars about the latest crap-news, just the information we were looking for. Needless to say, it also ate a lot less bandwidth.
Of course, they were down part of the morning, but when they came back in the altered format, I thought it was a great move. A few other sites were doing the same thing, and I think they'll remember the technique for the next time something big goes down (hopefully something pleasant next time? I can hope...)
Better still, get a wind-up radio such as the Freeplay ones. Then you don't need to worry about batteries, as you can run them off solar power if outdoors on a sunny day, or just wind them up every so often.
I've got one of the AM/FM models and it's quite good and saves having to get batteries. They also do shortwave models.
"Information wants to be paid"
It isn't just the news sites we have to think about. We should also be asking, when the next big event does happen, will people even be able to get online to access the news sites?
I'm not talking about some sort of damage to the communications network. I'm talking about ISPs that enforce strict rules on how many of their customers can get online simultaneously. They are the real threat to the Net as a primary source of urgent information, and it's all about money. They take on millions of customers but total capacity is measured in tens of thousands.
For example, on September 11th there were a few hours when tall buildings in London and other British cities were being evacuated, but many people over here couldn't get online to access vital information because our ISPs have notoriously low capacity and only allow a small percentage of their customers online at any one time.
Obviously this is a greater threat in rural areas because the only available connection method is dial-up.
I am subscribed to a couple of worldwide mailing lists and I have found that email simply rocks in high 'net traffic situations.
During the New York tragedy, much of the traffic on those lists was along the lines of "I can't get to the major sites because the web is clagged solid - can anyone tell me the latest?". And thankfully for a couple of days, the rules about straying from the topic of the mailing list were ignored.
Granted, many of the complaints were actually related to individual corporate firewalls, http gateways and proxy servers, rather than the sites themselves, but the situation stands: for whatever reason, you can't get to the site. Our web proxy fell over under the load, but our SMTP gateway just kept on going. And so did most others around the world. And I imagine that NNTP stuff worked just as well the SMTP stuff.
Remember folks, the Internet is a lot more than the Web!
Ahh - My eye!
The doctor said I'm not supposed to get Slashdot in it!
Multicast news services worked well during 9/11 and there is no reason to think that they won't the next time. Multicast is specifically designed not to "melt down" under extreme changes in audience.
The trouble is that not everyone is multicast enabled, but this shows real promise in handling news and emergency information over the Internet.
Ever since 9/11, I've noticed that the heavily-trafficked sites cope with sudden floods of hits by switching over to static pages with minimal graphics. The NY Times, for instance, did this when the AA flight went down in Queens last November. CNN's done it a couple of times as well.
When we're looking at scale, though, it's useful for us to remember that these sites can handle way more traffic than even the typical slashdotting can deliver. Most breaking major news can be handled by them with only a little bit of slowdown. It's only the 9/11-scale events that can really bring the news sites to their knees - so lets hope that we don't have to see anything that brings on a overload scenario for the big news sites.
The other thing to consider is that most of the news providers are still investing some money in their infrastructure - just less than before. It's very well possible that a 9/11-scale event might not hammer the servers the way they were hammered last year. A lot of web sysadmins learned valuable lessons that day that I'm sure have been applied since then.
-- Josh Turiel
"2. Do not eat iPod Shuffle."
It was Princess Di's death. I was on shift the night it happened and it pretty much brought all news websites to their knees. That was the first time I noticed the low bandwidth version of CNN. At first I thought the site was choking because it looked like some graphics were not loading.
Still, I'll give it to Slashdot and to IRC. I spent most of 9/11 on IRC transcribing what was being reported on CNN, since for a while the site was pretty much useless. A bunch of us where also taking screen captures and posting them online so people could see the horror. I still have captures of the first flyover of the Pentagon, which is less than 10 miles from my office.
Pedro
----
The Insomniac Coder
Sept 11 is just on the left of graph at the bottom. Interestingly a normal day's traffic is now greater than the sept 11th spike, maybe they could handle major news events.
(I can't think what the early april spike is, but the raised traffic in june/july is the world cup)
But should a major world event take place in the coming months/years, the Internet is going to be the primary news source for many millions of people, particularly those without access to a quality television news service.
Please be advised that your set needs adjusting... It's pretty clear from the evidence (and from a phenomenological point of view if you observe your own reactions) that the experience of watching a major event on television as it unfolds barely qualifies as useful information, due in part to the nature of the medium, but largely due to the nature of media filters and techniques. When you see something like 9/11 going on, it's much closer to entertainment, unfortunately, than providing one with reconnaisance leading to rational behaviour. The drama of the moment helps you develop powerful emotions in relation to the event, but what kind of info do you really get?
When it comes to war, TV obscures. For instance, see this study on media and the gulf war. [Remember that? Oh wait, it's still happening.] A salient quote:
In other words, you'd actually be better off combing through usenet than sucking on the immediacy of the glass teat.
Qualifier: I've worked in media-democracy-oriented film/video for years, I'm involved and devoted to the medium!
Damn those pesky terrorists
I've since built some even larger systems; I've no doubt that it's possible to scale Internet streaming media distribution to millions or even tens of millions of simultaneous viewers using today's technology and protocols.
ellbee
You can't fight in here - this is the war room!