Slashdot Mirror
Are Internet News Sites Ready for Major World News?
An anonymous reader asks: "Heading says it all really - are Internet news websites ready for the next big world event? news.bbc.co.uk already switches format under heavy load (not sure if this is automatic or not) and i'm sure some other sites do the same. But should a major world event take place in the coming months/years, the Internet is going to be the primary news source for many millions of people, particularly those without access to a quality television news service. How will / can it cope?"
personally, I didn't like watching TV on 9/11/02. They were repeating the same garbage all day long. The reporters were rather boring and the news coverage the same.
/. (as most other news sites were unreachable due to traffic) b/c of not only the news but also discussion w/others. It was interesting to read what other people were feeling, especially those that were not in the US.
I preferred to read
On September 11th, major news sites like Yahoo, BBC, CNN were entirely flooded with traffic much like the phone system was, going as far as taking down some fairly large servers altogether. What ended up happening was that a bunch of IRC channels (specifically on SlashNET) cropped up with people giving live webcam shots, rumours and snippets of information, mirrors. Then the CNN closed captioning bots started relaying to IRC for those without the cable service. It was interesting as it showed the Internet both failing at and succeeding in its primary designed function, as a communications and information network that could survive a major catastrophe.
s200.org - visit it (me), love it (me).
My observation for 9/11 waw that major news site crawled under the load. However, less often visited news site were responsive all day and gave the same news with the same level of coverage than the big news sites.
So I must say, find some smaller news site and bookmark them. When your big-shot news site will crawl under load, just go to the small one and you will get your news.
BTW if you just want nice video, the Intenet is not the place to go, turn on your TV, you'll get far better image quality and you don't have to wait until the video is buffered.
news.google.com should hold up under even the heaviest loads, and while you might not get the actual site it links to, you should at least be able to get the idea of what's going on based on the headlines.
In a time of crisis, is it really necessary to know the details of a major world event immediately? If a nuke goes off somewhere, I'm not too concerned about who did it--I'm driving to some remote place, THEN I'll start asking the questions.
On Sept. 11th, what did we know for certain:
*4 planes were hijacked
*Two towers fell
*The Pentagon was hit
*A plane went down in PA
everything else was mere specualtion at the time, and everything above could be read by headlines alone.
Just a thought,
sig--we don't need no goddamn sig
The answer to scalability has been there for years, and it's multicast. Multicast is a protocol that implement a one-to-many distribution of the information, allowing very efficient distribution of contents on the internet (the target is that the information should not pass more than once on any given physical line), and dynamic group joining and leaving.
However, ISP and users are confronted to a chicken-and-egg problem: ISP pretend there is no demand for multicast, so that can't justify the investment in increased NOC knowledge, users don't know what it is, and content providers have no support from ISP or user.
Multicast is however the scalable answer for live broadcast and scheduled replay, it's been there for years and I do not loose hope that it will be better used one day.
During the horror of the attacks last year, I was surprised and thankful for CNN's approach which allowed them to withstand the barrage of hits:
They switched to an old-school, how-the-web-used-to-be, no-nonsense design. It was basic HTML, with some embedded pictures that contribued to the information. No frills, no ads, no sidebars about the latest crap-news, just the information we were looking for. Needless to say, it also ate a lot less bandwidth.
Of course, they were down part of the morning, but when they came back in the altered format, I thought it was a great move. A few other sites were doing the same thing, and I think they'll remember the technique for the next time something big goes down (hopefully something pleasant next time? I can hope...)
Better still, get a wind-up radio such as the Freeplay ones. Then you don't need to worry about batteries, as you can run them off solar power if outdoors on a sunny day, or just wind them up every so often.
I've got one of the AM/FM models and it's quite good and saves having to get batteries. They also do shortwave models.
"Information wants to be paid"
It isn't just the news sites we have to think about. We should also be asking, when the next big event does happen, will people even be able to get online to access the news sites?
I'm not talking about some sort of damage to the communications network. I'm talking about ISPs that enforce strict rules on how many of their customers can get online simultaneously. They are the real threat to the Net as a primary source of urgent information, and it's all about money. They take on millions of customers but total capacity is measured in tens of thousands.
For example, on September 11th there were a few hours when tall buildings in London and other British cities were being evacuated, but many people over here couldn't get online to access vital information because our ISPs have notoriously low capacity and only allow a small percentage of their customers online at any one time.
Obviously this is a greater threat in rural areas because the only available connection method is dial-up.
I am subscribed to a couple of worldwide mailing lists and I have found that email simply rocks in high 'net traffic situations.
During the New York tragedy, much of the traffic on those lists was along the lines of "I can't get to the major sites because the web is clagged solid - can anyone tell me the latest?". And thankfully for a couple of days, the rules about straying from the topic of the mailing list were ignored.
Granted, many of the complaints were actually related to individual corporate firewalls, http gateways and proxy servers, rather than the sites themselves, but the situation stands: for whatever reason, you can't get to the site. Our web proxy fell over under the load, but our SMTP gateway just kept on going. And so did most others around the world. And I imagine that NNTP stuff worked just as well the SMTP stuff.
Remember folks, the Internet is a lot more than the Web!
Ahh - My eye!
The doctor said I'm not supposed to get Slashdot in it!
It was Princess Di's death. I was on shift the night it happened and it pretty much brought all news websites to their knees. That was the first time I noticed the low bandwidth version of CNN. At first I thought the site was choking because it looked like some graphics were not loading.
Still, I'll give it to Slashdot and to IRC. I spent most of 9/11 on IRC transcribing what was being reported on CNN, since for a while the site was pretty much useless. A bunch of us where also taking screen captures and posting them online so people could see the horror. I still have captures of the first flyover of the Pentagon, which is less than 10 miles from my office.
Pedro
----
The Insomniac Coder
Whether the fibre is dark or not isn't the problem. "The Internet" can handle the ammount of traffic that an event would create. The problem is the servers that everyone want to access will fail under that load. I bet dollars to donuts that news sites were going down on 9/11, but the sites where you can see chicks going down on each other were just fine that day.
You're probably right; the report notes that the infrastructure was fine but the web servers were overwhelmed. Lighting up that dark fibre would make it easier to deploy Akamai-like solutions to replicate content to distribution points closer to the consumer.