Slashdot Mirror
UUNET/WorldCom Backbone Diffiiculties
FearlessFritz writes "UUnet seems to be having a bad time recently. Several sites in the SouthEast of the US have been slow or down. Here is Worldcom's quote from their web page: 'WorldCom is currently experiencing an interruption of service in various hubs in the U.S. We are working to restore a routing anomaly, and making necessary progress toward resolving this disruption in service.' There are several rumors abounding, but the best is that they performed a hardware upgrade that failed. Is anyone outside of the Southeastern U.S. experiencing the effects of this outage? (I am peered to several providers so I can post!)"
I have had very slow internet access most of the day in the upper midwest. The problem started at about 8:15 CT this morning.
Im in Alexandria, VA, and I've had horrible service all day long. I could get local sites, but nothing far away. Pinging slashdot wont even work half the time :(. And no google....
I can verify this issue. I work for a company out of the DC area and our data lines are provided by UUNet. I live in Kansas though and do all my work via SSH to the servers in DC. This morning my SSH sessions would randomly seem to hang. After a bit of investigation I started noticing it was taking like 600ms per hop once I got to the UUNet network. So that is North-East there.
Unstable Apps: Our Android Apps Don't Suck
a whole lot of red over at the InternetTrafficReport any other good informative sites?
May this post be indexed by spiders, and archived for all to see as my Internet epitaph.
I work at a small hosting company and our UUNet connectivity (Central California via Anaheim hub) has been screwed since around 6am pacific time. Up and down all morning with latency between 500 and 2000 ms when it is up. Yay worldcom.
Check it out here
http://www.merit.edu/mail.archives/nanog/msg040
nothing concrete and MIDS doesn't show anything on the weather reports (not that it means anything).
Conceptually, the logic states that there should be multiple backbones through multiple geographic areas, such that a failure of one provider could be dealt with by routing traffic through the alternate backbone. Realistically this is difficult and expensive, and the primary reason that there are very few top tier connections running across the united states.
If you look at the map from 1992 (NSF Net | XO OC192 Network), you'll notice that there really are only 2 main paths from east coast to west coast. The southern path is probably at least slightly affected by the incoming hurricaine, and the northern path seems to be overloaded or failing for some other reason.
Precautions? Make sure the hardware is sound and easily replaced, and that alternate routes are available in case of failure. The problem is finding alternate routes that aren't completely congested due to the failure.
Mooniacs for iOS and Android
There's been discussion of this on the NANOG list, and my DS3 in Chicago was taken down hard by this. Physical layer okay, but traffic died once it was two or three hops into UUnet/Worldcom's core. First outage was from 2am to 8am, second outage from approx. 10:45am (CST) to 2pm. The master tickets for this outage are 651744 (DS1 and below) and 651751 (DS3, OC3 and above). I just got off the phone with Worldcom's NOC and the story I got is that all the border routers that took a dive are back up save a few that they're bringing back up here in Chicago. Worldcom has provided confirmation that the Reason For Outage was a wildly unsuccessful BGP config propagation.
. We've got computers, we're tapping phone lines, you know that ain't allowed - Talking Heads, "Life During Wartime"
Following is WorldCom's maintenance announcement about today's work, which I recieved because WorldCom is my company's broadband ISP.
During the Normal operations window on Oct 3, 2002
WorldCom will be performing the following scheduled maintenance
activities.
This activity is scheduled to take place from 3:00 a.m. to 6:00 a.m.
(local hub time) in the contiguous US and elsewhere from 3:00 a.m. to
7:00 a.m. (local hub time) and may affect your connectivity. The
following
customer ID will be impacted: XXXXXXXXX.
If you have any questions, please contact our local Customer Network
Support Center. Please reference the internal ticket number 645346.
Quality System Management-Global Maintenance Planning
Worldcom (http://www.uu.net)
1(800) 900-0241 / +1(703) 886-5440
WorldCom United States 1-800-900-0241 (select the following options in
order: 2, then 4, then 1)
WorldCom Denmark (45) 80.30.50.50
WorldCom Italy (39) 02.3600.1887
WorldCom Sweden (46) 8.750.88.50
WorldCom Switzerland (41) 1.580.86.11
While the Wired News article says that some people were speculating that it was the Slapper worm, other people were speculating that it was a fiber cut, but it first quotes the UUnet page which says they're having a routing anomaly and that it was affecting multiple gateways. That means it's not likely to be a cable cut, because that would be more localized, and it's also not likely to be the Linux worm because the routing stuff isn't happening on Linux boxes - it'd be either Cisco or Juniper, and I'm not aware of any reports that the worm affects those platforms.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks