UUNET/WorldCom Backbone Diffiiculties

FearlessFritz writes "UUnet seems to be having a bad time recently. Several sites in the SouthEast of the US have been slow or down. Here is Worldcom's quote from their web page: 'WorldCom is currently experiencing an interruption of service in various hubs in the U.S. We are working to restore a routing anomaly, and making necessary progress toward resolving this disruption in service.' There are several rumors abounding, but the best is that they performed a hardware upgrade that failed. Is anyone outside of the Southeastern U.S. experiencing the effects of this outage? (I am peered to several providers so I can post!)"

Comments

Probably an IOS upgrade gone to hell...

[Agent Green]

Anyone who's done any kind of IOS upgrading on some of the upper-end Cisco routers and Juniper routers knows that the upgraded images aren't always the most stable items around.

At one point, there was a severe outage at Genuity referred to as "Black Tuesday", when an IOS upgrade sunk a majority of the network and caused a ripple that made for a really shitty morning.

That was a few years ago, though. I can't go into the specifics of the RFO...but the failure was a very visible issue which resulted in modifications to the testing and change management processes.

Unfortunately, sometimes testing production software doesn't sufficiently break until actually put into production.