Slashdot Mirror
How a Router's Missed Range Check Nearly Crashed the Internet
Barlaam writes "A bug by router vendor A (omitting a range check from a critical field in the configuration interface) tickled a bug from router vendor B (dropping BGP sessions when processing some ASPATH attributes with length very close to 256), causing a ripple effect that caused widespread global routing instability last week. The flaw lay dormant until one of vendor A's systems was deployed in an autonomous system whose ASN, modulo 256, was greater than 250. At that point, the Internet was one typo away from disaster. Other router vendors, who were not affected by the bug, happily propagated the trigger message to every vulnerable system on the planet in about 30 seconds. Few people appreciate how fragile and unsecured the Internet's trust-based critical infrastructure really is — this is just the latest example." Vendor A, in this case, is a Latvian router vendor called MikroTik.
Is this related to the story posted that stated:
"One Broken Router Takes Out Half the Internet?"
http://tech.slashdot.org/article.pl?sid=09/02/16/2233207
It just amazes me how differently presented this story is compared with the previous.
In fairness, there is much more information about this 'outage' now.
This news is alarming. Thanks for not making in alarmist this time.
Vendor B is Cisco btw. Dunno why they were being vague.
If you have no clue what offensive fortunes are try 'fortune -o'.
(in bold) Please, please, please request a potentially offensive fortune if and only if you believe, deep down in your heart, that you are willing to be offended...
If you don't have fortune installed then you are clearly on the wrong website ;)
Hey, I've got it installed! "fortune -o" says: No fortunes found.
(Sorry, I'm new.)
The summary used Company A and Company B, the editor's comment tagged the Latvian vendor.
Speaking of RTFA'ing you should maybe take your own advice:
As it turns out, the reason for all those routing resets and general instability was due to a previously unknown Cisco bug involving AS paths close to 255 in length. If you try to prepend to a long path that you receive and by doing so, create a path longer than 255, you are toast. So the maps we gave in our our last blog were more of an indication of Cisco market share (at least among prependers), rather than the propensity of outdated routers. Kudos to Ivan for figuring this out.
You have to have a support agreement with Cisco to get the latest IOS. They won't even give you the last version when your support contract ran out. Also, older routers do not always have upgrades available for various reasons, either they do not have enough space or hardware limitations or Cisco End-of-Lifed it and hasn't bothered.
There's also the "if it isn't broke don't fix it" mentality in the networking world. A new version may fix some bugs but it might add some bugs as well. An upgrade, even if minor, generally means a lot of work testing and reconfiguring before you roll it out. Network engineers are expensive and that time isn't free. Sometimes the devil you know is better then the devil you don't.
In an ideal world it wouldn't be an issue, but when it comes to networking it's NEVER an ideal world. There's always too much to do and never enough budget/manpower to do it. Every network admin probably has 10 things on his mental wishlist right now, upgrades he would like to make, redundant hardware he would like to purchase, failover contingencies he needs to test, etc. Upgrading IOS on an old router in a rack somewhere (and hoping it doesn't blow up in your face) can be pretty far down the list.
The critical bug is with the Cisco routers; a Mikrotik router merely nearly triggered the bug.
It would be possible to trigger this bug with any routing software that does not do range checking on the amount of times the ASN is pretended.
The summary is spreading FUD by making Mikrotik, the only named vendor in the summary, look like the vendor at fault.
You should check out alterslash.org. It's an excellent way to sort through the shitty /. comments and get to some decent threads.
Mikrotik are known GPL violators, that use a modified Linux (they re-branded that as "RouterOS") and a terribly bad implementation of the BGP protocol..
In some custom community network, where MikroTik has been deployed internally, that stolen-Linux is being hacked to use the Quagga instead of MikroTik's BGP.
In short: that "RouterOS" has been higly unsuitable for the Internet. I can't believe somebody was so stupid to trust it.
Or, they move it to a separate package. For example, on Ubuntu, this is fortunes-off.
No need to make it more complicated than it is.
Don't thank God, thank a doctor!
That happened to my account once when I bitched about an editor too, almost ten years ago now. (Within a week of pretty simple, thought-free karma-whoring comments, I was back posting at +2.)
"Remember when the U.S. had a drug problem, and then we declared a War On Drugs, and now you can't buy drugs anymore?"