Slashdot Mirror

← Back to Stories (view on slashdot.org)

Researcher Resigns Over New Cisco Router Flaw

Posted by samzenpus on from the don't-go-down-with-the-ship dept.

An anonymous reader writes "Michael Lynn, formerly a researcher for Internet Security Systems resigned today rather than conceal his research into serious new flaws in Cisco routers, according to stories at Washingtonpost.com and CRN. Interestingly, Cisco says the the problem is not a security vulnerability, although it chided Lynn for not going through proper vulnerability disclosure channels. Both stories note that Lynn is in danger of being sued by Cisco for revealing the information, details of which were pulled at the last minute from the materials handed out to Black Hat attendees." Update: 07/28 12:23 GMT by Z : SimilarityEngine writes "Cisco and ISS are filing a law suit against Michael Lynn and the management of the Black Hat Conference, following Lynn's presentation discussing a vulnerability in IOS."

28 of 423 comments (clear)

  1. I wonder... by leonmergen · · Score: 1, Interesting

    From the article:

    According to several people who made it on time to the 9 a.m. presentation, Lynn began his talk with a discussion about security issues surrounding services that allow people to make Internet-based telephone calls. Then, they said, Lynn suddenly changed topics and began discussing the highly technical details of his research into the Cisco flaw, saying he would rather quit his job at ISS than keep the information from conference attendees.

    Why would anyone, after clearly being informed NOT to talk about this information, talk about this information ?

    I know, freedom of information ideals and the like, but couldn't he at least have waited a few weeks to see how Cisco responds, instead of simply revealing the information of a hardware-level exploit

    --
    - Leon Mergen
    http://www.solatis.com
    1. Re:I wonder... by Anonymous Coward · · Score: 1, Interesting

      When do you reckon the research would have been finished? Another few weeks? A couple of months? Why not give it a couple of years, just to be on the safe side...

      What the hell do you expect them to say? "The decision was made on Monday to pull the presentation because it would make us look like morons caught with our pants down around our ankles...?"

    2. Re:I wonder... by Cereal+Box · · Score: 4, Interesting

      The way they prefer it to go is that someone contacts them secretly, tells them the hole, and they can have it fixed all up by the time the vulnerability is published.

      Then they get to look super-secure, since they were "too quick" for the bad hackers.

      ... And this happens in the Open Source world too. Mozilla, for instance, has "classified" bugs, which are not opened up to the public until a fix (or whatever) is available. Take for instance, the Windows chrome:// bug from a few months to a year ago. They sat on it for over a year (and it was classified, of course), and didn't do anything until an exploit appeared in the wild. The fix was issued right away. "Too quick" for the hackers, indeed.

      What I'm getting at is don't say that this sort of behavior is limited solely to closed source software. No one wants to have the pressure of handling a security fix WHILE an exploit is out in the wild. Would you rather have the opportunity to fix a security flaw while no one else (but the person who discovered it) knew about it, or would you prefer the person who discovered it announce it to the world and release an exploit first?

    3. Re:I wonder... by n0-0p · · Score: 4, Interesting

      That was true a few years ago, but its rarely the case these days. Once you contact the correct people at the vendor they generally move fairly quickly to resolve the issue. Independant researchers can contact CERT and they'll handle all of this legwork for you and make sure you get the credit. Of course the patching process still takes time for development, porting across platforms, and regression testing. So you do have to cut the vendors some slack.

      In the case of ISS there's almost no excuse for not getting some serious cooperation from the vendor. ISS has the weight and all the contacts they need to notify the vendors and get a fairly quick response. This was either an extreme circumstance, or Michael had another job lined up and he wanted to exit with a big splash. For that matter, he may have just made enough noise about his Blackhat presentation that he didn't want to have to pull it back.

      On an entertaining side note, Blackhat actually reburned all the CD's and cut his section out of the convention notes. Cisco must have come down pretty heavy for them to pull such a strong CYA move.

    4. Re:I wonder... by garcia · · Score: 3, Interesting

      c'mon... you're telling me that out of 5+ billion people on this planet, that only the person that found the exploit is the one that knows about it?

      We know, from the last time a story about this topic was posted, that Cisco was alerted to the issue and had supposedly "been working on a fix" during that time.

      So, no, we aren't that dumb -- what's dumb is that they believe that they can threaten people with lawsuits to keep them quiet.

      This is nothing but a corporate scare tactic to keep people from disclosing issues w/their shit in the future.

    5. Re:I wonder... by Calyth · · Score: 2, Interesting

      I can't remember whether I saw this from the Outer Limits or some other Sci-Fi series, but it was about a guy who discovered that cold-fusion bombs were feasible, and built one. Eventually he was killed, but at the end, some other person also stumble upon the same solution.
      I much rather have the security flaw be exposed, and they get to scrambled into a more heightened mode and fix the problem then let it be silent. He discovered the problem publicly, but that doesn't prevent other hackers from knowing the exact same thing.

    6. Re:I wonder... by n0-0p · · Score: 2, Interesting

      I'm not assuming that at all. I explained the process in more detail in my previous post (http://it.slashdot.org/comments.pl?sid=157252&cid =13184604 ) but I didn't want to repeat myself. I suppose I should have should have thrown the link in.

      The funniest thing though, is that this isn't even a true vulnerability in the strict sense. It demonstrates how to circumvent certain protection mechanisms to build a more reliable exploit for an existing vulnerability. What's more, Cisco was very obviously trying to address the concern, but resolving the issue was taking time. With that in mind, I'm not sure how you can even make the argument that full disclosure was necessary at this time.

    7. Re:I wonder... by Lost+Found · · Score: 2, Interesting

      Well, you're right. But I don't think the Mozilla project is a shining star in the security department.

      I rather like Daniel Bernstein's policies on his software... publish a verifiable exploit against my software and I'll give you $500.

    8. Re:I wonder... by abaddon314159 · · Score: 5, Interesting

      I am Michael Lynn...I'd like to clarify things

      Cisco was notified of the vulnerability in question many months ago and the issue has been patched for about 3 months now.

      Furthermore I did not disclose the details of this vulnerability at all. The presentation was merely a demonstration that IOS was exploitable just like any other OS.

    9. Re:I wonder... by nasor · · Score: 3, Interesting

      You often hear that, but I wonder if it's always a valid line of reasoning. Do you think it's more of a risk for a few malicious people to possibly know about an exploit while the company takes its time fixing the problem, or for the entire world to definitely know about it while the company scrambles to cobble together a quick fix?

      Some security flaws require such detailed technical understanding of the systems involved that not many people are really likely to uncover them. If a professional security researcher with very specialized knowledge who works full time trying to uncover new exploits succeeds in finding something, it doesn't n necessarily follow that many other people will, or even that anyone else will. It's certainly possible that someone else will find it, but I think people should try to balance the possibility of some malicious people knowing about the flaw for a long time against the certainty of everyone knowing about the flaw for a shorter time.

  2. Hmmm, perhaps he needs whistleblower protection? by meburke · · Score: 4, Interesting

    As dependent on as our economy is upon routers, and Cisco in particular, it seems that his disclosure was definitely in the public interest, and if he isn't entitled to whistleblower protection, we need to mount a campaign to get him protected. Write your Congressoid.

    --
    "The mind works quicker than you think!"
  3. Re:new flaws by megla · · Score: 5, Interesting

    The thing is (from what the articles say) it's not about one particular flaw. It's that ANY overflow flaw can be exploited to take control of Cisco IOS, which is bad news. Add Cisco's plan to abstract the hardware from IOS and then you've got a major problem. Basicly, it's about time Cisco implimented some form of DEP protection offered by current Intel and AMD processors + software, to prevent this from being an issue. Or check their bloody code of course.

  4. Why? by MyNameIsFred · · Score: 4, Interesting

    The articles cited are light on details. But nowhere do the articles suggest that Cisco was burying the flaw. In fact, the opposite is indicated. ISS and Cisco are apparently working on a fix. In my mind whistle blower protection is valid if the whistle blower is uncovering corruption. Which does not appear to be the case here. Based on the information presented, the system was working on the problem, he just wasn't happy with it.

    1. Re:Why? by Fenresulven · · Score: 2, Interesting

      In fact, the opposite is indicated. ISS and Cisco are apparently working on a fix.

      For four months... Come on, how long should he be required to wait?

    2. Re:Why? by HopeOS · · Score: 2, Interesting

      ISS and Cisco were co-presenters for the talk up until a week before the conference. The conference organizer, Jeff Moss, is quoted as saying that Cisco, not ISS, pulled out. Moreover, Cisco provided the people who removed the 15 pages of text from the conference proceedings.

      I can see no viable solution that includes Cisco paying ISS to locate and publicly disclose flaws in their software. When companies like Cisco hire third-party firms to audit their code for security flaws, the result of that work is universally subject to NDA.

      Second, Lynn is reported to have reverse-engingeered the code in order to discover the flaw. Why would Lynn need to do that if Cisco contracted the work to ISS? Would he not have access to the source code under NDA?

      Finally, Cisco stated that Lynn obtained the information "illegally." They did not claim that he disclosed the information in violation of an NDA. Had Cisco contracted this work to ISS, they would instead be suing ISS for breach of contract, and Lynn for breach of NDA.

      It would be very interesting to see the text for the temporary restraining order. What exactly did Cisco claim? At any rate, a TRO is trivially easy to get; in fact, it's nearly automatic. As for a permenant restraining order, that will be something to watch.

      -Hope

  5. Re:Cisco themselves said it was not a new flaw by Joehonkie · · Score: 3, Interesting

    Where does it at all apply that the one follows from the other? Presumably they are saying that he was involved in confidential research into the flaws and was not supposed to make any statement on his own. His simply quitting the company does not remove his obligations. He was not some outside agent who found out about this flaw independantly and cannot be expected to be treated as such.

  6. Lawsuit? Lynn says "bring it on" by kriegsman · · Score: 4, Interesting
    From today's Wall Street Journal:
    When Mr. Lynn took the stage yesterday, he was introduced as speaking on a different topic, eliciting boos. But those turned to cheers when he asked, "Who wants to hear about Cisco?" As he got started, Mr. Lynn said, "What I just did means I'm about to get sued by Cisco and ISS. Not to put too fine a point on it, but bring it on."
    Somehow, I suspect he's going to get what he asked for.

    -Mark
  7. Surely a decent way of resolving these issues by goldcd · · Score: 2, Interesting

    that would keep all parties happy, is a modification of the current craze for bug-bounties.
    Flaw is reported, accepted and cash is paid on a daily/weekly basis until the issue is resolved.
    Submitters would get more for a complex bug that involves more work to fix it and the can happily keep their gobs shut from announcing the problem as they're getting paid to be quiet.
    Just a thought..

  8. Dangerous Precedent... by gillbates · · Score: 4, Interesting

    "It is our belief that the information that Lynn presented at Black Hat this morning is information that was illegally obtained and violated our intellectual-property rights,"

    Lynn decompiled Cisco's software for his research and by doing so violated the company's rights, Noh said. [emphasis added]

    So basically, Cisco is claiming that decompiling their object code is illegal.

    Isn't it a greater violation of the customer's rights to prohibit them from decompiling the code on their own equipment to check for security vulnerabilities?

    We've come to the point where corporations believe they have the right to impose conditions of operation on equipment they no longer own. If Cisco sells someone a router, the customer now owns it. Cisco doesn't have any right to impose any conditions of use on the new owner, because they no longer legally own the product. The owner has the right (and some would claim even the responsibility) to decompile their router's code to check for potential vulnerabilities.

    It seems that Cisco believes that even after they've sold it to you, they still own your router. And who knows, maybe this vulnerability was deliberately placed so they could own your router anytime they pleased...

    --
    The society for a thought-free internet welcomes you.
  9. Whose rights were violated again? Hmm? by StandardCell · · Score: 3, Interesting

    The filing in US District Court for the Northern District of California asks the court to prevent Lynn and Black Hat from "further disclosing proprietary information belonging to Cisco and ISS," said John Noh, a Cisco spokesman. "It is our belief that the information that Lynn presented at Black Hat this morning is information that was illegally obtained and violated our intellectual-property rights," Noh added.

    Ok, let's look at this objectively, shall we? Proprietary information belonging to Cisco and ISS is nonsense. That information should belong to the customers who bought the router so they can take the appropriate steps; for example, a customer should be able to replace an affected router with something else if they're concerned about the problem, or modify the software on the router to alleviate the problem itself (and this is again another example of where OSS is so important).

    In terms of violating intellectual property rights, what about violating the property rights of the people who own the router? What rights do they have in this whole situation? Are they expected to sit their with their collective thumbs up their collective asses and wait randomly for a fix? Don't the people who use the routers have the right to uninterrupted network services? What happens if this router belongs to a large ISP and a DoS attack brings the router down? Are they supposed to be stuck with the bill? I'll tell you this much - if this happened, Cisco would never credit them with the cost of service refunds to their end customers. Of course, this would be hypocritical on Cisco's part for obvious reasons, but I digress.

  10. sued? by digidave · · Score: 2, Interesting

    How can he be sued if "the problem is not a security vulnerability"

    Way to go, Cisco.

    --
    The global economy is a great thing until you feel it locally.
  11. Re:Responsible Behavior? by justins · · Score: 2, Interesting
    I can't help but wonder, if this in the end really about gaining some publicity and in the end making more money.

    It's hard to imagine giving the finger to his employer in a very public manner was good for his long term employability.
    --
    Now before I get modded down, I be to remind whoever might read this that what I am saying is FACT. - bogaboga
  12. Re:Cisco has gone downhill recently by ciroknight · · Score: 3, Interesting

    Ridiculed? They built a backdoor into their product that was such a security flaw that it made IT professionals worldwide look at Cisco in awe. Who the hell would use a master password for a product that's going to be in the server rooms of a thousand businesses?

    I don't think "ridiculed" is the right word at all. They deserved the attention that was directed at them, as a master password is no small oversight. That'd be like Windows shipping with a master password.

    --
    "Victory means exit strategy, and it's important for the President to explain to us what the exit strategy is." G.W.Bush
  13. "Cisco credits you"-when they're not attacking you by toby · · Score: 5, Interesting
    See the unfortunate case of Fernando Gont, and his attempts to responsibly disclose ICMP implementation flaws (not even a Cisco-specific problem):
    Once Fernando understood the vulnerabilities he'd found in the ICMP protocol, he began to try and safely report the problem ... To begin, he wrote an internet draft which he submitted to the IETF in August of 2004. At that time he contacted CERT/CC and NISCC, and privately notified several open source projects ... as well as larger vendors such as Microsoft, Cisco, and Sun Microsystems. ...

    Around this same time, Fernando began receiving emails from Cisco who had numerous technical questions about his solutions to the problems. He continued to reply thoroughly to all their questions, until two months later when he received an email from Cisco's lawyer claiming that Cisco held a patent on his work. He asked their lawyer for specifics, but they refused to reveal any details. For two more months this continued, until Fernando was cc'd on an email thread between Cisco, Linus Torvalds, and David Miller. Reading back through the thread, Fernando found where David Miller had asked Cisco how they could possibly patent sequence tracking as Linux had been doing it for many years, and later in the same thread Cisco noted that they had withdrawn their patent. ...

    While the patent issue was happening with Cisco, CERT/CC created a mailing list to allow vendors to communicate amongst themselves about the newly discovered vulnerability. "They blamed me for submitting my work," Fernando said in exasperation. "One of Cisco's managers of PSIRT said I was cooperating with terrorists, because a terrorist could have gotten the information in the paper I wrote!" Fernando was familiar with intellectual property arguments with last year's Slipping In The Window paper, so he had intentionally publicly published his findings to prevent it from being patented. "Then they accused me of working with terrorists, and even still tried to patent my work!" He noted that he now suspected had he actually worked exclusively with Cisco as they had requested, they probably would have managed to patent all of his ideas. ...

    Fernando also found Microsoft difficult to work with. "Microsoft's acknowledgment policy says that you must report the issues to them 'confidentially'", he explained. As he chose to contact CERT and various open source projects as well, he claimed that they refused to give him credit for the discovery. Only with much effort did he finally get them to acknowledge that he had discovered the issue.

    --
    you had me at #!
  14. Re:Cisco has gone downhill recently by mysticgoat · · Score: 3, Interesting

    [re "master password thing"]That was from a while back. They had set up a master "backdoor" password in a version of IOS

    So since that didn't work, they put a backdoor into the hardware, then slapped a superficial patch on the first (of a number of possible exploits) that has come to public attention. And now they are persecuting the guy who has publicized the underlying flaw, which they have neither patched nor fixed.

    So I think it is time for these questions:

    1. When did Cisco first become aware of this hardware backdoor, and was it purposefully put into place?
    2. Who have they shared this knowledge with?
    3. Who has been listening in on which routers, for how long have they been doing it, and for what purpose?

    I guess I'd better get myself a new tinfoil hat. This one is worn out...

  15. Re:why did they.... by mysticgoat · · Score: 2, Interesting

    What changed at the last minute?

    Makes you kind of wonder who else has known about this vulnerability and told Cisco to dummy up about it.

    So again,

    1. When did Cisco first become aware of this hardware backdoor, and was it purposefully put into place?
    2. Who have they shared this knowledge with?
    3. Who has been listening in on which routers, for how long have they been doing it, and for what purpose?

    BTW, if anybody in a trenchcoat asks, I'm just going for "funny" here... and don't tell them that I'm opening a discount store for tinfoil hats, okay?

  16. Professional Obligation by randyflood · · Score: 4, Interesting

    Two words "Professional obligation".

    There used to be two general ways to handle security flaws when you discovered them. Either you could privately exploit the hell out of them. Or you could just privately report them to the company involved and wait patiently for them to release a fix.

    However there is a big problem with this particular model. The problem is that companies like Cisco, Microsoft, etc. don't really seem to think that exploits that allow people to remotely execute administrator level code are really that big of deal, and they figure that they can just create a patch when "we get around to it" or "next year".

    Meanwhile, do you really think that you are the only person in the entire world who is guaranteed to find the exploit? The black hats of the world have probably already found the exploit anyway in many cases. It's just the customers who are suffering because a patch is not available.

    This model of waiting around forever was a dismal failure. So, security professionals found that by publicly releasing their findings, they could force companies to take security more seriously. The responsible way to do this is to first inform the company privately of your finding, and give them a reasonable chance to fix it.

    What you think is reasonable is up to you, *not* them. They are playing by your rules. You are not playing by theirs. Remember, that you are being nice to them by not just publicly releasing the exploit the day that you found it. So, they should respect that. If they do not, that is their problem. Still, as a professional, you should rise above them and try to give them a reasonable time to fix the problem.

    Now in this case, what he did was he informed them 4 months ago of the vulnerability along with a proof of concept. They decided not to fix the problem. They claimed there was no problem. He waited patiently for *4 months*. They said that this wasn't really a vulnerability. Then, they knew well in advance of his presentation at Black Hat, and yet they still chose not to fix the problem.

    So, what is he supposed to do? As a security professional, it is his ethical obligation to publicly disclose his findings at that point.

    In conclusion, Cisco should spend more money on engineers instead of lawyers.

    --
    Randy.Flood@RHCE2B.COM
  17. Re:What idiots modded this thread informative? by Creep73 · · Score: 2, Interesting

    What idiots modded this thread informative?
    Probably the same idiots that modded yours "Insightful".

    The following is off the IIS webpage.

    About Internet Security Systems
    Internet Security Systems, Inc. (ISS) was founded in 1994 by Christopher W. Klaus and made its initial public offering on the NASDAQ on March 23, 1998.

    Profile The company provides security products and services that preemptively protect enterprise organizations against Internet threats.

    ISS celebrated its 10th anniversary in 2004 and has commanded the leading edge of security innovation, inventing cornerstone technologies such as vulnerability assessment and intrusion detection/prevention.

    The company continues to set standards in the security space with its Proventia Enterprise Security Platform (ESP), offering enterprise-wide preemptive protection that is tightly integrated with existing IT business processes.

    X-Force Research The foundation of ISS' preemptive approach to Internet security is its X-Force research and development team. ISS can stop more threats because it knows more: by discovering, researching and testing software vulnerabilities and collaborating with government agencies, industry consortiums and software developers.


    This is not a donation business. Companies and governments pay these people to provide products and services.

    In response to:

    Lynn did NOT work for Cisco, nor does ISS work "for / with" them.

    I want you to read the following line very carefully ok!

    The injunctions filed against him state that ISS and Cisco had been working together on the flaw for the past four months, and that up until earlier this week, a Cisco executive was slated to co-present the findings with Lynn at Black Hat.

    This came from the washington post

    Here is another one just in case you didn't like that one

    We appreciate the cooperation we have received from ISS in this matter. We are working with ISS to continue our joint research in the area of security vulnerabilities."

    Wow, joint research.

    The court injunctions stated that they had worked with each other for months on this specific issue. Cisco states that they were doing joint research on security vulnerabilities. I can't believe people are making this big of a deal over this one point. The two companies worked with each other. I do not know if Cisco was a client of IIS but they at least worked with each other. It is hard for me to believe that IIS volunteered their time working with Cisco. I am sure a little money changed hands but that doesn't matter.

    I can't prove that someone has not used this exploit; however I can indicate that no case has been found. Nothing has been reported. With that in mind what are the odds?

    Let's look at a few things. While the exploit was a secret the only people who were likely to identify the exploit were people who could reverse engineer the Cisco OS like Lynn supposedly did. Not many people are able to do that. Fewer yet want to.
    Even if several people did go through that process there is no guarantee that they would identify the exploit and then we have to assume that those individuals that did make such a discovery would act maliciously. What is the likely hood that a problem will crop up under those circumstances?

    Next we have Lynn (Your Buddy) making a public display of how to exploit the Cisco OS. Now what is the likelihood that a problem will crop up? Did the chances that the exploit would be used go up or down genius?

    Did Lynn serve the public interest by going public against the wishes of Cisco and IIS? I think not. You are free to disagree. You are even free to be pricks about it.