Slashdot Mirror


How Cisco Fixed An Undocumented SSH Support Tunnel In Umbrella (umbrella.com)

"Vulnerability due to always-on SSH Tunnel -- RESOLVED" reads a Cisco service update. An anonymous reader writes: Described by a recent security blog post, Cisco hid a SSH backdoor in its Cisco Umbrella product, which they were using for support. Affected organizations can install version 2.1.0 of their virtual appliance which has the backdoor removed.
Cisco has described Umbrella as "the first Secure Internet Gateway in the cloud," though the now-closed tunnel "auto-initiated from the customer's appliance to Cisco's SSH Hubs in the Umbrella datacenters." Cisco adds that it "did not require explicit customer approval before establishment." Access to the terminating server required valid keys and was provided only to privileged support personnel within the Cisco Umbrella network space. Customers could prevent this tunnel from getting established by blocking the relevant firewall ports. However, in the case of customers who allowed establishment of the tunnel, an attacker who obtained access to the internal Cisco terminating server could use the SSH tunnel as a backdoor to obtain full control of the VA device at the customer's premises...

It is our policy that any undocumented methods of entry into your network devices be considered a vulnerability due to the potential risk of an attacker leveraging this tunnel to gain access to your network. While Cisco has NO indications that our remote support SSH hubs have ever been compromised, Cisco has made significant changes to the behavior of the remote support tunnel capability to further secure the feature...

To address this vulnerability, the Umbrella Virtual Appliance version 2.1.0 now requires explicit customer approval before an SSH tunnel from the VA to the Cisco terminating server can be established... . For additional security, customer is required to provide tunnel configuration parameters out-of-band to the Cisco support personnel before tunnel establishment.

24 comments

  1. don't worry by Anonymous Coward · · Score: 0

    Don't worry, they will leave the undocumented NSA backdoor open!

  2. How did they do it?! by Daemonik · · Score: 2

    Presumably the programmers who work there wrote a patch and applied it?

    1. Re:How did they do it?! by Anonymous Coward · · Score: 0

      The proper question is "why would anyone buy a Cisco "secure" appliance after this?" And the answer is: anyone with half a brain wouldn't!

    2. Re:How did they do it?! by Anonymous Coward · · Score: 0

      If they have half a brain isn't the chance 50/50?

    3. Re:How did they do it?! by Anonymous Coward · · Score: 0

      The proper question is "why would anyone buy a Cisco "secure" appliance after this?" And the answer is: anyone with half a brain wouldn't!

      No; the real question is a) did Cisco really find and fix this on their own and b) what are they doing to improve their processes in future? There are lots of vendors with backdoors. Most of them, if they ever do fix the backdoor, do the fix because they are forced to. I bet that some customer complained to Cisco about this once they saw it and demanded a fix, in which case it's not too impressive, however if Cisco's internal people were able to force the fix then that's actually a good sign.

    4. Re:How did they do it?! by Anonymous Coward · · Score: 1

      I have a CISCO router from my ISP and I've really really really wanted to be rid of it for sooooo long. I didn't get the choice, and repeated complaints have not replaced it. When you see the duplicate packets with Wireshark you know there's a problem but you're never really sure if its intentional or not. Only that this company has a history of appearing in Snowden documents.

      All these other companies made themselves into a stasi in waiting by putting in these backdoors. A FISA warrant from agent orange, or rogue employee and all their customers data is at risk. Vote results? At risk. Election rolls? At risk. Emails, at risk, corporate IP? At risk. Basic security of all infrastructure.... at risk.

      Remember HP? And its StoreOnce RAID that has an extra remote account that can add users? Cisco are not the only ones with backdoors, it has been a big problem with US corporations for a while now.
      http://www.theregister.co.uk/2013/07/11/hp_prepping_fix_for_latest_storage_vuln/

      Remember Blackberry?

    5. Re:How did they do it?! by Anonymous Coward · · Score: 0

      Except NSA kept all these backdoors, and got hacked by Russia, so all of those backdoors are now in the hands of a hostile nation. A few have been released, but how many zero days ones were kept back.

      You can pretend USA good, Russia bad, but with a puppet in the Whitehouse, there is no difference at this point. Trump decides implementation policy, Trump declassifies secrets freely to Russia, Russia is a hostile, ergo Cisco is a hostile.

    6. Re:How did they do it?! by Anonymous Coward · · Score: 0

      Stopped using Cisco kit after the ASA "security" appliances were compromised and Cisco's position was that we should just replace them.

    7. Re: How did they do it?! by Anonymous Coward · · Score: 0

      It's not a good thing. I used to be able to static route that ip in question to my own box and have easy access without VPN shenanigans.
      I mean, a uh, friend used to.

    8. Re:How did they do it?! by KiloByte · · Score: 3

      The "fix" is not what you think. The backdoor is still there, it's just hidden better so some random punk can't exploit it.

      If you claim there's really no backdoor, go ahead, prove it! As for Cisco's words, we had them the last time too.

      --
      The creatures outside looked from Alt-Right to Antifa; but already it was impossible to say which was which.
    9. Re:How did they do it?! by JohnFen · · Score: 1

      The proper question is "why would anyone buy a Cisco "secure" appliance after this?"

      For the same reason that they continued to use Cisco products after the last time a backdoor was found in them?

    10. Re:How did they do it?! by Anonymous Coward · · Score: 0

      I wonder why these backdoors are there in the first place. However, with the fact that vendors face little more than a day of bad press, it isn't surprising that this is done, especially because if an unscrupulous offshore dev does get a backdoor into firmware in a widely used device, they can turn around and sell the hole on the black market for a princely sum.

      Problem is that this is getting countries to start looking at other providers than Cisco. People are tired of vulnerabilities in US and Chinese made devices, and if some device maker from a country that "didn't have a dog in the hunt" stepped up and made verifiable, secure devices, the world would beat a path to their door.

    11. Re:How did they do it?! by Archon · · Score: 1

      Aside from the issue of not being able to prove a negative, no integrated vendor is going to offer their source code for peer review. Either you use OSS router & switch software or you trust the manufacturer. Cisco, Juniper, Fortinet, etc have all had widely-reported breaches (in Cisco's case, several).
       

  3. Life Imitates Art by Anonymous Coward · · Score: 1

    In the 'Resident Evil' movies, the big evil corporation that was responsible for destroying the world was called The Umbrella Corp.

    Coincidence?

    I think not.

  4. "Oh shit, better spin an excuse" by Anonymous Coward · · Score: 0

    before it's discovered and we're rightly accused of still inserting backdoors into our products. Do you care about the security of your networks? Then don't use American hardware and services.

  5. security is about trust by Anonymous Coward · · Score: 0

    So why do people keep trusting cisco for their security? Ive sat in some of their demo presentation and while their communication applications are fancy, they still have a track record of abusing their customers trust that just screams insecure.

  6. why a shock?? by Anonymous Coward · · Score: 0

    fuck tones of people do it
    I bet 70% of all support teams use dubious methods like this all the time.
    but hey Has Cisco fixed the exploitable hardware issues with the c3000 series chips, which was reported here some time ago

  7. "Vulnerability" by Anonymous Coward · · Score: 0

    "It is our policy that any undocumented methods of entry into your network devices be considered a vulnerability due to the potential risk of an attacker leveraging this tunnel to gain access to your network."

    No, not a vulnerability. A failure of your product. Any further inclusion of it is a failure of your management.

  8. Did they... by Anonymous Coward · · Score: 0

    ... build a wall?

  9. Lack of trust by Anonymous Coward · · Score: 0

    I think most of the world has stopped trusting Cisco, and America's dispicable nature for many months now.

    As an outsider, it's still sad to see a once great country decend to the level of other despotic states like Nork, Russia, and Iran - but that's all America is now.

  10. Proprietary insecurity remains as-is. by jbn-o · · Score: 1

    How did they do it? They took advantage of someone's desire for convenience over software freedom (and the practical security benefits one gains from software freedom) to sell them products and services with at least one backdoor.

    Naturally, nobody should trust anything from Cisco regardless of how much they paid for it unless that program is free software (free to run, inspect, share, and modify) because that means one's software freedom is respected. But Cisco isn't the only problem organization here, all software proprietors are just as untrustworthy. I know I've posted a lot about this, but blame /. for pointing readers to the same theme of story again and again: Proprietary software (such as Cisco's Umbrella product) simply can't be trusted. Neither can anything configured in them or output from proprietary software. Logs, filters, anti-malware code, etc. are all untrusted once you know they come from a non-free program precisely because you don't know what that program is doing while it ostensibly does what it's advertised to do. When you don't know how much information is left out, and you are not permitted to inspect or modify the program to do what you want it to do, you are at the mercy of those who know its true power (the proprietor) just as Cisco's clients were subject to Cisco's use of this backdoor (and anyone else Cisco decided to let in on the backdoor).

    This untrustworthiness extends to the announced "resolution" ("Vulnerability due to always-on SSH Tunnel -- RESOLVED") as well. For all we know a newer more stealthy vulnerability was introduced to replace the old one. Thus the functionality this tunnel provided isn't really gone, it's just changed. And the untrustworthiness remains.

  11. Am I reading this right? Seems nigh impossible by sabbede · · Score: 1

    to exploit the potential exploit. If the device is a tunnel initiator only (can't be connected to), then taking advantage of it requires hacking into Cisco and taking over one of their terminating servers. If you can get that far into Cisco, why stop at waiting for Umbrellas to connect?