Slashdot Mirror


23,000 HTTPS Certs Axed After CEO Emails Private Keys (arstechnica.com)

An anonymous reader quotes Ars Technica: A major dust-up on an Internet discussion forum is touching off troubling questions about the security of some browser-trusted HTTPS certificates when it revealed the CEO of a certificate reseller emailed a partner the sensitive private keys for 23,000 TLS certificates. The email was sent on Tuesday by the CEO of Trustico, a UK-based reseller of TLS certificates issued by the browser-trusted certificate authorities Comodo and, until recently, Symantec...

In communications earlier this month, Trustico notified DigiCert that 50,000 Symantec-issued certificates Trustico had resold should be mass revoked because of security concerns. When Jeremy Rowley, an executive vice president at DigiCert, asked for proof the certificates were compromised, the Trustico CEO emailed the private keys of 23,000 certificates, according to an account posted to a Mozilla security policy forum. The report produced a collective gasp among many security practitioners who said it demonstrated a shockingly cavalier treatment of the digital certificates that form one of the most basic foundations of website security... In a statement, Trustico officials said the keys were recovered from "cold storage," a term that typically refers to offline storage systems. "Trustico allows customers to generate a Certificate Signing Request and Private Key during the ordering process," the statement read. "These Private Keys are stored in cold storage, for the purpose of revocation."

"There's no indication the email was encrypted," reports Ars Technica, and the next day DigiCert sent emails to Trustico's 23,000+ customers warning that their certificates were being revoked, according to Bleeping Computer.

In a related development, Thursday Trustico's web site went offline, "shortly after a website security expert disclosed a critical vulnerability on Twitter that appeared to make it possible for outsiders to run malicious code on Trustico servers."

36 of 72 comments (clear)

  1. Bullet, Meet Foot by mentil · · Score: 1

    When Jeremy Rowley, an executive vice president at DigiCert, asked for proof the certificates were compromised, the Trustico CEO emailed the private keys of 23,000 certificates

    Those certificates are DEFINITELY compromised now.

    --
    Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
    1. Re:Bullet, Meet Foot by TechyImmigrant · · Score: 2

      When Jeremy Rowley, an executive vice president at DigiCert, asked for proof the certificates were compromised, the Trustico CEO emailed the private keys of 23,000 certificates

      Those certificates are DEFINITELY compromised now.

      TFA seems to imply that he emailed the private keys in order to prove that they were compromised. Which seems like an appropriate thing to do.

      --
      I should use this sig to advertise my book ISBN-13 : 978-1501515132.
    2. Re:Bullet, Meet Foot by mysidia · · Score: 1

      ... Which seems like an appropriate thing to do.

      No... it is NOT appropriate for a CA or a reseller of a CA to retain customers' private keys in the first place --- it is even MORESO inappropriate for a CA to deliberately extract and use in any manner for any purpose a customer's private key from "secure cold storage" without that customer's specific authorization.

      Basically this changed the situation from "There are security concerns related to this certificates", to --- This CA reseller deliberately compromised the security of their customer's certificates by appropriating their private keys during the ordering process and then later transmitting them insecurely over the internet.

    3. Re: Bullet, Meet Foot by dimko · · Score: 1

      because... managed SSL services. For people with more money than sense. In a nutshell, you pay them for that service and installation of SSL certs is now their business. Potentially maintenance too.

    4. Re:Bullet, Meet Foot by Nkwe · · Score: 4, Insightful

      When Jeremy Rowley, an executive vice president at DigiCert, asked for proof the certificates were compromised, the Trustico CEO emailed the private keys of 23,000 certificates

      Those certificates are DEFINITELY compromised now.

      The first to shoot themselves in the foot would be anyone who doesn't generate their own private key when they purchase a certificate. The CA is only supposed to sign the public parts of your certificate, it is not supposed to ever have access to the private key. Letting your certificate vendor create a private key (and subsequently have access to it) is unwise and insecure.

    5. Re:Bullet, Meet Foot by Solandri · · Score: 1

      No... it is NOT appropriate for a CA or a reseller of a CA to retain customers' private keys in the first place

      I'm guessing that was the compromise Trustico was reporting. The private keys should've been deleted immediately after they were generated for the customer. But Trustico probably found during an audit that they hadn't been deleted and were still on their servers somewhere. Since they couldn't prove that those private keys hadn't been copied, they erred on the side of caution and declared them all compromised.

    6. Re:Bullet, Meet Foot by gweihir · · Score: 3, Insightful

      The level of stupidity expressed in this is staggering. I mean it is not only the fact that somebody with the least bit of clue would never email secret keys without protection, it also is that he could get them in the first place and do this. This means that DigiCert is completely compromised itself due to non-existing or easily bypassed security policies and should under no circumstances be trusted again.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    7. Re:Bullet, Meet Foot by thegarbz · · Score: 1

      Those certificates are DEFINITELY compromised now.

      Wrong analogy. Bullet meets foot implies that the action of the CEO achieved something other than what he was hoping. He wanted to revoke those certificates anyway, and when questioned whether they were compromised he compromised them.

      Nail meets coffin.

    8. Re:Bullet, Meet Foot by Shuntros · · Score: 2

      You do not need a private key to revoke a certificate. You need the certificate serial number.

      The issuer should NEVER set eyes on a private key which isn't theirs. If you want to make life easier for your customer, do it client-side with JavaScript and throw a PFX at them once the issuance has completed.

      It still stuns me how many in tech, even CAs, have such a poor understanding of how PKI works.

    9. Re:Bullet, Meet Foot by mysidia · · Score: 1

      You do not need a private key to revoke a certificate. You need the certificate serial number.

      As a reseller, they wouldn't have any legal right to revoke the certificate without customer permission -- that should be an act of the CA.

      Notice how in the article that DigiCert Demanded Proof of the compromise?

      Hopefully now that this has happened: DigiCert will be more reluctant to permit this reseller in the future to have their own custom ordering process --- I would say they should suspend this reseller until they agree to a contract addendum that they will neither generate nor obtain or store private keys for customer certificates.

    10. Re:Bullet, Meet Foot by phantomfive · · Score: 1

      It makes you wonder how the CEO got the keys to begin with.

      --
      "First they came for the slanderers and i said nothing."
    11. Re:Bullet, Meet Foot by Megane · · Score: 1

      DigiCert didn't mail the private keys, Trustco's PHB did.

      --
      #naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
    12. Re:Bullet, Meet Foot by gweihir · · Score: 1

      Ah, sorry. Thanks for the correction. Then my statement applies to Trustco.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    13. Re:Bullet, Meet Foot by TechyImmigrant · · Score: 1

      >No... it is NOT appropriate for a CA or a reseller of a CA to retain customers' private keys in the first place

      Well that's not what I said, is it?

      Emailing the keys to someone is an appropriate way to prove to them that the keys have been compromised. Now go and be contrarian somewhere else.

      --
      I should use this sig to advertise my book ISBN-13 : 978-1501515132.
    14. Re:Bullet, Meet Foot by modmans2ndcoming · · Score: 1

      ever here of secure file shares?

    15. Re:Bullet, Meet Foot by FormOfActionBanana · · Score: 1

      I don't get the fuss. If they are compromised, they are compromised.

      --
      Take off every 'sig' !!
    16. Re: Bullet, Meet Foot by TheRaven64 · · Score: 1

      This is only true if the domain uses DNSSEC. Most laptops / phones trust whatever DNS server the access point that they're connected to tells them to use via DHCP. If you set up a malicious AP then you can return your own server address for the compromised domains. If you deploy malware that infects a few thousand vulnerable WiFi routers then you can do this on a large scale quite easily.

      --
      I am TheRaven on Soylent News
    17. Re:Bullet, Meet Foot by TechyImmigrant · · Score: 1

      Emailing the keys to someone is an appropriate way to prove to them that the keys have been compromised.

      The keys are compromised for sure so its not inappropriate, but a better method might have been to sign something with the keys as proof instead. Though either way the keys are just as bad off. Now why did digicert only suspend 23k keys instead of the full 50k as was initially reported? digicert is in essence saying they must send all 50k keys before they'll suspend them.

      My crystal ball isn't working today. It seems like a really bad idea to be a CA and have people's private keys on file. It seems like a bad idea to be taking people's word for it when you're in a position of issuing revocations. Not my CA. The real CA I set up (in a former job role) held onto nothing but its own keys and the cert list. Certs are needlessly complicated, but you can and should make the process as simple as possible.
       

      --
      I should use this sig to advertise my book ISBN-13 : 978-1501515132.
    18. Re:Bullet, Meet Foot by mysidia · · Score: 1

      The keys are compromised for sure so its not inappropriate, but a better method might have been to sign something with the keys as proof instead.

      What they showed wasn't proof of the compromise the reseller had been claiming happened. What they showed was proof that the reseller had (IMPROPERLY and DELIBERATELY) retained customer private keys, WHICH IS INAPPROPRIATE --- and the reseller then literally compromised their keys (that might not have been compromised before) to "prove" it, which was ALSO highly inappropriate.

      why did digicert only suspend 23k keys instead of the full 50k as was initially reported?

      None of the keys are KNOWN compromised according to the reseller, Only suspected that they could have been, and ultimately it's the customer's decision how to proceed if there's merely a suspicion or belief ---- PROOF would have been obtaining compromised Keys through the flaw, not going to deliberately retained cold storage files and retrieving customer keys.

  2. That's Nothing by NicknameUnavailable · · Score: 1

    Sophos has a trusted root CA embedded in their enterprise firewalls which allows the firewall to launch man-in-the-middle attacks against clients to spy on them. That means all you have to do to launch a successful man-in-the-middle attack yourself against HTTPS traffic is to gut a Sophos firewall and find the private key embedded in it.

    1. Re:That's Nothing by Pinky's+Brain · · Score: 1

      They use the same key pair for every firewall?

    2. Re:That's Nothing by Nkwe · · Score: 1

      Sophos has a trusted root CA embedded in their enterprise firewalls which allows the firewall to launch man-in-the-middle attacks against clients to spy on them. That means all you have to do to launch a successful man-in-the-middle attack yourself against HTTPS traffic is to gut a Sophos firewall and find the private key embedded in it.

      You would have to install the root cert certificate from the firewall CA into all your clients for that to work. In an enterprise if you want to sniff HTTPS traffic, you may chose to do this (since in an enterprise you control the client machines), but as soon as you chose to do this, you open up huge security holes.

    3. Re:That's Nothing by NicknameUnavailable · · Score: 1

      They use a keypair that's embedded into all the major browsers as a trusted root CA - so chances are yeah.

    4. Re:That's Nothing by NicknameUnavailable · · Score: 1

      Nope, it's included in all the major browsers (I only noticed it after installing a privacy-geared browser far from the mainstream which didn't include the bundled root CAs for Sophos.)

    5. Re:That's Nothing by BlueUnderwear · · Score: 1

      Does anybody know which root CA this is? So that I can mark it "untrusted" locally in my Firefox.

      --
      Say no to software patents.
    6. Re: That's Nothing by orbit500 · · Score: 1

      Thatâ(TM)s really not true though, is it? There is an appliance CA keyed for the machine and which generates an appliance cert. The CA is NOT in the default trust store of any browser. It has to be explicitly downloaded from the appliance and added to trust. No idea where you got that nonsense from!

    7. Re: That's Nothing by NicknameUnavailable · · Score: 1

      Firefox, Chrome, Opera, IE, and Safari didn't complain about it - I switched to a more secure version of Firefox with locked down privacy features when I noticed it, turns out they don't include the certs allowing for corporations to override websites to monitor traffic going over their network.

    8. Re:That's Nothing by modmans2ndcoming · · Score: 1

      it was a re-seller that did this, not the root. DigiCert is the root and they asked for proof from the re-seller that the keys were compromised before they revoked them. The re-seller CEO sent the private keys to DigiCert via standard email with no protection.

  3. Idiots. by Narcocide · · Score: 1

    What, were they just loose on his desktop next to the vacation photos?

  4. Oh Well, by rotorbudd · · Score: 1

    Dumbasses gotta dumbass.

    --
    A bullet may have your name on it, but artillery is addressed to " Whom It May concern"
  5. Re: Is SSL just "Security Theater"? by jecowa · · Score: 1

    I think SSL is good technology, but the certificate authorities are a scam. I believe you can create your own certificates for free and without some third-party having access to the key and make money from selling it to you.

    --
    my opportunity to freely express myself with the potential persecution and hangings and such
  6. Either encrypt your email or stop using it by Murdoch5 · · Score: 1

    If you're using email in 2018 and it's not encrypted it, stop using it, simple! The average person and especially executives, have no sense of security or secure operations. I can point to numerous companies, where even the CTO's and CSO's are widely unqualified to hold those positions, and they would and have, send non-encrypted email containing very sensitive information.

    If society isn't going to grow up and start encrypting all email communication, then it's time to get rid of email.

  7. Never give your CEO... by Tony+Isaac · · Score: 2

    Many CEOs are just technical enough to be dangerous. Never give your CEO:
    - Direct access to your database server
    - Administrator passwords of any kind, even to their own laptop
    - Access to server rooms
    - PRIVATE KEYS!

    You CAN give a CEO a MacBook Air. They'll be happy with the sleek design, and they won't be able to do much damage, since not a lot of "work" software actually runs on it.

  8. why do executives have access to this type of data by modmans2ndcoming · · Score: 1

    This is why Executives are not kings. There are parts of the business that they should not have casual access to. That is not to say they do not have a right to review and inspect with appropriate parties involved in the process, but access to data and tools like this is not the same thing as keys to the front door.

  9. Re:why do executives have access to this type of d by Teun · · Score: 1

    Although his action to mail the certs was not smart, the initial problem was not the executive.
    The Initial problem was his company had kept copies of the private keys, an absolute no-no and when he(?) found out he wanted to communicate which keys were to be revoked.

    --
    "The likes of Facebook and WhatsApp are free to those whose privacy is of zero value."
  10. Re:I might be dumb... by TheRaven64 · · Score: 2

    But why the fuck isn't the PUBLIC key signed, and the end user sends a message to the private key to verify it is authentic?

    It is. You generate a certificate signing request (CSR) from your certificate (which embeds the public key and the metadata fields such as organisation name, host name and so on). You send the CSR to your certificate authority (CA). The CA then gives you back a signed certificate (which may strip out some fields from the cert that the CA doesn't want to attest to). The key exchange phase of TLS then sends the certificate to the client, which can walk the certificate chain to verify that someone (hopefully someone trustworthy) is willing to attest that the public key belongs to the organisation that you think you are communicating with. The client then encrypts using the public key and the server decrypts using the private key.

    I will personally just stick to self signed certificates for exactly this private key threat model

    You use self-signed certs for a threat model that doesn't exist? This problem existed only because they had customers that decided to outsource certificate creation to them, which is a bad idea and would have failed a security audit.

    but obviously the entire chain of trust is untrustworthy in this day and age

    If you get a CA to sign your certificate then, at worst, it is no less secure than if you don't. You are still free to distribute the hash out of band and check it. You are still able to use certificate pinning to ensure that you notice if it has unexpectedly changed. And if you don't sign it, then a malicious CA (e.g. one compromised by an intelligence agency) is still able to sign a cert claiming to be yours and have other people trust it. If you use DNSSEC and publish CAA records then you can at least narrow this down to one CA that they must compromise.

    --
    I am TheRaven on Soylent News