Slashdot Mirror
Verisign Certificate Expiration Causes Multiple Problems
We had to do a little sleuthing today. Many readers wrote in with problems that turned out to be related. A certificate which Verisign used for signing SSL certificates has expired. When applications which depend on that certificate try to make an SSL connection, they fail and try to access crl.verisign.com, the certificate revocation list server. This has effectively DOS'ed that site, and Verisign has now updated the DNS record for that address to include several non-routable addresses, reducing the load on their servers. Some applications affected include older Internet Explorer browsers, Java, and Norton Antivirus (which may manifest itself as Microsoft Word being very slow to start). Hope this helps a few people, and if you have other apps with problems, please post about them below.
(which may manifest itself as Microsoft Word being very slow to start)
But.. I thought this SSL certificate expired just today..
Trolling is a art,
In an effort to have us forget about SiteFinder, they're going for an even bigger fuck-up.
Nice try, guys... now turn the CRL server back on.
Well, it's good to know that not only crackers or script kiddies are good at taking down Verisign's services, that their own staff is good at it too.
Do not look into laser with remaining eye.
Heh.
The Army reading list
Self-signed certificates are fine for Joe-Hobby website, but when you're about to enter a credit card number online it's assuring to see that the SSL cert is signed by a real organization and not "l33t_d00d@hotmail.com"
Trolling is a art,
saying that your certificate is expired or not yet valid...except that it is...you need to go here.
and if you have other apps with problems, please post about them below.
I can't get the DOS version of Duke Nukem to run in Windows XP. Is this at all somehow related? Is there a fix??
On one of our customers' systems (IIS). Turns out they had already installed the new Verisign intermediate certificate but had not removed the old one. IIS happily used the old one...
Lesson: if the certificate expired yesterday, remove it from IIS and then reboot the thing.
Ceci n'est pas une signature
Unfortunately, unless you buy a cert from one of the officially blessed cert authorities, your users get this ugly-looking "security warning" popup from their browser. While this is fine for clued individuals, or internal sites and so on, things that are public-facing are more sensitive to that sort of thing.
It galls me every time I have to give someone on the officially "blessed CA" list money to do something I can do for myself in less time, but I don't know of an alternative that allows the public users of a secure website to not get alarming messages on their browser when they try to give us money.
unless your an average user who doesn't read certificates anyway, and will just click yes on pretty much everything
this sig is deprecated
In other news, Microsoft, Red Hat, Oracle, Sun, and Apple had to do a little coding today.
Rumors abound that Arnold Schwarzenegger had to do a little governing today, but these allegations remain unconfirmed at this time. More at eleven.
Obliteracy: Words with explosions
I find it particularly disturbing that their solution to too much traffic to their CRL server is to use non-routable addresses in DNS. As a result of this action, they have reduced the integrity of their certificates (yes, that means diluting TRUST, which is the foundation of PKI) by making the revocation lists unavailable. Without CRL checking, Verisign certificates have no inherit integrity advantage over self-signed certificates. This is what we pay for?
Non-authoritative answer:
Name: crl.verisign.net
Addresses: 10.0.0.1, 10.0.0.2, 10.0.0.3, 64.94.110.11
198.49.161.200, 198.49.161.205, 198.49.161.206
Aliases: crl.verisign.com
Go figure.
I noticed the problem last night while paying my credit card bill online. Got a warning from IE that the site's certificate had expired. I was a little confused because the date for my CC company's cert was indeed valid. I thought it was just IE being stupid, but it makes sense now.
"I turn away with fright and horror from the lamentable evil of functions which do not have derivatives."
sPh
route the traffic to some "SiteFinder service"?
Error: Id10t detected
It is stupid for VeriSign not to have taken the steps necessary to keep their CRL available under these conditions seeing that they get paid a lot of money to do only 2 things:
1) Be trustworthy
2) Be competent
> ...when you're about to enter a credit card number
> online it's assuring to see that the SSL cert is
> signed by a real organization...
Unfortunately, we usually have to settle for Verisign instead.
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
Well, not the Oracle database directly... But Oracle sent out a memo that certain Oracle products (Oracle Wallet Manager, in particular) would simply cease to function properly until the user upgraded their Verisign certificate(s).
I can't find ANY info on Oracle's website about this, though. The memo was sent to Oracle Premium Support customers but I don't know if the info has been generally distributed.
Woops!
The most unfortunate thing about this. Is that with VeriSign especially, I find them to be one of the _most_ untrustworthy companies on the planet (How many times have they mis-issued certificates now? And lets not forget all the screwups related to their DNS scams). So the question is, who do you go to for certificates? Can't sign your own because users may feel you're insecure (justifiable or not) and can't trust certificates from the "official" CA's, because... well that's like trusting the goverment to make sure you get all your tax deductions whether you knew they were owed you or not ;)
I just really wish I could find an affordable CA that I felt was trustworthy enough themselves as to feel safe making my customers trust their certificates.
I received the following email yesterday: Oracle Corporation has been notified by Sun that the set of VeriSign Class 2 and Class 3 Certificates used in Oracle products will be expiring on January 7, 2004. Please review MetaLink Doc 260332.1: Expiration of VeriSign Class 2/Class 3 Certificates on Jan 7,2004 for detail information.
...if you have other apps with problems, please post about them below.
Well, now that you mention it, my mother hasn't been able to print for a week, my uncle's PC keeps running checkdisk on startup, and I'm having trouble compiling kernel 2.6.0.
Oh yeah, and Unreal 2k3 has crappy frame rates on the 'Antalus' level, but maybe thats just my old ti4200 card.
Um. I think that's it for now. So when are you going to help me with these?
I'd rather be a conservative nutjob than a liberal with no nuts and no job.
I would have to say more users click on "yes" for everything. I have to reinstall several family members computer because of spy/ad ware and a ton of other crap because the click yes to everything.
I didn't use the preview button, so get over it!!!!
Mike
Unless you have a P75, I don't see what you are talking about. MSWord has always started in less that 3 seconds on my system (PIII 700) and I can tell you that sometimes it is terribly bloated (My system, not Word).
Wait, did I just admit running Windows on slashdot? Bye bye Karma.
Write boring code, not shiny code!
I use Instant SSL cheap, good service and I haven't seen any compatibility issues.
Thawte - cheaper than Verisign, much easier to work with them, and will work fine in any 4.0+ browser.
Because of the crl problems, Explorer has been acting slowly doing some seemingly unrelated activities. Copying or right-clicking on folders often is followed by a several second hang. To workaround, deselect "Check for publisher's certificate revocation" under the Advanced setting for IE (even though it is not IE running, that's where the setting should be changed). After this, no more Explorer hangs. Hope this helps someone. If you know why Explorer is checking crls for anything when doing a copy operation on files, please post.
There's software out there so anyone can sign a certificate. Who needs the suits at Verisign?
Because a cert signed by you is useful for nothing more than "This conversation is encrypted, and I say I'm me." A cert signed by a Verisign translates to "This conversation is encrypted, and Verisign says I'm me."
What good is that? Well, not much among geeks, we don't trust Verisign further than we can throw them, but we're depending on them to keep this silly DNS thing going. However, web browsers are set with a default list of trusted "Certificate Authorites" who are allowed to sign certificates. Companies who are on those lists can sign a certificate that'll work without errors, anybody else's certificate will prompt a message indicating that the name's right, the time's valid, but the issuing authority isn't on the list of authorities you trust. (You can manually add a new authority if you want... but try convincing users to do that!)
The problem is, so many cheapskates have now signed their own certificate that the bogus authority error isn't stopping users since it's so common when nothing's really wrong. As a result, we're seeing a lot of look alike sites use SSL to get the padlock to come up, and users not being phased by the red-flag alerts that this doesn't seem to be the site they think it is.
To get Word and Excel to start working again:
Open Nortons Control Panel - this might take
a few minutes while it is broken but it
will come up eventually. Under the Miscellaneous
Section of Anti Virus, deselect the Enable Office
Plug-in.
That will not fix any general slowness in Norton,
but it will allow you to read your Word/Excel
documents.
I hear that to get it to work with XP you need to upgrade to Duke Nukem Forever.
*ducks*
I think it beats another new "helpful" feature like "CRL Finder."
Uh, Thawte is owned by Verisign, smart guy...
But they are a lot cheaper for some reason... Go figure...
Unroutable addresses? Anyone on private corporate networks which are large enough to use 10.0.0.0/8, who are unfortunate enough to have been allocated the IP addresses 10.0.0.{1,2,3}, may be experiencing a little more network load than usual today as every machine in the place tries to query them.
GROGGS: alive and well and living in
Very nice of them to.. I don't know.. let someone know before today. We spent a ton of staff time this morning trying to figure out why we could connect to our servers but not the payment engines via ssl. 4 hours later we figured it out.
= fs alert%2F57436n dors/exp-gsid-s sl.html
Couple of nice links.
http://sunsolve.sun.com/pub-cgi/retrieve.pl?doc
http://www.verisign.com/support/ve
Just three more hours seapeople and you can finally take me away from this crappy God Damned planet full of hippies
Depending on how you have your server configured, it either means you are accepting revoked certificates, or are UNABLE to accept ANY certificates.
The default for some web servers is that if the CRL is unavailable, it will reject ALL presented certs.
Really the problem isn't just hte message. It's the Chain Of Trust. It works as follows: Verisign only (in theory! hah!) issues certificates signed by their CA to organizations that can fax in appropiate identificaton. A browser "trusts" VeriSign to make proper decisions. A browser can be extended to trust other CA's, the real world problem is you can't extend every consumers browsers. Or can you? Hmm. :0
For an office, you can create your own CA, to sign other certificates. You can use this one CA, to sign all your services, web, email, etc. Then install the public key of the CA in every workstation during the installation procedure. Proper trust hierarchy... no annoying messages. That would be the point of the entire thing.
It makes me wonder if you can attempt to install a self signed certificate in IE, will the user care? Is this a valid way to avoid VeriSign? You can do that by directing the user to a .crt file in IE... it will download it, and open it, and prompt the user to install it.
I wonder if there is a way to make this more friendly for the user, through JavaScript for instance.
"Dear Customer: you will be prompted on weither or not you trust Shopping.com's Certificate Authority to establish secure connections to our server. Accepting this is required in order to establish a secure connection to our server."
I wonder if that would go over well....... seems like a easy way to escape VeriSign.
What the hell does that mean, what does it do, and who do we sue[...]?
With that kind of reaction, I think you've more than proved you've got the mettle to be in management.
If it's for-profit but free, you're not the customer -- you're the product (e.g., the Slashdot Beta's "audience").
Or is it merely that some software automatically calls the mothership for new information on expiration, and the hostname of the mothership happens to start with "crl"?
(Antidisclaimer: I operate five private CAs and delude myself that I basically understand this stuff.)
"But all your emitter and collector are belong to me!"
"Although VeriSign has been providing instructions on how to manually install
the new Global Server Intermediate Root CA to all GSID customers since
December, 2001, it is possible that some customers may not have noticed the
reminder and are unaware of this issue."
Or like me, it's a case of it was fixed (I know it was because I was the one that did it in early 2002) and now they are trying to figure how (and when) it got broken again....
- Not fuck up,
- Not fuck me over
But don't let it go to your head, l33t_d00d, that says more about them than you.the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
You AREN'T going to believe it, but when I lived in the state of Delaware, they actually did this. Granted, they didn't notify me just so they could send me more money. They sent me a letter because one of my pieces of documentation somehow never got to them. When I called to find out exactly what they were missing, they told me that I had also missed one of my deductions that I could have taken. In the end, it only amounted to about $50, so it wasn't worth it to file an ammendment and chance the audit flags in the future, but I was completely shocked that they pointed my mistake out.
GreyPoopon
--
Why is it I can write insightful comments but can't come up with a clever signature?
Except the Verisign cert actually translates to "This conversation is encrypted, and I paid Verisign a bunch of money so they'd say I'm me." Verisign does fuck all for identity checking. I'm sure they'd gladly issue an SSL certificate to Santos L Halper, as long as he paid them.
The fact is, this is a huge problem, in that you have to basically pay protection money in order to sell stuff online. SSL certificates should be available from state governments, when you get your "Permit to Make Sales at Retail" and that sort of thing. It wouldn't be that difficult to implement.
Also, someone needs to get together and start a new, free Certificate Authority. Or perhaps a nominal processing fee, like no more than $10. They could easily get their root CA into Mozilla and the other open browsers. Netscape probably wouldn't be terribly difficult. IE would of course be nigh on impossible, but that wouldn't be too terrible, I guess. There are enough huge companies backing Free Software these days that it wouldn't be impossible to convince them to start using this new root CA. After all, a free CA is a logical next step from Free Software, in my opinion. Of course, there's the problem of how to verify that people really are who they say they are, and there's no good way to do that without at least coming in in person. Which is probably why local municipalities are a better choice. Companies have to fill out a bunch of paperwork when they want to get started in an area - it wouldn't be hard to issue certificates then.
The problem is, so many cheapskates have now signed their own certificate that the bogus authority error isn't stopping users since it's so common when nothing's really wrong. As a result, we're seeing a lot of look alike sites use SSL to get the padlock to come up, and users not being phased by the red-flag alerts that this doesn't seem to be the site they think it is.
Calling them cheapskates is a bit harsh. It's like saying "those cheapskates who walked to work instead of buying a Lexus". Personally, I think they're quite right to sign their own certs, explain it to their customers, and help to undermine Verisign's "trust", since it's not really trust anyway. The problem is with the system itself, not that people don't want to prop it up.
There is no sig, there is only Zuul.
I'm guessing that this Denial of Service effect is largely due to the known scalability problems with X.509 CRLs. In a mature Public Key Infrastructure (PKI), about 1 in 6 certificates is revoked. A CRL is around 20-30 bytes in length for every revoked certificate.
That means that if you've issued 250,000 certificates, you can expect to have a CRL of about 1MB.
This aggregate information isn't bad for some back-end processing, but when a lot of clients try to grab the CRL, you can quickly saturate even a high-end 100Mbps hosted server farm.
Virtually every serious large-scale PKI (including VeriSign and Microsoft) is moving to OCSP to replace CRLs since each client will retrieve ~1kB per status request rather than a full 1MB CRL.
There is a file in the JDK called cacerts.
(find . -name cacerts is your friend), this contains the certificates Java uses when initiating ssl connections.
As of yesterday Sun was still shipping java with the expired 3a certificate.
The way to include the new 3a certificate is to use the keytool command.
The format is somthing like: keytool -v -keystore cacerts -import newcert.pem
The default password for java's cacerts file is "changeit"
VC
ps how many geek points do i get for fixing this last week?
Official GOD FAQ.
This vaguely reminds me of the fraudulent Verisign / Microsoft code-signing digital certificates that Verisign issued a few years back.
While not an identical problem, an essential element of why those certificates were potentially harmful was also because of a problem with the CRL checking. Verisign didn't support CRL distribution points in their certificates and you all remember the problems that ensued.
I found security researcher Gene Spafford's comments on the PKI / Verisign issue interesting, which were picked up in Bruce Schneier's Crypto-Gram. Schneier's comments on the incident as well as the Microsoft response are also worth reading.
It's unbelievable that Verisign which claims to be in the business of Internet security and SSL/TLS digital certificates - the dominant company with 95%+ market share - could let their Root Certificate Authority expire, then force its users to effectively patch their systems by importing the new certificate for the root CA after the fact. That's just bad engineering.
Yes, end-users need to take some responsibility for their systems, but PKI and related technologies are complex and not for novices. It's no better than the keep-your patches-updated-and-use-a-firewall comment that Bill Gates made a couple of months ago. That's a bandage, not a solution.
Well, after all, I should not be surprised to hear that, after the wildcard affair. They are definitely the masters for messing their DNS...
Free six-month certificates - these really work, at least for recent versions of IE. I have one installed on the SSL server in my garage. Issued by some good people in Barcelona.
IPSCA
"Don't belong. Never join. Think for yourself. Peace." V.Stone, Microsoft Corporation
updated to reflect real world:
[root@kjell root]# host crl.verisign.net
crl.verisign.net has address 198.49.161.206
crl.verisign.net has address 198.49.161.200
crl.verisign.net has address 198.49.161.201
crl.verisign.net has address 198.49.161.202
crl.verisign.net has address 198.49.161.205
as of
serial = 2004010701
Thu Jan 8 23:17:57 CET 2004
note the 01 in 2004010701
-- for undocumented cisco commands, take a peek @ dotu
Except for the fact that Verisign owns Thawte.
Excuse me, but I work not 50 feet from VeriSign's Authentication and Verification department, and they do so much verification of businesses purchasing SSL certs that they regularly get bitched out by customers for all the information they have to provide before the cert gets issued.
State DBs are checked, D&B is checked, and multiple phone calls are made. With the obvious exception (remember the MS code signing cert misissue? or do you even know about that, you fucking moron?) of an employee who failed to follow procedure (and was subsequently let go for it), those people work their asses off to verify identities, regularly working overtime to make sure certs are issued in a timely manner.
In short, shut the fuck up asshole. You don't know what the fuck you're talking about.
Self-certificates are worthless except when distributed through an existing secure channel. Without a proper certificate, all I know is I'm encrypting the session key with someone's public key, but I don't know whose it is. I might as well send the contents in the clear.
Let's be honest. Who here trusts Verisign? If you trusted them before, do you trust them now?
All this whole ordeal seems to have shown is that Verisign (or in general SSL's) method of verification and validation is completely unscaleable.
Why don't we use a loose-knit network of trust like GPG? We could still have root certificates which are ultimately trusted if the user wants, but would be able to set up little isolated trust networks which wouldn't be crippled by this sort of stupidity.
Karma: It's all a bunch of tree-huggin' hippy crap!
True, but there are far cheaper options still that are effectively as good for 98%+ of the web surfing population. Go to www.ev1servers.net and get a GeoTrust certificate (GeoTrust acquired the old Equifax cert business, and the Equifax root cert is in browsers going back to IE 5.0 and Netscape 4.something I believe). And ev1servers.net will sell you a $150 retail price GeoTrust cert for 49 bucks. You'd have to really want to capture the "wicked old web browsers and Windows 95" market to justify the marginal cost of a Verisign (or Thawte) cert over this (900 bucks for a 128-bit cert from Verisign... lol).
beTRUSTed, which recently purchased Baltimore's CyberTrust and OmniRoot businesses. I used Baltimore's certs all the time to avoid VeriSign.
Digital Signature Trust, a subsidiary of Identrus. I've used their TrustID certs to avoid giving money to VeriSign as well.
Both of the above certificate authorities have their roots in the most current IE and Netscape/Mozilla browsers. Digital Signature Trust does a lot of stuff with banks (being owned by Identrus, which was created by a bunch of banks).
I haven't tried them personally
I have, and we are now actually a reseller for them (although we only "resell" it to the people we host). ChainedSSL (Equifax in Astroturf) has been working hard to switch us over to their certificates. They're trying to spread a bunch of FUD because the InstantSSL certificates have a root that is owned by Baltimore, which has just been bought out. But InstantSSL has much better browser compatibility (something like 99% of all browsers vs. Equifax's 95%).
They generally have very fast turn around, usually you can get the certificates that day if you have your documents in order.
The nice thing is that once you're a reseller, you become responsible for the the validity of the seller, which means that certificates are issued as soon as you submit them.
Karma: Chevy Kavalierma.
Verisign once issued a certificate to a fraudster who claimed to be Microsoft, prompting MS to issue an emergency patch for even otherwise-unsupported OSs.
If Verisign won't even bother to verify the identity of their own partner in monopoly, do you really trust them to check anyone else's?
I suppose I should have linked. Here is a link:
n load/pages/US-N95.html
http://securityresponse.symantec.com/avcenter/dow
falcontx
It is easier and less detectable to sniff a connection than it is to intercept and modify all data flowing over the connection. Thus a self signed cert is better than nothing, but it does indeed have obvious security failings.
I work at a CNC machine shop and the app that sends programs to the machine broke today because of that. I would have never heard about it if it wasn't for my brother in law, who works for a company running the same application.
The fix was as follows: Open Internet Options, click Advanced tab. Under Security turn off both Check for Server Certificate Revocation and Check for Publisher Certificate Revocation. I think this fix should work for other apps that are affected by the same problem...Thought I'd pass it along.
On a side note, it's pretty scary that this has happened to begin with. What I had to go though was pretty minor since the problem was on one machine, but what about an entire enterprise with an app installed on 1000's of computers that were broken because of this? Because of all this ridiculous "signed app" nonsense, not only are you down, but through proxy Microsoft made you dependant on one of the biggest bastardized companies I know...Verisign. Don't expect this problem to fix itself in a timely manner.
If this is a sign of things to come, Palladium will bring Hell on earth.
-R
I would love to see the Federal Trade Commission start granting digital certificates for little or no cost. Governments are already responsible for public security, and for granting identification documents such as social security cards and drivers' licenses, and for communications services such as running the postal service and opperating the Do Not Call Registry... why don't they do these things in the digital realm as well?
Mind you, I'm not calling for government regulation of the Internet... and certainly there is no way that government certificates should be in any way a requirement for opperating a secure website. There must still be commercial options available - and I'm sure they would become a lot more reasonably priced in the face of public competition. But if govenments are going to start taxing the Net (which they will), then certifying SSL certificates is the kind of service that they should be giving people in return.
SpyDock: Scientific Python in a Docker container
Wouldn't have been so bad if it was just my company, but folks from other companies, friends of friends, political buddies of friends of friends...
A certificate is so named, because the signer has CERTIFIED the holder to be trustworthy.
...] so if one of them says he's confirmed Joe Bloggs' identity, that's good enough for me; (full trust)
You'd think so, wouldn't you? Unfortunately for the sanity of anyone using a certificate architecture, you're wrong.
The certificates issued by Verisign and other Certifying Authorities are more "proof of ID" than anything else; the CA makes no assertions about the trustworthiness of the owner, they just assert that the public encryption key you've just been sent belongs to the same people who own the server you're connecting to.
A typical CA certificate as used in SSL, translated into English:
"We hereby certify that the following RSA key [...] belongs to the owner of shopping.example.com. Signed, Verisign."
When your browser connects to https://shopping.example.com, the server sends you its certificate, and the browser checks Verisign's signature on that certificate. If the server proceeds to steal your credit card number, subscribe you to undesirable mailing lists, etc., that's between you and example.com; it's only Verisign's fault if it turns out they issued a wrong certificate.
PGP uses the same principle: when you sign someone else's key, the statement you're "signing" is something like this:
"The following public encryption key [...] belongs to Joe Bloggs ; I have met Joe and verified the photo on his passport. Signed, pclminion."
GnuPG (and probably PGP) never talks about certificates, only about signatures.
If that certificate is later used to commit a felony, say, credit card fraud, then YOU could be held legally liable, because YOU CERTIFIED that this guy was trustworthy. You were negligent in failing to find out that he wasn't.
The only way you could be held responsible is if it turns out that you were so sloppy about checking Joe Bloggs' ID that you were actually negligent; (i.e. didn't check it at all, or accepted an obviously fake form of ID, or something); in most jurisdictions digital signatures aren't legally binding anyway.
Anyway, this is what the trust mechanism in PGP is for.
[Digression: You can build up a "web of trust" by saying things like:
- I trust [... some people
- these other people: [...] I don't trust so much, but if three different people all say they've confirmed Joe's identity, I'll believe that they're not all conspiring against me, so that's OK too; (partial trust)
- everyone else either I don't know, or I know but don't trust, so I'll ignore what they say when I make my decisions.
(These trust values are a private decision, there's no reason to reveal them to the world.)
end digression]
If you incorrectly sign someone's key, and a third party gets hurt as a result, you could easily argue that it's that third party's fault for trusting your opinion.
Incidentally, you can emulate the "certifying authority" model in PGP by giving full trust to Verisign, Thawte et al, and no trust to anyone else. This is a painfully limiting model compared with the full web of trust, though; to me it looks as though the whole mechanism was designed to make money for certifying authorities.
P.S. That was a joke....
Ummm, no it wasn't. You may *think* it was a joke, but trust me it wasn't.
I have walked a user through performing the following procedure, and she has reported success with her two machines. She is running Windows 2000 Pro with Office 2000 and NAV 2003 (only 99% sure about the last one).
:P
- goto http://windowsupdate.microsoft.com/
- click Scan for Updates link (may be prompted to accept the ActiveX thing)
- Navigate to the page of non-critical updates (ironic, no?)
- Find the update named something like "Root Certificate Update" or "Root Certificate Authority" (can't remember which)
- Install it
- rejoice at the ability to use MS Word again
Keyboard not found.
Press F1 to continue.