PHP, Python and Google Go Fail To Detect Revoked TLS Certificates (softpedia.com)

An anonymous reader writes: Four years after the release of a groundbreaking study on the state of SSL/TLS certificates in non-browser applications (APIs [to be exact]), some programming languages fail to provide developers with the appropriate tools to validate certificates. Using three simple test scripts connected to a list of known vulnerable HTTPS servers, researchers logged their results to see which programming languages detected any problems. According to the results, all tested programming languages (PHP, Python, Go), in various configurations, failed to detect HTTPS connections that used revoked SSL/TLS certificates. This is a problem for HTTPS-protected APIs since users aren't visually warned, like in browsers, that they're on an insecure connection. "PHP, Python, and Google Go perform no revocation checks by default, neither does the cURL library. If the certificate was compromised and revoked by the owner, you will never know about it," noted Sucuri's Peter Kankowski.

Comments

Sooo...

[ADRA]

I don't use any of these programming languages. Maybe a little more exhaustive list of working / no-working languages and libraries would be a great resource.

Re:Sooo...

[DNS-and-BIND]

"Most languages do OK while these few aren't" isn't a news story. I think you don't understand how clicks on the internet work. Alarmism gets people to click, reporting facts doesn't.

Re:Sooo...

[Anonymous Coward]

Or, more to the point this retarded summary ignores the fact that the article itself is about libraries supporting those languages.
And of course plenty of other libraries as well, and old versions of browsers, and a ton of other stuff.

In fact working TLS Certificate revocation (not the theory, the actual use) is a relatively new thing, so it is not that surprising that
there is plenty of infrastructure out there that either does not support it, or does it wrong. It is also not exactly simple.

Is it a bad thing to report? of course not.
Is this a hyped and wrongly focused article? You bet.

Welcome to the InterWebs.

the tools should make this easier

[WatchMaster]

in java at least, it is quite a project to properly validate a chain of certificates to a root with full support for the several types of certificates. These libraries should provide a simple binary function that can validate a certificate, including revocation. That is - this is an area where experts in this subject should provide standard libraries. Much like one should be wary of writing your own encryption, one should be wary of writing certificate validation code without the required expertise.

Re:the tools should make this easier

[Anonymous Coward]

The only problem with that is many of the CRL servers respond very slowly or not at all. For our customers, especially the ones using a GoDaddy cert, we had to disable CRL and OCSP checks in Java to make connecting to them reliable. For our PHP services, not checking those slow and unreliable lists made our services much faster and more reliable. Until companies like GoDaddy get their act together, I'm glad PHP doesn't check.

Re:the tools should make this easier

[Jesus_666]

Not binary. I'm thinking of an asynchronous (because CRL servers can be slow) function that returns a status like "valid", "invalid", "self-signed", "expired", "revoked", "known-bad CA" or "request timed out". That way you can check for validity (result == CertificateCheckResult.Valid) but still react appropriately to specific issues, even if it's just to display the appropriate error message. Something like this (in pseudo-C#):

Certificate foo; // magically appears from somewhere
var result = await foo.CheckValidity(2000); // timeout in ms

switch (result)
{
case CertificateCheckResult.Valid:
return "It's valid! Hooray!";
case CertificateCheckResult.Expired:
return "2old4me";
default:
return "I can't be arsed to write code for all the cases. Something is wrong."
}

Re: the tools should make this easier

[Jesus_666]

Good point, although this can happen transparently in the background. The method should still be called asynchronously because you have no idea whether a call will be made and how long it's going to take.

Re: the tools should make this easier

[ista]

Exactly that's the point: the certificate has an expiry date set, but what if the certificate's private key has been compromised or the certificate should not have been issued at all? Revocation services exist to invalidate a not yet expired certificate. Basically the browser/client locally validates certificate chain, expiry date and so on. If everything looks fine, the client does query an online service (ocsp) of the issuing CA if the current specific certificate has been revoked; this result may be cached for hours or even days. Another older technique are Certificate Revocation Lists (CRL), who are simply cacheable blacklists to be downloaded once in a while(crls typically may be cached for a week, but they do tend to grow to unusable large sizes - hence most browsers prefer to go for OCSP only).

Libraries

[110010001000]

This is a function of libraries, not languages. People are getting dumber in this industry.

Re:Libraries

[Fnord666]

This is a function of libraries, not languages. People are getting dumber in this industry.

The article makes this clear and lists the libraries from these languages that were used in the testing. The IQ drop seems to be on the slashdot end.

Re:Libraries

[Tablizer]

Maybe we need a new term for it all. "Language kit"?

Re:Libraries

[110010001000]

The point is use a different library then if you need the functionality. It has nothing to do with "Python" or "PHP" or "Go". And if there isn't a library that does it for your language, then write your own librarary. Don't expect everyone to implement everything under the sun.

And the problem is???

[guruevi]

TLS/SSL certificate revocation has historically never been and still isn't very robust. It's been around for decades, I remember trying to work with Verisign (I think they're still around although their certificates were awfully expensive) in the early 2000's when CRL's were introduced and more down than up and they didn't think it was very important, I simply ignored it then since most CRL servers either didn't exist, didn't work or were overloaded.

In API's the certificates that may be accepted are typically well defined where any random certificate on the other side just won't work, valid or not. I know PHP, Python and Perl allow/require you to define a certificate store where trusted certs live and it is relatively well documented not to use the systems' store unless you trust all the root certificates in it.

In addition, you can't just change a commonly used API like cURL and suddenly require things that were previously not required for no good reason. It's the same reason SSL libraries support(ed) old versions of SSL that were dead decades ago (RC4, SSLv2/v3) until an actual protocol flaw comes up.

TLDR: It's not a problem for anyone knowledgeable of the situation, API's don't just connect to random sites on the Internet and trust their data (or shouldn't) and those that do should be using the built-in features to check OCSP/CRL's.

Re:And the problem is???

[Sneftel]

I agree with most of that, but

In addition, you can't just change a commonly used API like cURL and suddenly require things that were previously not required for no good reason. It's the same reason SSL libraries support(ed) old versions of SSL that were dead decades ago (RC4, SSLv2/v3) until an actual protocol flaw comes up.

if an application is currently connecting to a host with a revoked certificate... maybe it should break?

Re:And the problem is???

[guruevi]

The developers should indeed make the program check the certificate IF they deem it necessary. OCSP/CRL's are only useful when SSL is used to authenticate a host, not just to encrypt. SSL IMHO should not be used for authentication in the first place, that's not what the protocol was designed for, we have Kerberos and other encrypted methods of authentication over unsafe channels.

To make outgoing calls just to check OCSP/CRL's which incurs a lot of extra time and resources for every call (although there is some caching possible) when in 99.9% an OCSP/CRL doesn't even exist because usually you're dealing with self-signed and internal certificates, is just ludicrous. You shouldn't expect libraries to do things it previously wasn't doing just because some developers/sysadmins are morons.

It's the same reason I don't like OpenSSL removing support for libraries between minor versions, because some developers may need it and instead of fixing the attacks by having developers/sysadmins fix their setups (ala ssl_methods = !SSLv2 !SSLv3 !RC4) or end software (Postfix, Dovecot etc) removing the feature, we're just removing support for features forcing rewrites of existing software that didn't have problems with the issue to begin with.

Re:And the problem is???

[MikeBabcock]

There have been good stories about this, like here: http://news.netcraft.com/archi...

curl | sh

[Anonymous Coward]

Can we stop it already?

Cert revokation is terminally broken. Wontfix.

[Anonymous Coward]

This is not something that should be held against the programming languages or their standard libraries. Cert revocation has never worked and will never work. All reasonable developers have given up on it. Trying to make it work is an exercise in futility and can only do more harm than good. That's why Let's Encrypt only issues very short lived certificates and plans to reduce the lifetime of certificates even more.

Re:Cert revokation is terminally broken. Wontfix.

[Anonymous Coward]

OCSP Stapling, and specifically Must-Staple, which is now RFC 7633, mean we can have revocation that actually works one day, perhaps soon. Here's how:

1. When the certificate is issued RFC 7633 is used to assert that this certificate is "Must-Staple" which means it is only valid when accompanied by a stapled OCSP response for the certificate.

2. The web server (or whatever else is doing TLS) on start-up and then periodically (say, once per hour) goes to fetch an OCSP response about its own certificate. If that stops working, eventually it's a critical error, just like if the disk broke, or the network went down. The OCSP responses come from the company that sold you a certificate. If they're unreliable, you're the customer, you demand your money back. Guess what - suddenly fixing the OCSP is a priority.

3. Whenever the web server provides its certificate to a client, it staples the last OCSP response it has.

4. The client spots the Must-Staple certificate, and checks for the OCSP response stapled to it. The OCSP response (just like today, when it's done by the client) has a timestamp and a maximum-age. If it's out of date, this certificate is worthless.

That's how we can actually do revocation. Not today, but maybe tomorrow and definitely within five years.

wget?

[Anonymous Coward]

what about zoidberg?

Requests

[Riddler Sensei]

For Python they used urllib and ssl which, yeah, are in the standard library, but Requests is kind of the de facto HTTP library. Requests does do SSL cert verification, although it doesn't mention revocation specifically.

Re:Requests

[Riddler Sensei]

Oh, no, wait, I they did try Requests and it mostly passed EXCEPT for revocation. I simply checked the github repo that was posted and only saw the urllib/ssl test.

Revocation

[Anonymous Coward]

Isn't cert revocation done by os updates these days? You cannot expect a language to maintain it's own ca store. Though it would be excellent if it did, and validated automatically.

Re: Revocation

[Anonymous Coward]

Java does, and it's a pain in the ass sometimes.

Not a suprise

[Anonymous Coward]

PHP is utter dog shit developed by clueless dumbfucks.

Python is slightly better than utter dog shit.

Google is flat out incompetent.

Main reason

[KhawarNehal523]

I think the main reason is that the software is useless if it cannot connect. So developers are forced to make clients which work first and then give options for "enabling" security. It is left up to the user to decide what level of security they want to use to connect. The servers and clients can easily decide as far as management is concerned if they want to implement strict security or not. So the main reason why some browsers do implement is because people want to be aware. Until major attacks do not occur, and customers do not care, then there is no real reason to implement stricter security. It costs developers time and makes the software unnecessarily complex. Also it runs the risk of not working which in some customer cases is not acceptable. So this issue shall be around for ever as long as people do not need security is applications running on secure and controlled networks. Technically SSL is rumoured to be breakable so only OTP could be considered for serious applications. Regards, Khawar Nehal

State and the Hot Potato

[timotten]

I suspect that most HTTP client implementations bundled with languages aim to be stateless by default. For anything that looks like state, the buck gets passed downstream. There is merit in this -- it gives the downstream developer a lot of flexibility. But then we have the problem: several standard HTTP behaviors that we usually take for granted are unworkable by default (i.e. HTTP cookies, HTTP caching, and CRL/OCSP -- which needs caching to perform reasonably).

Typically, a developer wants to say something simple (http.get("http://example.com/the/data")) rather than

cookieStore = new FileCookieStore("/var/lib/my-app-dir-that-someone-has-to-configure/cookies");
cacheStore = new FileCacheStore("/var/lib/my-app-dir-that-someone-has-to-configure/cache");
certValidator = new CachingCertificateValidator(cacheStore, {crl => true, ocsp => true});
httpClient = new HttpClient(cookieStore, cacheStore, certValidator);
httpClient.get('http://example.com/the/data');

Who wants to be responsible for administering/documenting/supporting all those fiddly bits of state?

Who Cares ?

[Anonymous Coward]

I connect to a lot of endpoints published by orgs like government departments which use private certificates issued by private root authorities, oddly enough the original principle behind certificates applies; do I trust the issuer ? If I do then I manually accept the CA and the cert into my certificate store in order for my systems to trust the remote endpoint.

If all you are going to do is blindly accept certificates on the basis that they pass verification checks then you are doing it wrong.

I laugh when I see how many "trusted" root authorities there are and just how many of them are practically untraceable.

Re:News Flash:

[Anonymous Coward]

agree , it is not the task of a language to check for revoked certificates by default . The programmer need to do that , and if not done you cant fault the language .

Re:News Flash:

[TheRaven64]

If you hit your thumb with a hammer, then it's your fault. If you cut your hand with a hammer designed by someone who thinks that razor blades are an ideal decoration to hammer handles, then it's probably the tool's fault. It's not the fault of the language, but it is the fault of whatever library you are using to handle the SSL connections if the simplest and recommended use of a library designed to establish a secure connection does not, in fact, establish a secure connection (or, at least, provide an easy-to-check error condition if it has failed to do so). If the language bundles this library as part of the standard library, then it's definitely a problem with the language.

Re:Apples and oranges

[DarkOx]

You don't have to use programmatic channels to do certificate revocation. E-mail and trouble tickets will suffice to have the certificate replaced sufficiently quickly.

And by doing so you have introduced an effective DOS vulnerability ( even if human based ) because you are asking people to make decisions to not trust a certificate based on communication over channels with poor authentication and integrity controls.

Re:Apples and oranges

[i.r.id10t]

IIRC it is either the http wrappers for the various file functions (file_get_contents, etc) or cURL in PHP that needs a special argument for a self-signed certificate, and it throws an error if you don't use the argument. Wouldn't something similar work for a revoked certificate? Assuming the back end code was updated to do the revocation check?

Re:That's called OCSP

[Jesus_666]

This is not about whether or not support is available, this is about how the API works. The GGP said that existing APIs tend to be too complicated and asked for a simple binary function to check a cert. I pointed out that a function that can return more than just "good" nd "bad" would be more useful. Neither I nor the GGP ever asserted that no API exists at all.

uh, yeah. You just noticed that?

[the stapler]

CRLs are poorly used and often ignored, largely because there is no simple built in standard for keeping the CRLs updated and having all connections check them. Its bad enough that our team has gone to only issuing certs with very short life spans and just renewing them often. See https://www.openstack.org/summ...