Slashdot Mirror
Questioning Google's Disclosure Timeline Motivations
An anonymous reader writes "The presence of 0-day vulnerability exploitation is often a real and considerable threat to the Internet — particularly when very popular consumer-level software is the target. Google's stance on a 60day turnaround of vulnerability fixes from discovery, and a 7-day turnaround of fixes for actively exploited unpatched vulnerabilities, is rather naive and devoid of commercial reality. As a web services company it is much easier for Google to develop and roll out fixes promptly — but for 95+% of the rest of the world's software development companies making thick-client, server and device-specific software this is unrealistic. Statements like these from Google clearly serve their business objectives. As predominantly a web services company with many of the world's best software engineers and researchers working for them. One could argue that Google's applications and software should already be impervious to vulnerabilities (i.e. they should have discovered them themselves through internal QA processes) — rather than relying upon external researchers and bug hunters stumbling over them."
If your company is producing security critical software and doesn't have a plan for quickly dealing with bugs you suck.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
For article. Sadly you can't moderate an article.
Just saying it like it are.
Google has a decent point, but they're also trying to nudge the industry in a direction where more businesses will leave the driving to them, or to cloud competitors like Amazon.
Why single out Google? Shouldn't traditional software vendors have also run programs through QA?
Just because you commit to a certain turnaround, doesn't mean you won't beat it. Anyone in business knows not to produce warranties or guarantees that you can't meet every time. People that over promise and under deliver typically aren't around too long.
And they simply don't do anything? I've contacted companies about security flaws I've found in their products and was met with deafening silence.
Until I publicly announce them on platforms like Twitter, then you have their full attention.
Testing can find the presence of bugs, not the absence of them. An ages old adage that has withstood the test of time because it is true. Only the new to the game or naive would think otherwise.
-- I ignore anonymous replies to my comments and postings.
Google recommends 7 days for "critical vulnerabilities under active exploitation", and 60 days for vulnerabilities that are assumed to not yet be known to attackers.
Frankly, even 7 days is too long for active attacks. Publishing the vulnerability lets users to use a workaround or shut down the service or app entirely until a fix is released.
A vulnerability that is already being exploited needs to be fixed right away. It's called 0-day for a reason, not 7-day. It should be disclosed immediately to force the vendor to do something about it.
You call Google practices naive - and then you say:
"One could argue that Google's applications and software should already be impervious to vulnerabilities (i.e. they should have discovered them themselves through internal QA processes) — rather than relying upon external researchers and bug hunters stumbling over them."
I don't think it's relevant in the context of the article in the first place, but. Are you maybe aware of any engineering process that would find and eliminate all bugs in software? No - so why are you even proposing or referencing such idiotic ideas.
The argument "Google can do it, because it is web software company and others are not" is not true. Google Chrome, Android, Google Earth.....
Sounds like 95+% of the rest of the world's software development companies should not be making thick-client, server and device-specific software as their security detail is unrealistic. Many businesses are rather naive and devoid of exploit reality.
a 7-day turnaround of fixes for actively exploited unpatched vulnerabilities, is rather naive and devoid of commercial reality. As a web services company it is much easier for Google to develop and roll out fixes promptly — but for 95+% of the rest of the world's software development companies making thick-client, server and device-specific software this is unrealistic
Hello there, mr/ms/mrs anonymous COWARD, what are you saying there? It COSTS TOO MUCH to prompty (as in a week) fix ACTIVELY EXPLOITED vulnerabilities? When you get the actual problem handed to you on a silver platter? What company do you work for?
When the copyright term is "forever minus a day", live every day like it's the last.
I thought for sure this was written by a teenager. You generally don't hear this kind of naievety from adults.
"That's too fast, it's unrealistic, by the way, Google should write perfect choose and fix vulnerabilities 10 times faster than the unrealistic timeframe I just complained about."
Yes, there's a contradiction in that sentence. It's not my fault.
a 7-day turnaround of fixes for actively exploited unpatched vulnerabilities, is rather naive and devoid of commercial reality.
I read that and I was thinking, "Well, yeah, sure - I shoot for one hour and can't recall the last time it took more than a day to get a critical bug patch out, but that's not really reasonable for everyone. The team I work on is pretty focused on keeping the tracks polished so we can get high priority things through. I think 7 days is OK. It could be better, but it's OK. And Google isn't even saying it will take 7 days, they're saying 7 days is the max. But, whatever, I guess -- ultimately agitating for faster patches is something I support."
for 95+% of the rest of the world's software development companies making thick-client, server and device-specific software this is unrealistic.
What?!? You mean it's not realistic to get the patch available within 7 days? I mean, obviously you can't expect users to have their systems patched immediately, and sometimes a third party (like a walled garden approval path) can lock you out. But is the writer saying 95% of companies can't even have a patch pushed for release in 7 days?
If that is true, we, as a society, need to drop what we're doing and focus on security, build management, QA workflow, whatever it is that is making that a reality. 7 days is acceptable. 95% of companies can't hit 7 days? First, that is not true in my experience. But if it is? That is not acceptable, if it is true. There really are bad people out there trying to root our electronics. Seven days to get a patch out for an actively exploited in the wild vulnerability is enough. Work the problem. Figure out why you can't hit that number, and fix it.
Stop-Prism.org: Opt Out of Surveillance
So the OP is bitching Google is like the Empire offering rewards that people like Boba Fett go after? I don't even
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
"Security critical software" is anything on the Internet.
Let's say I write a clever new web game, called "Cyber Tic Tac Toe." It's based on the classic idea of tic tac toe, but with the novel twist that it's on the Internet! As fun as my game is, and despite the fact that you are totally addicted to my awesome game and play it about 20 hours every week, questing to get your 3 Xs in a row, some people would say this is completely mundane and unimportant. (Killjoy bastards!) They would say my game, no matter how kickass it is, isn't in the same realm as your electronic payment verification app.
But I screwed up. My online game was rather poorly written, and someone can rather easily use my poor code to get the server it runs on to become a spambot. So cybertictactoe.com is sending out a ten million Viagra mails per day. Someone can also use this hole which lets them take over my underlying server, to have my game attempt to use any known browser exploits to attack my game players. So now the people who play Cyber Tic Tac Toe are also sending spam, and also running keyloggers (installed from my server) whenever they're talking to your electronic payment verification app.
"Security critical software" == EVERYTHING THAT EXISTS
Having worked in software development for 30 years, let me tell you a dark little secret.
QA programs are never enough.
It matters not how good your QA section is, things will get through. Why? Because the QA people have looked at the code, because QA managers are fond of believing that filling out paperwork is the same thing as careful retrospection, because the QA people work for a company whose vested interest is in getting that chunk of software out the door.
In order to do a thorough job, one needs Different Eyes.
I cannot tell you how many times I've had an intractable problem with a piece of software, and shown it to a coworker not associated with the product, only to have them turn to me and say (in effect) "You dumbass, it's right here!" Which, in fact it was. I'd have looked at it countless times, and I might well never have seen the problem.
Running software through extensive QA is an excellent idea, but in and of itself is not sufficient. Having someone not connected with the product, someone with "different eyes" look at it, is a time proven method to find things that your excellent and well managed QA department will never see.
Don't take life too seriously; it isn't permanent.
It's no wonder this article was posted anonymously. The whole tone and writing style is exactly what one would expect in a position statement cranked out by a corporate PR flack. I wonder whose flack it is.
~~~~~~~
"You are not remembered for doing what is expected of you." - Atul Chitnis
I think what you're saying, is that if someone is going around stabbing people in the heart, and if a doctor says these victims all need immediate medical attention (even the victims which are in isolated areas far from hospitals), then that doctor is being naive and devoid of medical reality.
I personally think you should quit blaming the doctor for the unfairness and horror that is inherent in the situation. Declaring the urgency of a problem being addressed, isn't "naive". It's not naive, even if addressing the problem is incredibly hard or even if it's effectively impossible.
If the doctor truly thinks the victims all really will get "immediate medical attention" then he'd be naive. But advising it isn't naive. Yelling at people "get that victim to the ER as fast as you can!" isn't naive. Telling people that heart stab wounds are very serious, isn't naive.
And the analogy with Google here, is that you just got stabbed in the heart, they're advising and hoping you get immediate medical attention, and 7 days from now, if your wife asks Google if they've seen you lately, they're going to tell your wife, "I heard he got stabbed in the heart last week. You took him to the hospital, right? If not, you better get on that, right now." You're concerned Google is going to scare your wife?! Be concerned that you're not at the hospital yet!
You think Google is being naive with unreasonably high expectations, but the need for those high expectations isn't their fault!
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
Google's standards are too hard to be realistic for the rest of the industry, but the standards for Google itself are 0 security defects ever. How does that work again?
(disclosure: I work for Google. I don't speak for them.)
What a dumb opinion piece.
The main difference between a client-installed application and a web-app these days is that a patch on a web application is available as soon as you deploy it, while the patch for the client application needs to be downloaded and installed, which is mostly done automatically.
So, in terms of time, the difference is on the order of minutes, hours at most.
Is it more difficult to create and/or test updates for clients or for browsers? Hard to say, but the difference isn't fundamental. On the one hand you have to test a variety of OS versions, maybe hardware versions. and environments. But you have to do that for any update and for development, so you already have your test lab up and running by the time you need to roll out a security fix. On the other hand, you have to test a variety of browsers and OS versions... and again you already have... basically, same thing with a different test lab.
Will customers, especially corporate customers, delay the deployment by running their own internal tests and/or waiting for their next internal patch day? Sure, but that's not your problem.
Can you afford to delay a security fix? No. Ever since Code Red, Slammer and my own and other people's work on various worst-case scenarios for flash worms, we know that a remotely exploitable issue that allows code execution can be used to infect 90% of the vulnerable systems in less time than it takes humans to react with any temporary workaround such as additional firewall rules or service shutdowns.
And before someone starts the argument: That is true for undisclosed 0-days as well. If some whitehat found it, chances are good some blackhat has also found it already and is at least working on an exploit.
Assorted stuff I do sometimes: Lemuria.org
Industry is irrelvant, if your customers are being actively exploited you are or ought to be liable if you don't fix it as fast as possible
SURELY NOT!!!!!
Google isn't taking into account the time required of customers to test and install the patch. There are commercial software systems consisting of more than one product that must be carefully deployed to avoid downtime and a fix isn't a patch, but an upgrade. This isn't even accounting for hardware based vulnerabilities.
Sixty days mean the vendor must aim for at least 30 days to create fixes for all affected releases on every platform and pass quality assurance while allocating resources away from planned time-dependent projects. It would be a fire drill every time. And pray that the vulnerability isn't a design issue.
It shouldn't be shocking to find most commercial software contains vulnerabilities. Even with active exploitation, publicly publishing vulnerability information before a vendor can create a fix will mean higher levels of exploitation. And once that hole is fixed, the crackers will find the next one. The cycle continues.
Any company that performs software security auditing will find many times the vulnerabilities that are reported by security researchers. Google, no doubt, audits their code, but there are still vulnerabilities being reported by independent researchers. They haven't figured it out even with their resources.
Google receives quarterly net profit in the billions and cannot imagine the perspective of the software company with razor thin revenues. It has the same feel as a multimillionaire saying the poor should just make more money and that will solve their resource issues.
Oh, I'm completely with you: Developers should run QA, but there will always be bugs that slip through. I was responding to the AP's insinuation that Google should be catching all their security problems in internal QA but that Microsoft should get a pass for some reason.
The debate has always been about whether to give any advance notification at all. That's why full-disclosure@lists.grok.org.uk is called "full disclosure". That's the spectrum of behaviour that's allowed, anything up to 0, "notify vendor and public simultaneously":
https://en.wikipedia.org/wiki/Full_disclosure#Arguments_against_Coordinated_Disclosure
Even if not for that, the complaining seems to rest on a vague suggestion that Google's acting somewhat in their own interest instead of always doing the most altruistic thing. Fine so far. Google will probably listen to such a complaint. But that's not what this really is. The reason Google's disclosure practice makes their security look better than your security is that their security _is_ better than your security, so this devolves into "why isn't Google acting in my interest instead of user's interest?" derp.
As a user, I don't care about the vendor's ability to fix it quickly. Really I don't. That's their problem. My problem is that my systems are vulnerable to compromise and I have to do something about it. I need to know what the vulnerability is, in enough detail to understand it myself, and I need to know the possible workarounds (not just the vendor's recommended one(s), which is another reason I need to know what the vulnerability actually is so I can understand all the other possible ways of dealing with it). I need to evaluate my options and take whatever steps I need to to protect my systems. If the vendor needs a month to get the fix through their change-control process, I still need to protect my systems today.
The vendor's advice will be based on their most-likely scenario. Problem is that my situation may be radically different from the vendor's most-likely one. There's definitely going to be local considerations even if my situation's one the vendor's workaround covers. I need to understand the vulnerability to be able to evaluate it intelligently. It may not even be relevant to my setup. If it is, I may have less-intrusive workarounds (eg. for the SSH OTP authentication bug, if we've got a purely internal network that isn't accessible to the outside world or the Windows desktop portion of the network it may be less intrusive to just monitor for attempted exploits and defer doing anything until I see someone having gotten past the air gap rather than changing an authentication method that a lot of people depend on and that can't be exploited easily without being physically in the building). And if I need to take drastic steps like disabling the vulnerable SSH authentication method, I may have clients who insist they must be able to use it (maybe because their systems are based on it and they need my systems to integrate with their authentication because I'm providing services to them) and I need to be able to intelligently discuss exactly what's wrong and why it's simply not possible to use that method without being vulnerable and we've got to change to a different method despite the disruption. I can't do that unless I understand the vulnerability.
Notice that in all the above I haven't mentioned the vendor at all. Like I said, the vendor isn't relevant at all. It's my systems that're vulnerable and me that has to do something about it. If the vendor already has a fix then well and good, but if they don't it doesn't change my situation. When vendors say they need more time, they're asking me to leave my systems vulnerable without telling me they're vulnerable. Sorry, but no. Not, that is, unless they're willing to shoulder 100% of all the costs resulting from that vulnerability being exploited. Not just direct costs, things like the costs of lost business and clean-up if the vulnerability is exploited and liabilities I may incur because of the compromise. If a vendor isn't willing to take on that liability, then they don't get to tell me I shouldn't have the information I need to protect myself from that liability. If they don't like it... this is the sound of the world's smallest violin, playing the world's saddest song just for them.
It would be nice if Anonymous Coward who posted actually knew what it's like to produce software. "A simple code change" can often have unexpected repercussions, including but not limited to expose additional (sometimes worse) bugs/vulns, and can impact usability. This is why commercial companies have QA teams. And it's not like automated QA is instant or complete, and it's not like manual QA is instant. 80 days to turn around a fix is completely reasonable for some fixes. Google fixes bugs all the time and I'm sure that they picked 80 days based on the actual time it takes them to fix other bugs.
OP should get a job in the real world before s/he complains about how the real world works.
What an idiot. Is every security defect supposed to be one line of obviously incorrect code whose fix requires no QA / review to ensure that it doesn't introduce new security defects? I guess so since Google should catch all of them in the first place...
Pitiful cries of "*SNIFFFF* Butttt, butttt, we've been using paying customers as guinea pigs for decades, we'd actually have to pay someone to test our shit code now!!"
"That's just not fair, we charge more because we have to pay stockholders and multi-million dollar bonuses to the dickheads who make our fucktarded decisions!"
"Without paying testers, how else can we rake in more cash, more quickly, especially when our EULAs make it so they cannot sue us when our security software kills their PC!"
Let's cut through several layers of meta-issue here and get to the heart of the matter, and then work our way back outwards to the issues this article is so wrong about:
1) Software Development is arguably one of the most difficult, complex, and daunting tasks humankind can endeavor at. That is, assuming you care about factors like quality, efficiency, security, and correctness. What makes it so difficult is that you're molding things out of the most flexible clay ever dreamed of. Physics is pretty much the only limit on pouring out any machination your brain can dream up, good or bad. Only a very small slice of the human population has the right sort of brain to do it well. Of those, only a small percentage bother to take up this profession and endeavor to improve their skills over time. Of those, only a small percentage are still writing software a decade or two later, when they're finally seasoned enough to do it *really* well. Many burn out from bad jobs, or just build up enough cash and resume length to shift out to retirement and/or management and/or running businesses.
2) Given the above, there are very few active, highly skilled, and seasoned developers at any given point in time. As our world increasingly comes to depend on software, however, we keep needing more and more developers. There is vastly more demand for software than there are quality software developer hours to go around. If this were accurately reflected in salaries, good developers would make much more money than they currently do, perhaps by an order of magnitude. They're worth that much easily. The reason it isn't accurately reflected is that many non-developer decision-makers who start/run businesses fail to understand the vast guld of difference between a random junior coder or random H1-B import and a truly talented developer. They figure they can trade one good one for 5 bad ones at a cheaper rate and everything's fine. So that's the market...
3) ...Which brings us around to most software sucking. Most software is written by people who have no business doing so. They cling to certain practices and ideologies hoping they can systematically improve software quality without improving developer quality. Here's a hint: there is *no* formal process that can squeeze high-quality code out of low-quality developers. None. Period.
In the modern era, everything is connected. All software is connected. This means security suddenly becomes even more critical. Security can't happen without quality and correctness. If you can't write secure, good, software, you are to blame for releasing your ugly, buggy, insecure mess. You charged customers money for that piece of shit, and they reasonably expect your bugs to be minimal and your security response to be swift.
If your business model does not include (a) putting out reasonably high- quality/security code from the get-go and (b) very quickly resolving any security bugs that arise, then your model is broken and you deserve to go out of business.
And here's a hint: it's far easier to release bugfixes quicker and keep a handle on security issues when your code is running in-house instead of out on N-million customer computers/devices. Therefore it makes sense for a smart company to shift as much code in-house as possible. This is why every website isn't a separate desktop application, for example. That just wouldn't work in the modern world. You want to minimize the device software. Keeping a small core device codebase secure is relatively easy (e.g. Android, ChromeOS), and then have it access service software over standard protocols and keep the service software "in the cloud" (ugh) where you can keep a close professional eye on security incidents and respond to them swiftly (near-realtime) without worrying about how you get an update to N-million devices and not miss any.
So basically, Google's business model solves the real problems in today's world well. They hire really smart guys and they keep as much code as clos
I guess Chrome, ChromeOS, Android, numerous Android apps, Google Earth, Google Drive, Picasa and Google's many other traditional installable software products don't count.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Is the suggestion seriously that Google's products should somehow be 100% secure? Do we live in a magical dream world where complex software products can be produced without any possibility of security issues? That's insane.
Well, I think the AP's point is that Google has many of the worlds best software engineers working for them, and Microsoft does not. :-)
Sadly there's a lot of code that sees neither QA people or QA robots.
I may be wrong, but unless you hire attackers and hackers to truly find what can be exploited you can do all the QA you want with standard minded folks and find the obvious holes that your programmers didn't look at more carefully, or hire attackers that are going to find those holes, it will never be enough or there will never be "full proof" software but even with QA they seem to almost intentionally allow or overlook holes.
I find the comments in the article to be naive, while Gaagle should be able to find holes in its software or any ware, it is often the holes that most do not see or bother to notice that get blown up, ironically they (other besides Gaagle) also have a habit of allowing such blatant holes, and always come up with some good PR to excuse such stupidity.
I would add the obvious, it costs money to hire full time attackers, and even in doing that they may not be able to find much if anything, or you can just use the wild environment to get younger folks, and outside researchers to find them for you. Gaggle has a habit of cheap skating when it comes to making others do all the work for little to nothing while they take credit for it.
QA should be performed by an independent 3rd party that didn't have any direct knowledge about the development. Companies should have a team dedicated to nothing SQA, even if that annoys the hell out of developers. A team who's main purpose is to find ways to break stuff.
Unfortunately, that rarely happens and the same people who wrote the software are the ones writing the test procedures and running the quality test. And even the most dedicated and honest professional has blinders turned on and will write the procedures to support the way the software was designed and not the way the software is intended to be used.
In the case of Google, they just don't do anything beyond your basic automated unit test .... which rarely catches anything mayor.
.... is one thing. Publishing the code that exploits it is another.
What Google is saying is you have 7 days before I release the code that will screw your users.
We all know where this is coming from. Microsoft can't fix their so-called software because their organization is designed to crank out shovelware for retards, not professional-grade tools for serious adults.
What more proof could you need that Microsoft "products" should be avoided like the radioactive plague that they are?
People who approve of Microsoft software for professional environments should be fired for incompetence.
> As a web services company it is much easier for Google to develop and roll out fixes promptly — but for 95+% of the rest of the world's software development companies making thick-client, server and device-specific software this is unrealistic.
If you are unable to sell safe bicycles, do not sell bicycles. It really is that simple.
Having worked in software development for 30 years, let me tell you a dark little secret.
QA programs are never enough.
Dropping previous moderation to comment here:
Having worked in SQA for over 17 years I can tell you PM NEVER listens to SQA unless the app would be DOA on more than 20% of the target customer's machines. On occasion a XXO would override PM and tell them no... you cannot RTM until you fix *THIS* issue... but security, data loss, customer satisfaction were never significant guidance to PM or XXO decisions.
And these issues are not deep... this is low hanging fruit, more often than not. PMs and DEVs hate SQA and wish we would DIAF. They see us as a headache to be mitigated, not an instrument to measure quality.