Slashdot Mirror
Vulnerability Numerology - Defective by Design?
rdmreader writes "RDM has a point by point disassembly of the security vulnerability story phenomenon. We regularly see these, comparing various vulnerability lists for different operating systems. ZDNet's George Ou, for example, condemns Linux and Mac OS X by tallying up reported flaws and comparing them against Microsoft's. What he doesn't note is that his source, Secunia, only lists what vendors and researchers report. Results selectively include or exclude component software seemingly at random, and backhandedly claims its data is evidence of what it now tells journalists they shouldn't report. Is Secunia presenting slanted information with the expectation it will be misused?"
> Is Secunia presenting slanted information with the expectation it will be misused?
Yes. However Secunia only does this from time to time like most companies who realise the press is a tool to be used.
Unlike Roughly Drafted Magazine, which is the most sickening pandering fanboyism rubbish published on the net. Please. Eran gives real mac fanbois and girls a bad name
... with regard to security as expressed by the faith that pure frequencies are a proper means of assessing OS vulnerabilities must inevitably lead to misuse, since any use of such measurements is.
CC.
TaijiQuan (Huang, 5 loosenings)
Did the guy who titled this know what the term Numerology means? It's usually associated with wild "magical thinking" about numbers, and is at best a rather silly form of pseudomathematics.
</Skeptical Nitpick>
Ryan Fenton
Does Secunia present slanted information?
No, it just lists vulnerabilities. But it also lists them AND presents these two important things: (a) the importance of the vulnerability, and (b) whether or not it can be triggered through the network or not (local/remote vulnerability).
Furthermore, it separates Windows vulnerabilities in system and application vulnerabilities, if memory serves well. It's not able to do that with Linux, since different Linux distros incorporate different applications.
The matrix therefore becomes a lot more complicated. You can have a 'local only' problem (meaning: no remote exploitation) which can be considered as 'critical' on some Linux/BSD systems and not on others. You can have a remotely-exploitable problem which is critical on all systems that have application XYZ installed. But if I don't install XYZ (or if it's not activated by default) on my PC, I don't have a problem. And so on and so forth.
Which is why people that point at Linux/Mac and say: "Aha! More insecure than Windows!!" are not truly honest: I have Linux and OpenBSD machines with up-to-date SSH servers, no users, a good password, and no other network service running. These machines are almost perfectly secure -- except when it comes to an OpenSSH vulnerability -- even though there are plenty of applications on them that could be considered obsolete or vulnerable... if you can gain local access in the first place. The only point of vulnerability is OpenSSH. And I update it religiously.
All in all, don't blame Secunia: blame people (especially journalists) who know nothing about security and jump on meaningless numbers pulled out of thin air to blame Linux.
The right to offend is far more important than the right not to be offended. (Rowan Atkinson)
So his list is based on vendor FUD-slinging? I don't even need to RTFA to know not to waste my time. How is this news?
The game.
Any operating system can be broken into. A bank vault can be broken into. Any OS can be rooted given an attacker has the expertise.
Any OS can be trojaned, but only one company's OS has viruses and spyware. And I think it incredibly unprofessional (incompetent?) that AV companies can't seeem to tell the difference between a virus and a trojan.
-mcgrew (not the security mcgrew, not the comedian mcgrew, but I do what I can to secure my PC and sometimes I can make people laugh).
mcgrew's razor: Never attribute to stupidity that which can be explained by greedy self-interest
And then unfortunately, their supporters like to bash Linux and Mac for actually working with security agencies and fixing their bugs as well as reporting them. This will forever be the bane of open source and it's benefit... that everyone gets to see its flaws but at the same time, everyone gets to contribute to fix them.
This is my sig. There are many like it but this one is mine.
Here's one even better: We use GeSHi (Generic Syntax Highlighter) in WikkaWiki. We often scour the so-called "security vulnerability" databases because we've found many inaccuracies. In this specific case, Secunia issued this statement:
WTF? This was a vulnerability in PHP's htmlspecialchars() function, NOT GeSHi. Yet, Secunia was planning on milking this vulnerability in order to boost its "vulnerability count" at the expense of a project that had absolutely NOTHING to do with the vulnerability.
You see, these so-called "vulnerability experts" try to wring out as many vulnerabilities as possible, because we all know that the most effective "vulnerability expert" will be the one with the most posted vulnerabilities. So they go on fishing expeditions to uncover vulnerabilities that really don't exist.
Or an even worse practice: "bottom-fishing" changelogs and bug trackers in order to discover vulnerabilities that have already been addressed. Here's another instance where Secunia was caught trying to boost its street cred through disingenuous reporting: They apparently scoured our bug tracking database and discovered an issue (already fixed!) and falsely implied in their report that the content of wiki pages marked private might be accessible via RSS. This was clearly false, as the original bug report indicated that the page name (not content) could be accessed. Secunia later corrected the false report.
We've caught Secunia doing this on several occasions. My advice to anyone who is involved in an OSS project is to regularly scour the vulnerability databases and challenge each and every advisory that you believe is not accurate. You might be surprised at the amount of so-called "vulnerability intelligence" out there that is blatantly false, outdated, or inaccurate.
I'll go you one further; vulnerability != exploit. Show me a tally of exploits in the wild, or better yet, exploits that aren't proof-of-concept. I don't think you find a single one for Macs or Linux, while the number of dangerous exploits for Windows numbers in the tens, or even hundreds, of thousands.
www.lucernesys.comHorizon: Calendar-based personal finance
..well maybe not a thousand times, but maybe I should. Security of software isn't just a product of how many flaws found. Rather it's an equation of how many people looking for flaws, the nature of the flaw and the reluctance of the company to report it (rather than just silently patching it, or worse just removing the evident symptoms but not the flaw at all.) We all know who I'm talking about with each argument.. Open source, where all changes are viewable, listed (and so on) is much more trustworthy than completely private software where the public discretion comes about from a marketing department. Additionally where the seriousness of a flaw can be completely downgraded by sole discretion.
We keep hearing this again and again and again.
It's very simple, really.
You can _never_ know the relative security of two systems. There simply isn't any way to measure it fairly.
Count disclosed vulnerabilities? What about the vulnerabilities that weren't disclosed?
Have teams search for vulnerabilities and compare the results? What does that tell you? Was one team equally good at finding vulnerabilities in one system as the other was at finding them in the other system? What if one system had many easy to find vulnerabilities, and the other had a couple of severe but harder to find vulnerabilities?
Count actual break-ins? Well, was that due to the system being vulnerable the way the vendor left it, or because of the administrator? What about break-ins you don't know about?
It's always a matter of what you don't know about. You don't know the vulnerabilities that weren't reported. You don't know the vulnerabilities that weren't found. You don't know the relative skills of the teams you used. You don't know if you tested for all possible classes of vulnerability.
And I haven't even mentioned the severity of vulnerabilities, the availability of exploit code, the way vulnerabilities are dealt with by the vendor, and a host of other issues.
The take home message is that you just _can't_ know. It's a hard pill to swallow, but you will just never know which system is more secure. All you have is flawed metrics and your gut feeling.
Please correct me if I got my facts wrong.
You must be joking! Slashdot removing dupes is like Microsoft removing backdoors, Apple removing trendiness, and FOSSies removing oppression complexes!
Windows has hundreds of thousands of known viruses and trojans, but the malware for MacOS X can be counted on your fingers. Just because Apple periodically publishes security updates doesn't mean that these vulnerabilites have ever been found outside of security labs and been exploited in the wild.
When I read the summary, I thought TFA could actually be interesting. But it's not any better than what it is criticizing.
Long story short:
ZDnet published an article comparing Secunia vulnerability counts in Mac OS X and Windows Vista/XP. They spun it the Microsoft way, so Mac OS X loses big time. A mac fanboy wrote a reply spinning it the Apple way.
TFA starts with a long-winded attack against the author of the ZDnet article without ever getting to the point. Let's just say that it talks about Zunes, XBoxes, train wrecks, ballet dancing and many more things.
Then it explains what Secunia does (in about two pages): they track software vulnerabilities which are - among others - reported by the vendors. So "honest" vendors get higher vulnerability counts. Who would have thought.
On it goes by saying that the "border" of an operating system is nowadays blurry; should the vulnerabilities in bundled applications be counted? Even if they are by another vendor?
Then he babbles about how most of the cited vulnerabilites in Mac OS X are related to what he calls "external software" - things such as python, java, perl, samba, tcpdump etc and that those same programs have the the same (or a similar) amount of vulnerabilities on other platforms. What he fails to point out is that Mac OS X *consists* of such "external software" for a big part, and that they are *part* of Mac OS X and cannot be removed easily.
Conclusion: a pointless (and extremely long-winded) article full of Microsoft bashing, as reply to an equally pointless article full of Apple bashing.
And I never will again. He's as bad as the people he is criticizing, if not worse. He does exactly what he accuses the "Microsoft shills" of doing. From another article on the site: "I explained that he could just drag the application to the trash, and that in the Mac OS there are no DLL files to worry about."
Riiight. Mac OS doesn't have libraries. There are no possible library mismatch issues on Mac OS. Okay, buddy, whatever.
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
Reporting of exploits and reporting of security vulnerabilities requires full disclosure and some people who still believe in 'security through obscurity' as a viable model will never disclose these things.
This is my sig. There are many like it but this one is mine.
...is to construct a real-world test and repeat it fairly often, then tally up how each OS performs. Create a monthly or bimonthly hacking "tourney" with a money purse to properly motivate the contestants. Get "normal" IT staff (i.e. not experts hand-picked by MS or the OSS community) to "secure" the competing operating systems, then let the hackers loose.
Unfortunately this only gauges vulnerability to remote exploits, which probably aren't the most common means of penetration and which both systems probably do pretty well at preventing.
Even if the information about vulnerability counts were pristine, it still wouldn't be useful, and anyone who has been involved in security knows it.
Over the years, there's nearly one flaw in the methodology for every one of these surveys ever released:
* Counting vulnerabilities in services installed by default the same as a service that is optional and not frequently enabled
* Subjective rating of impact (mild/severe)
* Treating remote code execution the same when on one system it is as uid nobody, and on the other, it is as administrator
* Ignoring the ease of use of tools that can actually verify a system's integrity (e.g., tripwire with signatures on RO media
and booted off CD)
* Ignoring what a user may have to do to trigger a vulnerability (ie, visit a web page with a malicious image, vs downloading a dmg file, running an install, and giving your password to elevate to root)
* Ignoring how an operating system enables or discourages user stupidity (ie, hordes of useless, "This program wants to do something, yes/no?" vs rare requests for a password)
And on and on and on. The average PC has over 25 different pieces of Malware installed. I know dozens of people with macs, and I don't know anyone who has had a single piece of malware, ever. I've been running linux for 12 years, desktop and server, and I've had two compromises ever, and both were via wu-ftpd.
See here for a brief recap of Ou's idiocy (not a word but still).
Silly socialists, trix are for kids.
Defective by design! Defective by design!Defective by design!Defective by design!Defective by design!Defective by design!Defective by design!Defective by design!Defective by design!Defective by design!Defective by design!Defective by design!Defective by design!Defective by design!
STOP IT STOP IT It's not that clever! It's a play off of the old saying "Deficient by Design" -- and that referred to UNIX!
Roughly Drafted != News Source
It's the most whiny flame blog on earth- stop punishing slashdot readers with it.
You make the same mistake a lot of so-called socialist make. You think that equality and fairness is for your followers, who are all inferior to you. If you considered them your equals, you wouldn't be commanding them. It's an interesting choice of title for someone who's supposed to be for the body of the people.
So 1% of Microsoft machines are not being used for botnets?
I must be exceedingly lucky cause I have a few Windows boxes and they aren't part of any botnet. I did have one that got owned pretty bad this year, but it's now running Suse while I figure out if I want to fix the Windows partition (yeah, it was that bad).
Largely agree with you, but...
Treating remote code execution the same when on one system it is as uid nobody, and on the other, it is as administrator
Local security does need to be considered, but it shouldn't be depended on. A remote code execution vulnerability is still critical, whether it happens as LOCALYSTEM, root, Administrator, local user, nobody, or in a partial sandbox like a chrooted environment or Microsoft's new sandbox in Vista. Local privilege elevation attacks to exist, and even without privileged access a remote code exploit can launch secondary attacks, log user actions in the compromised application (eg passwords), or run a payload that doesn't require privileged access (eg, a botnet node).
At one point, I looked over all the Secunia advisories about OS X and came across one which said that OS X would send passwords in clear text without warning when logging into Appleshare volumes and that this vulnerability was "unpatched". I thought this was strange since I had, in fact, seen such warning dialog boxes in OS X. It was in an unusual case where I was connecting from OS X 10.2 to an old 68k Mac running MacOS 8.1. I also remembered seeing that there is an options button when you make an Appleshare connection. If you hit that options button, you get a screen with check boxes for allowing clear text passwords and warning when a clear text password is needed. The default is to allow with a warning. I sent email to Secunia asking for clarification about what circumstances would lead to sending a clear text password without notice. Do those check boxes not actually work? Are the defaults less secure in some cases? I never got a reply but the issue disappeared from the Secunia site. No explanation. Just gone. I wonder if enough other issues have just disappeared to affect the numerology.
It'd be nice if some international body examined the issue of software security risks and established a guideline so we didn't have this ongoing problem of what to call a bug and what not to, and to finally put to bed the notion that notifying users of newly discovered vunlerabilities is bad for security.
I for one would like to see a rating scale that factors in not just the problem, the severity, and the scope, but also the availability of information on the problem. For example, you couldn't score anywhere near a perfect 10 even if the problem was minor and affected very few people, if you failed to tell anyone about it until you had it patched. Failing to disclose a known problem until someone else blew the whistle on you (or released an exploit into the wild) should earn you an automatic zero on the attempt.
Like most other mistakes, in the end security flaws almost always become magnified if you try to hide them.
Either somewhere in their statistical models they have determined that snow white publicity combined with a large number of your customers getting zinged costs them less in the end than fessing up and protecting their customers better. Or they're just being stupid about it.
But I think they're just being stupid about it. Wasn't there a quote something along the lines of "never attribute to malace that which can be adequately explained by stupidity"?
I work for the Department of Redundancy Department.
Of course the reporting methods are flawed. If they report that there is ANYTHING wrong with Teh Lunix or OSX, we know they are just trying to pump up the price of all the stock Microsoft has paid them off with.
Because, as anyone here knows, OSX and Teh Lunix are perfect and flawless creations. Touched by the hand of SteveJob, or Teh Lunis himself.
So obviously, since Vista/IE7 have far, far fewer flaws than OSX or Teh Lunis, we can know just by that result there is a flaw in the reporting methodology. Our FUD is never wrong, and our FUD tells us that Microsoft is buggy and has to force people to use their software. Any evidence to the contrary must be denied without even being looked at, lest we taint our worldview with doubt.
The funny thing is that you think the Mac way of doing things is somehow new or different, and therefore the problems can't happen on a Mac, when all you've done is described the way everyone has been doing things for decades.
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton