Slashdot Mirror
WinInformant Says Windows More Secure Than Linux
nihilist_1137 excerpts from this WinInformant article, which reads in part: "For at least the first 8 months of 2001, open-source poster child Linux was far less secure than Windows, according to the reputable NTBugTraq, which is hosted by SecurityFocus, the leading provider of security information about the Internet. ... A look at the previous 5 years--for which the data is more complete--also shows that each year, Win2K and Windows NT had far fewer security vulnerabilities than Linux, despite the fact that Windows is deployed on a far wider basis than any version of Linux." I wonder how many sysadmins (Windows or Linux) would agree with this conclusion. Update: 02/04 16:54 GMT by T : Looks like the WinInfo site has gone down since the story was submitted, so you may have to content yourself in the meantime with the Bugtraq numbers. Update: 02/04 19:30 GMT by T :Several readers have pointed out that the conclusions WinInformant makes based on the Bugtraq data are not those of SecurityFocus; the headline has been changed accordingly.
Look, the obvious point about this should be that the reason Linux has more known vulnerabilities is that linux has always been very open about what is wrong with linux.
As for MS, I only have to point to the the major bug, that they knew about for weeks, but didn't let anybody know about!
Now Im not saying that linux is more secure (as much as i would like to) but the data and report based from it, just makes no sense, if you think about how vulnerabilties are and are not reported
Thanks for reading!
Sigs are dangerous coy things
Pure quantity of security holes really is not the most question. To me there are two factors:
1. How severe is the hole if exploited.
Are we talking a DOS, a root compromise, the ability to take over a domain controller. The effect of a compromise needs to be taken into account.
2. How easy to exploit is the whole.
Is it a theoretical exploit, or are there tools floating around? Can it be easily mitigated by a good firewall, or can viewing an email cause the problem.
These questions seem to me more important than pure quantity and should be taken into account when building a threat assesment of a system.
But it is possible to have a very secure Windows environent. No, it does not involve turning the box off ;^)
.SCR, etc) were banned long before I Love You came along.
Take this example: you have a highly competent NT/2K administrator (they do exist) and a pitiful *nix administrator. Which one is going to produce a more secure box? Any objective person would have to say the NT/2K guy would, because he knows his platform well enough to shore up vulnerabilities. Nimda, I Love You, and many other worms did not hit affect my company because we took security very seriously beforehand. Malicious attachments (.EXE,
Now, having played devil's advocate for a moment, let me say that if you have a tightly controlled *nix box with a competent admin and a focus on security, you can create a damn near impregnable system. The weaknesses then lie with the applications, not the OS, and that's something ALL vendors need to work on (you listening, Larry "Unbreakable" Ellison?)
In the end they will lay their freedom at our feet and say to us, Make us your slaves, but feed us. - Fyodor Dostoyevsky
You apparently didn't check out NTBugTraq. They simply added up vulnerabilities from different linux distros to come up with a high aggregate number. This is plain wrong because
1) If a package has a security issue, usually all distros announce the security bug. Thus, the bug gets counted multiple times.
2) Windows security bugs are all remote compromises, either email attachments, or remote roots. Over 90% of the linux security problems are local security issues.
As another poster noted, this is a very poorly researched article.
In the long term Linux will have progressively fewer bugs/vulnerabilities due to its open source nature. Look at the numbers on the same chart for NetBSD. There were 9 vulnerabilities found in 2001, and 42 found in Win 2K. 54 for RedHat and only 2 for TurboLinux.
Obviously everyone should switch to Turbo Linux.
Lasers Controlled Games!
Greetings,
:-
I wonder how they decided what is "more secure", but my guess is that it's based on the number of reported exploits/bugs.
Does anyone know if they used any weighting on the types of exploits/bugs. I would consider a remotely exploitable bug to be much worse than a locally exploitable bug as you can't control people that aren't on your box as well as the people that are. I would consider a root/administrator access bug to be worse than a denial of service type bug.
So, given a weighting scheme of
Remote Root = 4
Remote Denial of Service = 3
Local Root = 2
Local Denial of Service = 1
How would the different OSes stack up?
My guess is that without even taking number of installations into account you would find that Microsoft was at least as bad as the various Linux/Unix versions. I'm not going to say that they were worse.
Anyone want to do some analysis on the same information given a weighting scheme and see what the differences are?
Z.
-- Under/Overrated is meta-moderation, and therefore is Redundant.
1. How many of the Linux vulnerabilities are in services that aren't linux? IE: sendmail, apache, ftp servers, and whatnot? Just because something is packaged with linux doesn't make it linux. Do the windows bugs count IE bugs and every other MS software running on the system? What about other packaged software such as AOL and whatever other links they provide?
2. Sheer number of vulnerabilities mean nothing - are they counting the severity of the vulnerabilities?
3. Are they counting the time it took before A) someone discovered the vulnerability and B) a patch was issued?
4. If there are comparable numbers of linux vs. win2k servers out there, which actually had more break-ins? (This question not valid if there is a wide gap in numbers since then the lower of the two probably benefits from that "security through obscurity").
5. I think having full source code availability leads to people actually FINDING the bugs, whereas Windows could have way more, but we don't know about them unless people are actually TRYING to crack the system (as opposed to finding them working on source or whatever).
If Linux had the marketshare of Windows, you can bet there would be lots and lots of scriptkiddies writing Code-Red style worms. Linux has had some pretty major security flaws in the past. Although they were fixed quickly, that doesn't mean that lazy or incompetent sysadmins will patch it right up. This leads to an opportunity for a Code-Red style worm, and if Linux had high marketshare, you can bet that it would have spread rather quickly as well.
That's it. I'm no longer part of Team Sanity.
The security of any OS lies in the skill of its admin. An idiot with a 2k box is no more secure than an idiot with a linux box and vice versa.
- Toby
Again, Winformant, in a desperate attempt to seem like they aren't a bunch of toadies, has struck an "independent" blow against linux's "security myth," by proving that more holes were found in linux than in Windows.
Well, duh. Linux is full of holes. But that's not winformant's problem. You see, each of those holes was cleared up in a matter of days and a patch was freely available. There were no egos and press releases claiming there are no holes. There were no programmers waiting around while Marketing decided the best colour for the patch's installation wizard. There was no downtime as millions of machines had to get the file from a single MS server because the patch's license didn't allow redistribution. There were no hours of wringing hands as sysadmins watched hackers pick off their boxes one by one because there's no workaround while the patch was built. There was no possibility for diving into the code and fixing it yourself; and if there was there'd be no way to release the patched dll. Oh, and if a linux machine was compromised, there was little chance of it polluting the entire network...because the bug affected less than 1% of the install base of that particular OS, and not 100%.
Not to mention the reason that so many Linux patches were "found" rather than "discovered" is that bored sysadmins can sit around with sheets of source code, hoping to find a hole and make a name for themselves on BugTraq. With windows...well, you'd better be good with BlackIC and ASM, because it's the only way you're finding the hole.
Hey freaks: now you're ju
However, the conclusion being drawn here is invalid. The SecurityFocus vulnerability survey is interesting, but it is not itself a reasonable methodology to generate security metrics between operating systems.
I could pick nits at this ad hoc study for hours, but the biggest problems are also the most obvious:
First: the study associates third-party software with the operating system, and aggregates all the distributions together into a meaningless "Linux" category. This study is literally just pattern matching against advisories.
Second: there is no notion of "severity" or "impact" in the study. This is a shame, because SecurityFocus has actually put some real effort into deriving a taxonomy of vulnerabilities from their (enormous) vulnerability database. There is no way to determine whether the N Linux vulnerabilities were equivalent to the K NT vulnerabilities.
Third: the study compares a kit of open-source software, which has received extensive peer review, to a closed-source product. It should surprise nobody that Linux has more documented problems than Windows: it's actually possible to go find vulnerabilities on Linux. Finding Windows vulnerabilities requires black-box reverse engineering.
Finally, both Linux and Windows do a reasonable job of locking down server configurations out of the box. What IT people need to know is vulnerability breakdown by operating system and by deployed configuration. This study does nothing to inform us of whether a Linux web server is at more risk than a Windows web server, or whether it's safer to expose a Linux print server or a Windows print server. Organizations that deploy homogenous Apache+NFS+ssh server farms don't care about XFree vulnerabilities or Samba problems.
I don't think SecurityFocus is actually trying to make claims about the relative security of Linux and Windows. I think they've been a bit careless with this report though; it's a reasonable thing to try to generate from their database, but more thought should have gone into presentation.
SecurityFocus has the on-staff expertise to publish some real conclusions about the distribution of vulnerabilities between Linux and Windows. Before this database report is misconstrued by the trade press, it would be enormously helpful if they could publish a statement about the conclusions that can be legitimately drawn from it. It'd be good press for them, too.
Actually, there aren't SO MANY MORE windows servers on the internet than *nix boxes.
Please see this fine article http://slashdot.org/article.pl?sid=01/07/13/124025 7&mode=thread which tries to compare the number of windows systems vs unix systems on the internet.
Here are a couple of their conclusions:
Even taking the statistics most favorable to Microsoft, they had almost twice as many IPs on the public internet than Linux did in 1999. However, during that same period, there were many more than twice as many expoits, viruses, etc. that attacked windows vs unix.
Linux has far too many installations on the public internet to be dismissed as too rare to interest hackers.
Actually, IIS hasn't had a hole since last August and IIS 5.1 hasn't had one, period. XP has only had the UPnP hole (new technology, consider it a version 1.00 bug).
There are FAR fewer holes in W2K than people would like to admit. IE may have some problems but not the base OS. Even IIS has been tighted up a great deal.
People need to understand something, we know MS almost never get's it right the first time (see version 1.00 bug) and may not the second but eventually they do. OK, they sucked at security to begin but with all those resources and the pressure from the top and from outside - did you really think they'd sit still or get worse? Nope - ask Netscape what happens when you become their focus of attention. Tux comes out and smokes IIS 5 and everyone laughs... according to the results of my beta tests with IIS6, we'll see who's laughing when it's publically benched.
Your lesson is: MS learns. It's almost never right the first time but... it learns.
I can't connect to WinInformant, but if you look at the numbers available at SecurityFocus, you'll see that they did not simply add up the numbers. Linux is listet with 96 aggregated vulnerabilities for 2001, while e.g. Red Hat has 54, Debian got 28, and Mandrake got 36. There are more Linux distributions listed, but these numbers allone show that your claim is wrong (unless WinInformant has different numbers).
You'll also see that Red Hat had 54 vulnerabilities while Windows 2000 had only 42.
However, I'd still agree that the WinInformant article is badly researched (but please note that, as stated above, I've not read it, I only know the part that Slashdot quoted). The article claims that Windows is more secure "according to the reputable NTBugTraq," however, SecurityFocus does not make any claim concerning the security of either Windows or Linux, they just make the numbers available as a statistic. In other words, WinInformant doesn't have any source for their claims, they just found some more or less interesting numbers and made up a story.
Sig (appended to the end of comments I post, 54 chars)
Sigh...
I can't read the original article, It's been Slashdotted to death. But I think I can make a pretty good guess as to what happened.
First off, we host Bugtraq, not NTBugtraq, which is Russ Cooper's list. (Any chance we can get that fixed in the story intro? Anyone know if the same mistake is in the original article?)
Secondly, I'm constantly amazed at how people mis-read our stats page. The Linux aggregate stats are the total of all unique bugs across all the various distributions we track. It's supposed to answer the question "How many Linux-related bugs were there that year." It's based on things like which distro ships a particular package, and when that package is found to have a hole, it also gets attached to the distro. This is so you can look up your distro, and see what bugs you might need to patch.
Take a look at the top of the page, our script hasn't been running since August, when we switched from Roxen to Apache. So, we're missing the whole last quarter of 2001 stats.
Regardless of anything else, using these number to declare that one thing is more secure than another is a mistake. Based on our numbers, why didn't they declare that everyone should run MacOS for security? Or that if you want to be more secure, run Debian instead of Win2K?
l10n and ramen were two recent worms that attacked a bug in some versions of BIND on almost all unices. This would appear to be evidence against your theory that "no-one writes worms for *nix because of lack of market share".
Find another excuse.
--
E_NOSIG
Of course, that's not the case in the server market. If you want to talk about worms, remember one thing - the ONLY reason Code Red and other such worms exist is because of the popularity if the windows platform, on desktops and servers. Don't kid yourself for a second into thinking that the reason there aren't any widespread worms for *nix systems is because it's more secure.
Dont kid yourself. The various free o/s's are simply a harder target. They are more diverse, both across O/S's and distributions, and even within a distribution there are different configurations. On top of all that any individual box can be a totally custom system built from the source pool.
There are countless email readers, multiple web browsers, all types of competing server daemons. When you take the windows monoculture you simply dont find such diversity. The competing software are simply wiped out.
Its a well known and intuitive fact that monocultures are far more vulnerable to disease and parasites than a healthy diverse population.
Windows security holes typically have exploits in the field, whereas linux vulnerabilities are commonly realeased from code review- hence having no preexisting exploits (that are known and demonstrated). Some are in fact purely theoretical, and may have to use to a malicious user.
So even if you keep on top of your windows updates religiously, keep in mind that they are generally reactive. So there is always that window of vulnerability...
What I read was the original article before it went down by /.
So worry for the thing on Win9x/3.x + WinNT/2000.
So they are talking of Server OSes. So Win9x/3.x do not account as such.
What you say is that, of course, they do not include duplicates of the same vulnerability. But then there's no such program as rsync-2.07-3.i386.rpm on Debian 2.2 . Can you see it?
Also, why it is strangely coincidental de number of bugs for Red Hat Linux 6.2 for Alpha and Sparc? See:
For 2001, we see:
RedHat Linux 6.2 sparc - 18
RedHat Linux 6.2 alpha - 18
Debian Linux 2.2 sparc - 18
Debian Linux 2.2 arm - 18
Debian Linux 2.2 alpha - 18
Debian Linux 2.2 68k - 18
Coincidental? See it yourselves at SecurityFocus WebSite
Maybe is a cross-architechture bug? Will this mean that, in fact, it is the same bug?
Then the numbers for Mandrake, Red Hat and Debian are waaay too similar (2001) to be just a coincidence (Mandrake 7.1, Red Hat 7.0 and Debian 2.2 can be thought as "equal distributions" by means of timeline, packets versions and such):
RedHat Linux 7.0 - 28
MandrakeSoft Linux Mandrake 7.1 - 27
Debian Linux 2.2 - 26
Then, on 2001, we can assume that Red Hat 6.2, Mandrake 6.0 and 6.1 have the same package versions :
RedHat Linux 6.2 i386 - 20
MandrakeSoft Linux Mandrake 6.1 - 20
MandrakeSoft Linux Mandrake 6.0 - 20
And those numbers are also very very close to the ones for Red Hat Linux 6.2 on different architectures.
Maybe, just maybe... they are the same bugs?
Then, on previous years, the trend is the same.
With all the respects, I am no FUDing here. I post my comments to some piece of news that was flawled.
And I tried to explain why it was flawed. And I was vry carefull to not to blame conspiracy theories.
Then, again, I'm human. And I make mistakes. Like the Win0x/3.x and Win2000/NT of my previous post.
But this does not invalidate at all my message.
You mean, like this? The NTBugTraq site itself says (emphasis mine):
So, while there may be a stack of Outlook vulnerabilities, those won't get lumped in with Windows. But sendmail vulnerabilities might get lumped in with RedHat. They go on to say (emphasis theirs):
Further, the numbers themselves do not support the conjecture that Windows 2000/NT had fewer reported vulnerabilities reported over the 5-year period. Let's compare RedHat (the Linux distro for which the largest number of vulnerabilities was reported) vs. Windows 2000/NT from their data:
So even though the numbers are potentially skewed against Linux, the totals still come up less for RedHat than for Win2000/NT.
What the other article must be doing (I haven't read it yet, since I wasn't able load it) is totalling across all distributions, which is wrong. One FTPD vulnerability would get multiplied by all the vendors that ship that FTPD, which isn't quite fair.
--JoeProgram Intellivision!
The incompetence of the author writing this story, and of the Security Focus editorial staff for letting it through, is staggering. With this kind of security "expertise" is it any wonder at all that Nimda worms and the like run rampent across the net?
We didn't write the article in question, nor are we hosting, nor did we have any opportunity to see it ahead of time. (Or now... still can't see it.) Sadly, we have very little editorial control over other people's websites.
It gets worse than that. Let's consider:
Most bugs that show up for redhat or any other linux distribution will NOT affect a well-secured machine in the first place. If you plan, for example, a standard web or database server, you're only going to permit ssh and apache or ssh and your brand of sql. How many vulnerabilities in the past year have been on those services? Practically none. Only 1 in ssh, and there was AMPLE warning to get patched before exploits were in the wild. The majority of bugs are for packages not often deployed, or not relevent to a server system where there is no user access.
Meanwhile, an enormous number of these linux bugs are irrelevent on a firewalled system, never mind the incompetency of sysadmins. A firewall will protect your X font server or your installed-by-default nfsd/statd, but Microsoft has had many high-profile, extremely-widely-abused holes in a server's primary services (IIS, MS-SQL, etc).
Anyhow, trying to say these statistics show that NT is more secure than Linux is not only irresponsible but absurd.
They are looking at this from the wrong perspective. Instead of saying "Linux had more bugs than Windows in 2001" it should say "Linux *fixed* more bugs than Windows in 2001". Simply becuase those Windows bugs haven't been found yet does *NOT* mean tha they are not there waiting to be exploited (or are already being exploited).
"Your superior intellect is no match for our puny weapons!"
Another note from bugtraq that will really push the numbers in favor of Windows. I quote: "* There is a distinct difference in the way that vulnerabilities are counted for Microsoft Windows and other operating systems. For instance, applications for Linux and BSD are often grouped in as subcomponents with the operating systems that they are shipped with. For Windows, applications and subcomponents such as Explorer often have their own packages that are considered vulnerable or not vulnerable outside of Windows and therefore may not be included in the count. This may skew numbers." MAY?!?!? More likely WILL.
So let's see. IE vulnerabilities aren't counted. There goes the fairness in the numbers right there. Was IIS counted?
I posted a couple years ago on this topic. My hypothesis at the time was that Open Source would show more bugs for quite some time, as people poked through the code, but would gradually settle down and become very secure. I also believed that Windows vulnerabilities would continue to be discovered at a more or less constant rate.
The jury is still out.
The SecurityFocus statistics broke in August, 2001, per their web page, so one has to extrapolate the partial 2001 total to get the projected total for the year.
In that extrapolation, one sees that the expected number of bugs (assuming the 96 reported bugs cover through the end of August) would be 144. There were 153 the year prior, which is likely well within the margin of error. In addition, many of the black hats have STOPPED REVEALING their exploits, so in fact there may be many more than what we see.
Now, it's worth pointing out that this is not necessarily a good measurement of security. We may be measuring the wrong thing.
An example of bad measurement is the one the government used to determine how many cod were left out in the ocean, to prevent overfishing. Year after year, the catches were about the same, so the government assumed that the fish stocks were constant. But suddenly there were no more fish -- the industry collapsed.
Why? Because they were measuring the wrong thing. They weren't measuring the total number of fish, they were measuring the fish that were caught. They didn't realize, as the fish stocks dwindled rapidly, that the fishers were getting newer and better technology to fish with. The total number of fish coming out of the water was constant -- but as a fraction of the total fish in the water, was going up very quickly. Eventually the fish were all but wiped out.
Measuring security by bugs reported is very similar. It may or may not reflect the number of bugs in the 'ocean'. It is an indirect measurement at best.
We need to differentiate between fish 'caught' and fish 'available'. From a security perspective, I think we are talking about TRUE security (the number of fish in the water) versus FUNCTIONAL security (the number of fish actually being caught).
Now, as security people, our goal is to reduce the fish catch as much as possible. There's two ways to do this; we can reduce the number of fish, or we can somehow control, limit, or damage the profession of fishing.
The real professonals are trying to reduce the number of fish in the water. That's the true long-term solution. But from a short-term perspective, what I care about personally is how many fish are CAUGHT. Every time they come up with a new exploit, I have to run around like a maniac patching systems.
However, the fishing analogy starts to break down, as most do eventually. Truly secure systems are still run by people, and people make mistakes. Even if the OS is perfect, the attack will often come against the weakest link, the employees. Thus, even though I would prefer to have true security, I have to argue that it isn't really necessary. The OS just has to be stronger than the other avenues of attack. ("Why are you putting on tennis shoes? You can't outrun a bear!" "I don't have to. I just have to outrun you.")
Security through obscurity, in other words, may be adequate for most uses. It slows down the rate of fish catching. If nobody discovers the bug until the next version of the OS is out, the bug is less important. The longer it takes to discover the bug, in general, the less damage it will do -- at least as long as we're on the upgrade treadmill.
But, a counter-argument to that just occurred to me: Security through obscurity may be long-term counter-productive -- making it hard to catch fish may have the effect of increasing the fish supply. Every time a fish is caught, it can't breed, and reduces the total population by that much. Likewise, in code, once a vulnerability is discovered, many related vulnerabilities may also be patched. Thus, security through obscurity may work well for a long time, but may actually be making the fundamental problem worse.
Another observation I have to add is that programmers like to create new programs. Very few of them like to audit code. New projects and programs are being added to the Open Source world at an amazing speed, and I don't think they're being stringently audited. In other words, they're adding to the fish stocks every day. There is no QA department in Open Source, and the code is getting more complex than individual people can understand anymore. I think, unless we come up with a better development method, Microsoft's ability to fund a billion dollar a year QA department is likely to reduce their fish count below that of Open Source.
So I think I will need to expand on my original hypothesis. I now believe that Open Source will probably lag behind closed source in terms of FUNCTIONAL security. In terms of TRUE security (absolute number of exploitable bugs, known or unknown) -- there's no easy way to tell. If catching fish reduces the fish supply, and if the programmers don't add too many new fish, eventually Open Source will start winning. But if Microsoft's QA department does a good job with their nets and lures, their fish supply may drop just as fast or faster. Money is definitely a good way to motivate people, and Microsoft has a lot of it.
It's also worth pointing out that even if things are getting more secure, the catch rates may be roughly constant, because presumably the crackers will get better and better, catching a higher and higher percentage of the fish. If the analogy holds, and I suspect it may, then eventually the fish stocks will be exhausted and the black hats will be very suddenly unable to crack machines anymore.
It's going to take at least five more years to know -- and twenty might be a more reaonable time frame. It took a long time to wipe out all those billions of cod. It may take just as long to wipe out the pool of security flaws.
<<RON>>