Slashdot Mirror
The Lessons of Software Monoculture
digitalsurgeon writes "SD Times has a story by Jeff Duntemann where he explains the 'Software monoculture' and why Microsoft's products are known for security problems. Like many Microsoft enthusiasts he claims that it's the popularity and market share of Microsoft's products that are responsible, and he notes that the problem is largely with C/C++ and mostly because of the buffer overflow problems."
I thought that's why Microsoft was pushing for "managed code" with the .Net framework. Though I think it's some what ripping the idea(s) from Sun's Java. But I'm sure even with .Net, there will still be buffer overflows. Well...the GDI+ exploit is one prime example of that fact.
Any compiled language is susceptible to security holes. The problem is that the process of turning source code into binary code is opaque to the developer. He puts some code through the compiler and some binary object code pops out. Things like memory offsets, code areas, data areas, and all these esoteric issues that need to be dealt with are simply left to the compiler to decide.
Unlike interpreted languages which for the most part implement all code as either line-by-line interpretation or in bytecode form, compiled languages talk directly to the CPU. Interpreted environments have the additional benefit that they run inside of a sandbox that is abstracted from the hardware by some large degree. Because of this, the running code never actually touches the CPU directly.
Things like the "no-execute" bit on modern CPUs provide an additional layer of security and prevent purposely damaged code from running directly on the CPU. However, until operating systems implement this in their own code, any application that does not want to adhere to the no-exec flag does not have to. This is like flock on Unix which only sets a file locking flag which applications are expected to obey rather than true file locking as implemented on other systems.
Then any apps that make requests through this kernel get the security benefits.
You shall always have evolution on a certain scale. Maybe you may "revolutionize" a single program but you cannot rewrite an operating system from scratch (meaning not even borrowing existing code and libraries as OpenBSD did heavily, such as many libraries, gcc, binutils etc). If you do, it will take years to "mature" which is also a kind of evolution.
On a somewhat larger scale, within companies you may replace one box with another (running another OS), but you cannot change your complete infrastructure overnight, i.e. replace all network protocols at once. Such changes take years and are a slow and evolutionary process.
Sometimes you have to take a step back and throw away some old cruft and make it fresh and new. However a certain degree of both evolution and also monoculture is unavoidable. If you have a 10000 employee company throwing in new technologies all the time, allowing for too much heterogenity, you shall have a maintenance and system-management nightmare very soon, leading to collapse of your IT infrastructure.
I'm not convinced this man, Microsoft or anyone else for that matter knows why they have the problems they do. If they did, I'm sure Microsoft would be very interested in obtaining this information so they could make higher quality software.
My guess is, and since I do not work at Microsoft or know their culture first hand, is they are a bloated, over managed institution that provides a fertile breeding ground for errors to compound. It's like NASA in some respects, where you just have too many layers of accountability which allows many things to slip through the cracks.
I'm not sure it's fair to blame the programming languages used for errors. Bad code is often proclaimed as a major short coming of C++, but in the end it comes down to the design, programming and process. Many very large and successful software projects have been constructed using C/C++, so I find it a lame excuse to blame the language.
One big problem that many agree on is in the case of Microsoft there is a large market pressure to release things before they are ready. This allows you to get your product out to customers who will then be less likely to use a computers product, even if superior, but released later. Everyone knows the price of bug fixes goes up after the software is released, but I'm sure the mathematicians at companies like Microsoft have calculated the bug cost to profit ratio in releasing the software in particular states and the most profitable option is taken, regardless of acceptance.
I would be interested in knowing what Microsoft's error to lines of code ratio is. Larger than typical, smaller? I mean, Microsoft apparently has really good talent working for them. You would imagine they would produce really good software. What gives?
"If you are a dreamer, a wisher, a liar, A hope-er, a pray-er, a magic bean buyer
After further review, the main reason they have error and exploit prone software is because they can. They don't care because it does not effect their sales at all. I'm sure they would prefer better systems, but really it is not a priority. The real priority is stomping out competitors with their monopolistic strategy.
This is the very reason we have antitrust laws. Face it, Microsoft doesn't have to make quality software because they own everything. They won't strive to make error resistant software until they are forced to.
The best part is, everyone has to pay for this through tech departments, maintenance, paid updates and general time spent fixing things as well as data loss, etc,..except Microsoft.
"If you are a dreamer, a wisher, a liar, A hope-er, a pray-er, a magic bean buyer
"A poor workman blames his tools"
Sorry, but IIS doesn't have a smaller market share. Considering that a server is vulnerable, not a host, there are more servers that run IIS than run Apache according to a (dated, but probably still relatively accurate) Netcraft study.
IIS runs on about 50% of the physical servers out there.
Further, IIS can be run as a non adminstrator as well, and defaults to this configuration in IIS6, which, btw, has only had 1 moderate vulnerability in it's > 1 year on the market.
If you need web hosting, you could do worse than here
The article says that IE is exploited so often because it is so popular. If Mozilla were as popular as IE, would it be just as often exploited?
It would not.
There are several reasons, but the biggest one is that Microsoft added some major features without ever considering the security implications. IE can install software on your system; this means you can use IE to implement Windows Update, which is kind of cool, but it also means that an exploit can use IE to put worms and viruses on your system. Firefox and the other web browsers do not have special permission from the OS to install things. In short, Microsoft spent a great deal of time and effort to tangle IE into the system, and that means that compromising IE compromises the system.
Microsoft was well served, for years, by a focus on features. Word 2.0 could be Word 1.0 plus a hundred new features; no need to redesign, just paste the features on top. As long as the applications ran on unconnected computers, this wasn't particularly a problem. Then as networking became more important, they still got away with it because a corporate intranet is still a pretty tame environment.
But now Microsoft software is out in the wild and wooly Internet and it isn't pretty. Features that were harmless or even useful in a private corporate intranet became big problems: apps that auto-execute scripts; the "Windows popup" service; remote execution; file sharing; dozens to hundreds of features, little and big, that were pasted on without any worrying about security.
Microsoft employs tens of thousands of smart people. They will improve their software, eventually. They need to start designing security in, and they need to give their developers and testers time to get the security really right, rather than trying to patch all the holes after release.
P.S. I think that another reason the free software is usually better designed falls out from the fact that free software is usually the work of small teams. Microsoft can write big specs and then have large teams go to work on them; if the teams aren't careful, their work can be a tangled mess. The free software projects tend to have clean, modular interfaces; this is partly because so often different pieces are coded up by people who don't even know each other. Also, the free software community values good design and good code, while Microsoft values features developed and shipped on time. (Good design and good code help the features to work and to ship on time, but for Microsoft the shipping is what is important.)
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
Jeff argues that monocultures are inevitable. This is not true. There are mechanisms in place in this country for breaking up monopolies. These techniquest have been used for nearly 100 years, since the presidency of Theodore Rosevelt. Using them in the past has brought great benefit to both consumers and businesses. In particular, one could break up Microsoft by splitting the application group from the OS group. One could furthermore split the browser out of the OS. Much of Jeff's argument depends on the false premise that monocultures in the computer world are inevitable, and can't be avoided. Take this point out, and the article has a very different flavor.
I'm not a contributor to Mozilla so I really can't answer that question fully. If someone reading this is, maybe they can give a more accurate explanation of why rather than my generalizations. But anyways, here goes... My reply is: the bugs aren't very similar, there are fewer, and they are handled better. I find it suprising that whenever a new Mozilla security issues is discovered, a certain % of Slashdot members always suggests the same argument as this article: it's the same crap as IE, but it has less exposure. I'm not going to use the Apache vs IIS example, I assume you've read it on the way down to this comment. Have you ever written code? Mozilla is software written by humans. Complicated software will always produce unexpected interactions of components. Bugs are inevitable. BUT: 1. How many bugs? 2. How long does it take to fix the bug? Look at the turnaround time. 3. Are you sure the bugs are fixed? Consider transparency. Were you kept in the dark? Did you not have access to the bug database? Were you not able to see first-hand the diff for the fix and verify independently that it was fixed soundly? 4. What type of bug? Is there a buffer overflow issue every two damn weeks? Most of the Mozilla security issues I've seen are high-level issues (XUL, tabbed browsing, shell://). It's not that the coder didn't pay enough attention or didn't dedicate himself body mind and soul to writing secure code, it's that these exploits are just clever ways of messing up software that most developers wouldn't think of. I believe these types of bugs are inevitable. Careful re-design could minimize them. Basically, I'm trying to say that bugs are a fact of life in software development whether closed or open source. Half-assed focus on security isn't.
Better compilers have a role to play. There was a great deal of work done on Ada compilers to be more intelligent about generating code for error checks. This greatly reduces the speed penalty for safe code.
Mea navis aericumbens anguillis abundat
2. Microsoft's Marketing - Yes, Microsoft are now a victim of their own lies as a result of convincing the public at large that their products are easy to use & maintain and that PCs never go wrong.
As an example, I just built a PC for a friend of mine who has never really worried about computers until it became apparent that her son needed Internet access to do some of his school homework. The sheer amount of information overload I had to give her was just frightening - update and run rhe virus checker regularly, update Windows regularly, update spyware programs regularly, don't use insecure passwords, don't duplicate passwords across different applications, etc. I ended up typing out 3 pages of hints and tips for her in the end.
3. User ignorance and greed - This follows on from 2. because far too many people have fallen for the Microsoft hype and have no clear understanding of how to keep themselves secure when on the Internet. Add to this that everyone wants something for nothing and the result is a whole heap of ignoramuses file-sharing all manner of nasty programs purely because they want their free music.
I don't care what anyone says but this will never happen with Linux. Linux will never be a mono-culture because the fact is that installing and using Linux automatically creates a learning curve meaning that anyone who uses it immediately starts becoming a more knowledgeable computer user. Sure, it takes a long time to become an expert but when you do, it is relatively easy to maintain a system to only run the services you need and to keep those updated. That's why viruses will never spread through a Linux user base because no two Linux machines are every entirely alike and because Linux users don't suffer from the same ignorance that plagues the Windows community.
I, for one, welcome it. I do not want inexperienced users flocking to use Linux purely because of the cool factor. The fact is that moving from Windows to Linux is like changing from being a child to an adult - the first step is to accept that you are responsible for your actions, not anyone else.
Gentoo Linux - another day, another USE flag.
What I find lacking in this whole discussion is that nobody seems to run their code through lint or splint.
I think that would narrow down a whole lot of errors.
If X is the new Y, and Y is "X is the new Y", solve for X.
I'm open minded to malice explanations too. Or malice out of stupidity.
In this particular case, doubly so. It fits just nicely with another "feature" of theirs, namely the ability of a user to erase all traces of themselves. You see, in the old system, users and records were never deleted, they were just flagged as deactivated. In their system, deleting a user, did just that: deleted the record. And because of foreign keys, it cascaded through all tables and erased _all_ records related in any way. Suddenly that user... has never existed, never posted anything and generally was figment of everyone else's imagination.
However... in this case if they weren't utterly incompetent monkeys, they were damn good at faking it. There were a ton of other bugs and issues for which it's hard to find a malice-based explanation.
Stuff like that they couldn't parse the phone numbers in the existing database, so they literally requested that some bugger edits a few hundreds of thousands of records per hand to match their format. (And if you think requesting that is idiotic, let's just say a PHB actually approved it. How idiotic is that?)
Try as I might, I'd be hard pressed to imagine what devilishly clever advantage they'd get out of that. What clever back door can be gained by having one poor soul go manually through that mountain of data? It's just stupid.
Or whereas the system they were replacing ran comfortably on one PC in Tomcat, their architecture needed a _ludicrious_ server farm to support the same load. Think: literally dozens of computers. Took many hours just to start or stop the behemoth.
Again, I'm not sure what devilishly clever 0wnage would result from writing piss-poor unperformant code. Were they planning to DDOS it into submission later? Not much point in that, when you can already make yourself the highest ranking admin and cause more trouble.
So on the whole, dunno, I have no trouble whatsoever assuming that they were, simply put, too stupid for those exploits to have been planned. In fact so stupid that if they had tried to code a back door, it would have probably been too buggy to work.
A polar bear is a cartesian bear after a coordinate transform.
Why for example is it a GOOD idea for AVAST's real time scanner to tell me it found a virus and then not doing anything about it? It knows it's there, kill the damn thing. Don't give me a message popup from the system tray telling me you found it.
Two words for you: False positives.
It's bad enough when an AV scanner accidently triggers and displays a message about a valid program. It would really drive people nuts if it kept immediately deleting valid programs as soon as they were installed...
...and here's what I have to say about it:
...) or anything as trivial as that can be without bugs, adding complexity doesn't mean adding bugs along with it. It has been shown time and time again that many exploit possibilities are visible in the source code simply because the writers are using unsafe coding practices (think gets();). With those facts in mind, it is conceptually possible to write bug-free, exploit-resistant code. The fact that the author of the article states otherwise doesn't make it true. The fact that the author states otherwise is an attempt to convince the reading public that we should expect and accept bugs rather than strive to a loftier goal. (There will be dirt in the world, so let's all live in filth happily... there will be disease in the world, so forget about prevention.)
The writer's attitude about software is simply all wrong and too tollerant.
Statements like "all software has bugs" is utterly ridiculous! There is software out there whose claim to fame is being bug-free/exploit free and continually strive to keep it that way. (Think Qmail and the same group's DNS solution.) Further, if it's possible that a program that can acquire a user's input and then print it all over the screen (think back to the says of BASIC: INPUT "Enter your name", A$
And I don't think that blaming the programming language is the right answer either. C/C++ are not inherently insecure languages. Can secure and safe code be written in these languages? Hell yeah. That would be like saying the French are rude because they speak French. Ridiculous. What is the author's intent in writing this article? I have to wonder...
"...software monoculture isn't good but it isn't bad... it's just the way it is so accept it. It's the programming language's fault not the company or the people who use it..." Can these messages possibly be true?
Some say Windows has so many security vulnerabilities due to its popularity in the market.
Others say bollocks to this and wheel out their poster-child, Apache, to refute this.
I say that it is a combination of three things, in this order:
1. Hatred of MS by so many programmers, network administrators, script kiddies, etc.
This is #1. Don't fool yourself into thinking otherwise. There are people who literally spend all their free time searching for ways to damage MS's software and/or reputation. (I know a few of them personally, as I'm sure you do.) There are companies who's biggest aspiration is to be the subject of an AP story regarding some new MS flaw that they discovered. Did you know there are even websites out there that are totally consumed with hating MS? They insert editorial comments in every MS story, good or bad, and don't treat security vulnerabilites or similar bad news about other platforms the same way.
2. Popularity.
I can show you a dozen programs that are trivially compromised. No one cares to though; because no one is using those programs. Rant about it if you will, but we all know that Popularity plays an important role here. e.g. Script kiddies want to control the most systems of the other kiddies on IRC. Most of the other kiddies are using Windows. You do the math.
3. MS not valuing security above features and/or rapid upgrades.
Arguably, this naive outlook has now changed with MS's security initiatives, the success of XP SP2, etc.
I beleive that all 3 of these things work in tandem to produce the sorry state of MS software security that we have today. (improving though, ever so slowly).
Its interesting to see that people still cling to the dead idea that you can design perfection up front if you spend enough time doing it, even though there is very little evidence to support this and mountains of evidence against it.
He assumes that if everyone runs away from IE (or MS in general), that they will all be running to the same thing, and create another monoculture.
The idea is to NOT have a monoculture. To have an environment where there are *many* competing platforms and applications, and (becuase they all comply with certain published standards for file and network data formats) any individual (person or business) can choose from among many alternatives, and still be able to communicate with and send and receive data to/from individuals who chose something else.
*Eliminate* the monoculture. The solution is not for MS to fix its bugs. The solution is for MS to not have a monopoly (wether they jump or are pushed)
MS biggest fear is that once they are a monopoly, they will lose *all* market share (and to be honest, IMNSHO, rightfully so, their stuff is crap). MS is desperately afraid of having to compete on the technical merits of their products. Its something they never had to do before.
C# would not have happened without the legal difficulties of Java. Without the legal difficulties, Microsoft would either _own_ Java, or have _destroyed_ Java. Embrace, Extend, Extinguish.