Slashdot Mirror
Why Vista Had To Be Rebuilt From Scratch
iliketrash writes "The Wall Street Journal has a long front-page article describing how Jim Allchin approached Bill Gates in July, 2004, with the news that then-Longhorn, now-Vista, was 'so complex that its writers would never be able to make it run properly.' Also, the article says, 'Throughout its history, Microsoft had let thousands of programmers each produce their own piece of computer code, then stitched it together into one sprawling program. Now, Mr. Allchin argued, the jig was up. Microsoft needed to start over.' And start over they did. The article is astonishing for its frank comments from the principles, including Allchin and Gates, as well as for its description of Microsoft's cowboy spaghetti code culture."
To quote
Microsoft's greatest enemies now are still two for-profit companies - Google and Apple. I'll rest easier when FOSS replaces them (as was promised in 1999). Instead it's just a new master instead of the old one.Quidquid latine dictum sit, altum videtur
It's interesting to hear how their software development survived in such an anarchistic environment - everyone producing their own code, with ad-hoc integration. It's a good example of how software development methodology can work though, even though the specifics of the specification design weren't discussed in the article - if everyone codes to a documented interface, software development can work on such a grand scale.
I personally would like to hear more about the software development procedures and methodologies used in other large projects - how successful different types of development are.
I work for an automotive parts manufacturer, and to see the lack of consistency within the organisation's software development is disturbing. Safety-critical parts are being produced, and the level of testing between said parts varies quite considerably. Additionally, the level of oversight and adherence to software development procedures is rather bad to say the least. I just hope it's not characteristic of the industry as a whole.
Why is it "astonishing" that the article does a decent job of providing hard-hitting information without spin? That's what we are supposed to expect of journalists. The Wall Street Journal is supposed to be (and often is) an example of real journalism. That makes it distinct from computer magazines that rely on advertising revenue from the computer industry, and from discussion forums whose course is steered by peeves and submission sequencing.
When I took C programming in College, one of the points our professors made was if you like your program, rewrite it...
;)
the first time you write something, it's always hackney'd - and it gets that way till you figure out what you want to do and how to do it - afterwards, it then becomes so much clearer to see ways to clean up the code and fix issues...
so one of the first rules he had was once we were almost done, restart our stuff - it ended up being a lot cleaner/modular the 2nd time around...
of course, that won't help MS, but good for the rest of ya to know
RB
----------
ah honey, we're all resplendent - Bill Mallonee
Just look at this quote:
They are comparing an operating system, which has to be backward compatible with a dozen or so earlier versions of Windows and DOS and support an oodle of devices and subsystems, with a bunch of mostly unrelated web-applications and gimmicks from Google.
All I'm getting from the article is that the "let's rewrite from scratch" crowd got the upper hand within Microsoft. But that doesn't necessarily mean that they are right or that the end result will be better than continuous improvements. At the beginning, it is easy to maintain a nice, clean and simple system. But a complex set of requirements can't always be broken down into simple Legolike blocks, as the article suggests.
Right, but have you ever noticed how many successful Free / Open Source software projects use modular architecture? Take (from my own area) Nessus, or Snort. Both consist of a core engine and frameworks that accept plug-ins and modules. Actually they both also have a lower level that allows ordinary non-programmer users to contribute signatures (rules) to the project.) This applies also to Apache, Mozilla, the Linux kernel, and plenty more.
Of course, you are right: Microsoft is indeed one of the most competently managed companies around. And that is exactly their problem.
Why is that a problem? Because their management, sales, and marketing are so good that their technology doesn't have to be. They can ship software with security holes, bugs, poor usability, and bad design, but the non-technical part of the company will somehow manage to still sell it and make a bundle on it.
Throughout its history, Microsoft had let thousands of programmers each produce their own piece of computer code, then stitched it together into one sprawling program.
Sounds like SOP for any massive program/OS. If you've ever been part of a truly massive product's development, you'd know what this is like. There are dozens, if not hundreds, of small groups that each specialize in a particular piece of functionality. Executives and architects determine the work items for a particular release. Responsibilities filter down the chain of command. Teams develop their work items for the release and everything is thrown together into the pot as it's done. Builds break frequently, and problems are addressed as they're encountered. Eventually testers can get their hands on decent builds, and testing/bug fixing commences during the whole process. Some ways down the road, a release finally occurs.
Really, I don't know what the executive in the article thinks should be happening. There really isn't any other way to develop programs on the scale of Windows without the aforementioned "organized chaos". It's not a text editor, it takes numerous small teams working in a coordinated manner to produce such massive piles of code. Obviously, the more teams there are, the harder perfect coordination is to achieve. Hence, things go wrong fairly frequently. This is to be expected, IMO.
You sound like somebody who hasn't used Linux in a long time. In fact, it's amazing how far Linux has come in the last few years.
You've obviously never heard of Synaptic. I suggest you take a look at some of the screenshots. Most distributions now come with Synaptic. To install software, you just load up Synaptic, select the programmes you want to install from a list and click a big "Install" button. What could be simpler?
You seem to have a hard time grasping this but this is actually simpler and better than Windows. Windows has no dependancy tracking. I can't count the number of times I tried to install game X and the installer has told me that before I install, I need to first manually install the latest version of Internet Explorer / Windows Media Player / DirectX.
With Linux, all my programmes are on something equivalent to Windows Update. Not just the OS but also Office Suites, Games, Media Players... you name it. I can install them easily using a graphical interface and they get upgraded automatically when new versions come out.
As for driver support, Linux beats Windows out of the box, hands down. Drivers for most devices come already included with your distribution. They get loaded at boot time if that piece of hardware is detected. On my desktop, my DVB card, Sound Card, Graphics Card and Display were all detected correctly first time. Windows might have a driver for the Sound Card but a DVB Card?
Installation is so easy too. My distro of choice, Ubuntu, all you have to do is select your keyboard layout and where you want to install to and it does the rest. No intervention necessary. If you can't do that, there's something terribly wrong with you.
There is, as I'm sure you already know, a difference between a C program you wrote in class and an OS. The reason your C program gets better when you rewrite it is because you now have a clear view of what it should look and work like. When it comes to a behemoth like Windows, no one understands the system fully. So even if we have all these people who understand parts of the system rewriting their parts, plenty of design errors can still persist in the way the system is modularized and put together.
So what should they do then? I have no idea.
There is sometimes a difference between what a word really means and what a court defines a word as meaning in a specific context. In MS's case, a court convicted the company of having a monopoly within the context of anti-trust law. The Wall Street Journal is using the word as it is actually defined by real people, which means to own ALL of a market. The newspaper is properly labeling reality, not showing evidence of bias one way or another. The fact that I detest MS and Windows doesn't keep me from seeing that the WSJ is just doing its job properly in saying "near monopoly." The moment you don't have ANY choice other than Windows in the market, it will be a monopoly. For now, though, the fact that I'm typing this on a Mac and can go buy as many non-Windows computers as I want says MS does NOT have a monopoly. Period.
That might work for small college projects but the real world is a different place.
;)
Often the rewrite never gets completed as there is too much crap added to it.
If you truly want to make something that works you need to plan for an evolution of your software. That is, write the first version with a modular design that can be modified or rewritten in phases. Doing one big rewrite on a non-trivial software system is damn near impossible. It's better to evolve the software over time, always keeping a working system and slowing moving parts in the desired (presumably better) direction.
I could write more on this but it's too early in the morning and I'm not even sure if what I wrote makes sense.
The ratio of people to cake is too big
As much as you won't want to believe it, Vista is very similar to the transition from OS 9 to OS X. The big different, of course, is that Apple took someone elses kernel and tools and built on top of them. Apple also essentially forced developers to start writing their software for the new OS, with kind of shoddy emulation of OS 9 programs. The important thing here though is that OS X was pretty much a completely new platform. Vista is not quite as much of a change, but it's pretty damn close. Vista is introducing a whole new API system (WinFX), graphics api (Avalon/WPF), communications platform (Indigo/WCF), completely new audio stack, completely new network stack, and a few other major changes. All this while maintaining compatibility with 95-99% of current windows applications out there without a shitty emulation layer. Microsoft simply won't make a revolutionary OS anytime soon. There are too many people running Windows that simply won't stand for very little of their software running, if any, on a new version of Windows. So Microsoft is doing what I think is a good decision, they are making giant evolutionary steps towards a whole new platform. A transition.
Put my two cents in as to how the article's storyline doesn't quite track. If Mr. Allchin, despite massive institutional inertia, gave the pig winglets and put it back on a track to actually being releasable then we're missing the motive for why he'll leave on Vista D-Day and why the company wouldn't fight to keep him. In some sense, the article is about the story Microsoft wishes to tell, which is we were writing bad code, but we've fixed that now (and look at the bruises: no pain, no gain, right?), which is what the parent posts suspects.
Now I suspect that the interviews took place before the Microsofta est omnis divisa in partes tres announcement, and there was no desire from Microsoft to have Mr. Allchin candidly describe his reasons for retirement (and maybe Mr. Allchin has a book up his sleeve), so off to press with this peek into the hallowed halls of Redmond.
One quibble I would have with article is in its suggestion that Mr. Gates, as Chief Software Architect has two paradoxical duties to reconcile: coming up with innovations and putting down unrealistic projects. A lot of the candid reporting I've seen is that there's a third element that he practices with zeal, which is to grind into a fine powder any idea he believes shakes a stick at the cash cows.
One implication of the story is that in Summer 2004 Bill Gates didn't know that one of the cash cows was flatlining. There's a thought to ponder.
I love Google, because they rarely promise something and don't deliver. Actually, they rarely promise something. It just shows up one day and it's elegant, clean, and fast.
Hear, hear. MS holds flashy press conferences to announce products that won't ship for a year (if at all), includes laundy-lists of features that will be radically pared down before release, and ultimately ships products that are, at best, incremental improvements over previous versions, although they are touted as 'revolutionary', eg Win 2k vs Win XP.
Google doesn't talk about products in preparation. They quietly release full-function betas before announcing them, and the betas offer features that really are revolutionary. No Gmail wasn't the first web mailer, but it redefined what a web mail program was capable of. No Google didn't make the first map, but maps.google blows everyone else away.
Yes, there is a big difference between between building something like Google Desktop Search and building a whole new filesystem and all the other changes that requires. But the point is what is promised and what is delivered.
Google promises nothing, and delivers products that become essential. Microsoft promises the sky and moon (I thought Windows was supposed to be voice-controlled by now, and my fridge was supposed to automatically order milk when I need it), and delivers products whose importance to daily life is based primarily on the difficulty in avoiding them.
When Google does drop the next bomb (Google TV?, GoogleFS?, Googlix OS for running a smart terminal?), you won't hear about it in a press release. You'll be an invited Beta tester.
Looking under the hood, the Linux development model is more organised than one might expect. Consider the parts that make up a Linux system.
Libraries and applications are typically managed by smaller teams, and even if people contribute, those contributions are reviewed. Accepting that, we only have to look at the big structure. Some observations about libraries:
The whole is a mixture of bottom-up and top-down hierachical control. To understand the dynamics, consider an individual project. At an early stage, the developers looks around to identify what is already done, and tries to identify reasonably stable, common, and well managed libraries which they can use. This is a very feasible thing to do due to open source licenses. They will then start from there, and do occational changes in dependencies throughout the lifetime of the projests due to new needs and changes in availability and quality of dependent parts. Sometimes, libraries are split out of projects by abstracting out identifiable tasks.
An important observation is that by maintainers of a popular project casts a vote when choosing dependent projects. The more important the project is, the higher weighted is the vote for the dependent parts to survive. When most projects thus migrites to a better library, the rest will have to choose to follow suit or to risk loosing ground due to a more difficult installation process. The distributors are the ultimate judges, though their power is limited by what software is available.
In other words, there is a semi-democratic system that organises a hierachical structure of componets, with no single central authority.
You forget Apple, they reinvented themselves more than once AND always have managed to be the frontrunner of computer innovation...
According to their marketing and PR departments, anyway.
That ultimately gave us osX, the ultimate in plug-in philosophy, from the kernel to the GUI.
Apple didn't give us OS X. The kernel came from CMU (an open source project), and NeXT and Apple spent the last 20 years making it less modular. The GUI software architecture came from NeXT, borrowed heavily from Smalltalk, and is client-server, like X11, only not as well architected or as efficient.
In fact, Apple's own systems programming staff screwed up so badly that Apple had to go out and buy a new operating system; all their attempts to develop a next generation Macintosh OS in-house failed.
system7 with its core os and extensions attaching to it, they invented plug-ins before browsers were even invented...
The MacOS extention mechanism was nothing like "plug-ins" -- There was not a defined "extention API" as with a browser, they were they were system call traps and often relied totally on undocumented behavior.
Someone could write such extensions for any OS, but it's generally considered to be a bad practice. As the unstable, conflicting mess of MacOS extentions proved.
Whenever I hear the word 'Innovation', I reach for my pistol.