Slashdot Mirror
Free Software and the Innovators Dilema
If you wanted to assemble a "must read" list for any businessperson looking at the Linux/Free Software industry, what would you include? Certainly "Open Sources" from O'Reilly, is the most obvious answer, probably followed by Bob Young's upcoming "Under the Radar," which details the story of Red Hat's rise. But I would argue that a third book belong in the top tier of that list as well: "The Innovator's Dilemma," written by Clayton Christensen and published by Harvard Business School Press.
"The Innovator's Dilemma" traces the histories of various industries, from disk drives and microprocessors to steamships and automobiles, in which established market leaders have been beaten out by smaller, more nimble competitors. This idea, that startup firms have noticeable advantages over their larger rivals, has been one of the cornerstones of the internet era, not to mention Microsoft's antitrust defense, but Christensen is one of the few authors to actually address the specifics of these show-downs. In almost every case, he claims, his example established industry leaders were managed well, by conventional standards. They listened to their customer base and constantly sought to increase their penetration in high-margin, high-end markets. These seemingly-innocuous strategies become disastrous, however, when a "disruptive technology" enters the low end of the market. The established firms shy away from these new technologies to avoid undercutting their core, profitable businesses, but this ultimately leaves the market open for a new player to implement the disruptive technology, then slowly march up the food chain, overthrowing the old market leader. Minicomputer manufacturers in the 1980s, for instance, diligently followed their customers' demands to invest only in faster minis, while ignoring the PC market, which held little interest for the companies' established base of scientific and business customers. How many of those companies are still alive today?
This isn't, of course, a book review. Instead, I guess you could call it my attempt at a wake-up call to those software companies (you know who you are) who still think they can make a living on "business as usual" in the next millennium. Specifically, I want to focus on Linux, which might be the best example of a truly disruptive technology that we've seen since the advent of the internet. In fact, this theory gives us a guide to understand how established software firms risk missing the boat with respect to Linux, just as brick-and-mortar retailers were overtaken on the net by smaller, more daring startups. Consider three statements that a software vendor looking at Linux in 1999 might make:
- "That sounds like an interesting idea, but we asked our customers and they don't seem interested."
- "That sounds like an interesting idea, but the profit margin sounds too slim."
- "That sounds like an interesting idea, but it would eat into our more profitable core business."
Now take those same three statements and imagine them coming from an executive from an established retail vendor considering e-commerce in 1995. It's not much of a stretch, is it? In this case, though, we're hampered by our hindsight. We fail to appreciate that the executives who turned down a chance to take, say, Barnes & Noble to the web in 1995 were in fact making a very reasonable decision based on the traditional, financial bottom line. They would have spent millions to set up shop, made it easier for customers to price compare (and see that, in fact, the Barnes & Noble of 1995 was a fairly expensive bookstore); the customers who did buy online would need at least some discount to offset the cost of shipping; and the site, like Amazon, would have been a spectacular way to drive the parent company into the red. By not embracing the net, however, they opened the door to the B&N's greatest threat in decades, and they ultimately had to spend even more money to build play catch up against the $20 billion internet rival that wouldn't even have been created if the existing booksellers hadn't dropped the ball.
How have today's successful technology firms, then, fallen into the "innovator's dilemma" with respect to Linux? By focusing on the short run and the high end. When companies like Sun, SCO, and Microsoft dismiss the OS, they talk about its lack of scalability, its relative newness, and its lack of a journalling file system. These features, however, are irrelevant at the workstation and workgroup server level, and, more importantly, they're all being developed at an unbelievable pace to help the OS scale to enterprise levels. As commodity hardware becomes more and more powerful, while Linux and Windows NT continue to scale up to new heights, the traditional Unix vendors will find themselves increasingly marginalized to the very highest-end of the computing spectrum, falling into what I call the "Silicon Graphics trap."
Silicon Graphics (now SGI) was notorious for their focus on the high-end, sexy technologies: Cray supercomputers, 128-processor Origin servers, $10,000+ workstations, etc. While each unit sale at this level seems highly profitable, in reality the R&D costs involved in pushing the envelope of technologies at the microprocessor, OS, server, and applications levels was simply impossible to maintain, and the company spiraled deep into unprofitability as companies like Intergraph ate away at their core graphics business with low-cost graphics workstations. Now SGI has become the first traditional Unix company to truly embrace Linux, making it their platform of choice for the IA-64 architecture. Perhaps they were lucky to fall into the trap while they still had time to ride the first wave of the Linux movement.
But that leaves a question for the rest of the OS, hardware, and computer services worlds: are you willing to concede the entire low-to-mid-range server market to smaller, faster companies that position themselves as Linux early adopters? Do you want your future customers to think of you as a Linux pioneer that can accurately evaluate and deploy the OS, or as a second wave Johnny-come-lately that doesn't really understand the phenomenon at all?
Clayton Christensen's prognosis for these established players is unequivocal: unless they form new business units with the autonomy to embrace such disruptive technologies "the probability that they will survive as industry leaders is zero" (Bloomberg Personal Finance, October 1999). In my opinion, some of these market leaders need even more fundamental changes in their structures to address this disruption; they need thorough forward-looking reorganizations, such as SGI is undergoing with Linux and open source, as Microsoft restructured their product lines to face the internet, and as I advocated that SCO should change in my last article.
3M Corp always stood out; IIRC every division is expected to garner 35% of its income from products that did not even exist three years previously. Obviously this kind of organisational dynamic requires some funky organisational structures, and these structures might provide cover for those "intrapeneurs" who would eat away at the edges of the large organisation's older, more established, still-highly-profitable core businesses. Case in point: for the IBM PC, they sent everyone to Boca Raton and sheilded them from the rest of IBM.
I'd like to hear some "war stories" from people in the "skunk works" of large outfits, about battles fought to keep alive ideas that scared the hell out of other parts of the company.
When companies like Sun, SCO, and Microsoft dismiss the OS, they talk about its lack of scalability, its relative newness, and its lack of a journalling file system. These features, however, are irrelevant at the workstation and workgroup server level
What does this have to do with the "Innovator's Dilema"?
Once again, an opportunity to do some useful writing is wasted by ridiculous, tired, pro-linux jabs.
Just for once I'd like to see some interesting technology writing that doesn't mention linux. You guys are really flogging the dead horse beyond all reasonable limits.
Speaking as "the bloke from work".
Free Software is at its strongest when it takes an existing idea, say a C compiler, or a Unix-like OS, makes a lifesize copy, and then keeps adding features until it is as good as, or better than, the commercial competition. I'd hazard a guess that very, very few Free Software projects are doing anything but copying, cloning, or porting existing products.
So if you're looking to throw a few million into research and development, in order to turn your Cool New Idea for the Next Killer App into a reality, do you really want to have to be asking yourself "So... how long until a few bearded hippies come up with a free (speech/beer)knock-off?"
There's a lot of talk about how venture capitalists are wary of high-tech startups, because they fear that if the product is too successful, Microsoft will just clone it and give it away with Windows. The Open Source factor is just as dangerous. Any product, useful or not, highly profitable or just scraping by, is fair game if a few hackers decide they want to clone it.
That said, Microsoft's "Defend the right to innovate" line is crap. I mean, since when have they innovated anything?
Charles Miller
The more I learn about the Internet, the more amazed I am that it works at all.
"Larger corporations tend to move more slowly and carefully, they are unable (due to commitments to their shareholders and employees) to set up risks where they might take a bad fall."
The customer's needs and demands play a part in this as well. An ongoing enterprise needs some assurance of longevity and stability from its vendors. If I am building a worldwide information system, at a cost of (say) 5% of corporate revenues, which is a huge investment, that system has to last x years. x can vary but certainly not less than 5 for a Fortune 500 company. Therefore, I cannot afford to base this system on untried technology from a small startup. No matter how good they are, I can't take the risk.
This will lead me to use large, established, conservative vendors, and pay them big bucks. Digital, (the old) IBM, Control Data, Sperry, etc. But of course, these large and profitable sales are exactly what holds the vendor back from innovating in the long run (note that none of the vendors I list above really exists today).
Crucially, this also affects the type of people a vendor has to hire and promote to fulfill these contracts. If you want a stable, secure, high quality system you have to employ knowledgable, experienced, conservative, belt+suspenders+extra-string-inside-the-pants guys, who are probably somewhere around 40-60 years old. It makes me shudder to hear that Microsoft doesn't hire anyone older than 22 (per Bill Gates interveiw in Newsweek 2 years ago) - these are the people I will trust with my mission critical systems?
But at the same time the conservative guys by their nature will resist innovation (having seen numerous failed innovations in the past, no wonder), and eventally prevent the vendor from advancing with the "next wave".
No real answers here - just some observations. But I think my observation does go a long way toward explaining why we have seen a substantial decrease in the quality and reliability of software and computer systems in general over the last 10 years: reliability and stability are in direct conflict with speed and innovation.
sPh
Part of the trick is understanding the difference between your job and your business. Example is the collapse of the railway tycoons who forget their purpose was transporation and not constructing superfast trains. Companies that understand what fundamental role they play and can stick with the "last-mile" to the consumer will have a decent chance of surviving disruptions. Thus companies like Coca-cola have shifted from dominating the carbonated drinks segment, to being the market gorilla of the liquid refreshment sector, to the mindshare game (associating their taste with pleasant memories such as rock concerts). It is usually the less complicated systems that are the most robust to disruptions (something to keep in mind when designing software). Any complex manufactured capital good can usually be undercut by a lower cost substitute and any fad (whether toys/games/culture) will be difficult to sustain.
One can apply this trick to existing OpenSource vendors to ask what their role is. The clearest example I think of is:
Mandrake - bundling/packaging
MacMillen - distribution/catalog
LinuxCare - support/reassurance
Thus Mandrake could probably be more profitable customising/bundling specific packages for various predefined sectors (business, education, small business, etc), MacMillen on finding alternative distribution channels besides books (e.g. sponsor contests to find who can convert the largest number of boxes to Linux), and LinuxCare in demonstrating key metrics such as cost of repair/replacement, as well as alternative mechanisms of support (e.g. user groups to weed out basic computer problems unrelated to Linux). So long as each company understands the core role and what are bolt-on businesses, then they can adapt even if new technology comes along.
The disk manufacturers are in a bit of a bind since they compete with anything that can store bits ranging from punch-card to next-gen holographic crystals. At least the file system experts like Veritas are less vulnerable.
Technology is like a treadmill in Alice in Wonderland, you have to run as fast as you can just not to go backwards. Stick with coke if you don't like the guessing game.
LL
The phenomenon of small nimble companies growing under the radar of a large corporation market leader is not exclusive to IT industries.
FedEx grew in the late 70s despite UPS's dominance of the package delivery market because it's hub and spoke system could get more packages to the right places faster and cheaper. It was ignored for a long time by UPS as a small time, regional carrier. Eventually UPS got screwed(though they still have the lead in overall number of packages delivered annually) by its willful ignorance of potential competition.
On a larger scale, Japanese manufacturers were able to slip under the radar of their American counterparts, manufacturers of everything from TVs to steamshovels, by coming out with a cheaper product, being ignored by the market leader, and improving until their quality met or bested that of the former market leaders. (BTW: It is a process currently underway with Korean manufacturers, of everything from TVs to steamshovels, who are undercutting both their American and Japanese counterparts).
This even happens with countries. France, the market leader in wool production in the early 18th century, ignored Britain's increasing productivity due to early industrialization and lost its lead, which contributed in part to the economic stagnation that predicated the French Revolution. Western Europe has consistently underestimated Russia, and was caught by surprise when, under Leninism, it exploded economically and fully became a world power. The Arab world ignored Zionism when it was a small and powerless movement and did not realize what was happening until control(or market dominance of the political sphere, if you want to think of it like that) had shifted, at least for part of Palestine.
Now these last two examples bring up an important point: sometimes the overturning of the market leader by a previously ignored underdog is a good thing and sometimes it is not. As applied to the current situation, I would hazard that an overwhelming majority of /. readers think that a potential for Linux to overturn the current market of OSes would be a good thing, but, looking at other examples of similar phenomena, we ought to hold judgement on the benificence of all such overturns until the facts are in.
The best companies realize that it is better to make their products obsolete themselves, rather than waiting for their competitors to do it. There's a good example of this in the current issue of wired - Schwab aggressively moving into on-line trading despite the fact that is would cut into the more lucrative traditional market. Sure the stock price took a short term hit, but the resultant growth more than made up for it.
Speaking as "a bearded hippie." :)
"Free Software is at its strongest when it takes an existing idea, say a C compiler, or a Unix-like OS, makes a lifesize copy, and then keeps adding features until it is as good as, or better than, the commercial competition. I'd hazard a guess that very, very few Free Software projects are doing anything but copying, cloning, or porting existing products."
Open source isn't that good at overall design and architecture. It works best when it forms around an existing kernel of code. This allows religious flamewars to be quelled with the cry: "Show me the source." This is why most projects are working on something definite, most usually an existing body of code, or an application being cloned.
"So if you're looking to throw a few million into research and development, in order to turn your Cool New Idea for the Next Killer App into a reality, do you really want to have to be asking yourself "So... how long until a few bearded hippies come up with a free (speech/beer)knock-off?"
Figuring a modest $40/hr per developer, and 100 developers on an open source project, a few million amounts to roughly 750 hours, or five months of full-time work from each developer. Figuring half-time instead, call it a year -- not that long. This suggests that new ideas won't fail to happen just because the proprietary model is no longer profitable.
Open source software is indeed a threat to software that is packaged and sold on shelves. Fortunately the vast majority of software is not of this type. Nearly all of it is built to purpose to solve some company's need, or to control some hardware. These jobs are not threatened.
It is correct that what is being contemplated is a major change in the software industry and painful displacements will no doubt result. It is not a question of eliminating software as a profession however, merely software as a product. The question is whether you think the goal is worth the strife.
The thing you have to remember is "what is your core market?" A disruption in your core market is devistating, but a distruption in an ancilliary market isn't- in fact, it's often an opportunity.
Two examples: Sun and Linux, and SGI and PCs.
What killed SGI/Cray was _not_ Linux- it was cheap PCs. SGI's (and Cray's) core market was selling hardware to solve large linear algebra problems (almost all uses for super computers- from scientific simulations to weather predicting to Hollywood special effects- are at heart simply large matrix problems). These problems parallelize very easy, meaning it doesn't matter if the MFLOPS come in one big box, or a whole bunch of little boxes, what matters is the total cost for the necessary MFLOPS.
As such, it wasn't Linux that killed (is killing) SGI/Cray, it's the explosion of cheap powerful PCs and low-end Workstations, which make it possible to buy a few hundred PCs and wire together the same number of MFLOPS as a major super-computer, at a much lower cost.
Now, let's look at Sun. Like SGI/Cray, Sun's core buisness is selling hardware. The main purpose Solaris has in life is to give customers a good OS to run on the hardware Sun is selling. But, unlike SGI/Cray, Sun concentrated on selling Database machines. Unlike large linear algebra problems, databases don't parallelize so well- the various CPUs end up needing to communicate a lot more. It's easy to replace a Cray doing weather prediction with a stack of PCs, but extremely difficult if not impossible to replace an E10K running Oracle with the same stack of PCs.
If Sun can convince Linux users to run Linux on Sparcs, then Linux is no threat to Sun's core buisness (hardware). In fact, it's an opportunity to cut development costs of Solaris (allowing Sun to target Solaris at the high-end, where Linux isn't a good fit at the moment). If Linux were to grow to the point where it could replace Solaris at all levels, then Sun could drop the overhead of supporting Solaris altogether. While I don't think Solaris is a loss for Sun, it certain isn't a profit center. In either case, Sun doesn't lose in it's core market.
Now, a company whose core market _is_ expensive PC-based operating systems has a lot to lose to a disruption in that space. So it's important to look at what the core market is for the companies before panicing about a possible disruption in the low-end.
I also recommend Christensen's book; Linux has many of the characteristics of the disruptive technologies he discusses, and his insights certainly have prompted me to look at the industry in a new light. However, I don't believe the Innovator's Dilemma was the primary problem that led to SGI's failure. Christensen does a good job of characterizing the ``flight upmarket'' that established firms do in the face of disruptive innovations. Although SGI maintained a strong presence in the high end (the high-end graphics group was still showing significant profit, last I heard), for years it was pushing actively downmarket into the ``disruptive'' areas -- with NT-based systems, and more importantly, with projects like Nintendo 64. I suspect the fundamental problem with SGI was not that it failed to handle disruptive innovation, but that it failed to remain competitive within its core markets. Sun, in particular, avoided this problem. Arguably so did HP. But SGI was late with new machines in its traditional strongholds of CAD and content-creation, and when the new machines shipped they were underwhelming in terms of price or price/performance. Like most SGI veterans, I have opinions as to why this happened despite the fact that everyone from senior management to the lab technicians saw the market shifts occurring. But I'll skip that for the moment. The Innovator's Dilemma is an excellent work, and well worth reading, especially in the context of Linux. But be careful when applying it to any particular corporate failure; it's not the only reason technology companies stumble! Allen