Slashdot Mirror
Shirky: Given Enough Eyeballs, Are Features Shallow?
cshirky writes "A persistent criticism of open source is that it is more about copying features than creating new ones. While this criticism is overblown, the literature of open source is richer on the subject of debugging than design. I've written an article about Ben Hammersley's LazyWeb.org, wondering whether open source methods plus RSS distribution can do for feature requests what open source already does for bug fixes, namely parallelize the problem in ways not available to closed source development methods."
Open Source development, as practiced by many Linux programmers, is not about pleasing the masses; it is about scratching an individual itch. As the ratio of users (ie. non-developers) to developers grows, there are bound to be more itches left unscratched.
The reward of scratching your *own* itch is obvious. The reward of scratching other peoples' itches, especially when they are not likely to even send you a "thank you", are more dubious.
I do not mean this as a criticism, simply as an observation of how it really works. Open Source works because of individual need. You cannot expect it to do the same kind of thing as paid development (which also tends to include nasty, unpleasant bits).
We can help a little bit by giving non-developers the ability to participate in a meaningful way. Think translations, documentation, icons, loading screens, project supports, web support, whatever. A lot of this is in place today of course, but it may not be obvious to many. We may need to work on that - both making these things more obvious and 'sharing the fame' with the non-developers who help out in a meaningful way.
The criticism that Open Source is always copying is highly unfair. It is unfair because Open Source is in many ways still catching up to closed source. Many of the tools are growing stronger, but are still lagging a little bit behind. Once that gap is filled I expect the Open Source community will boldly go on to create new and innovative features. Until then... Well, there are only so many hours in a day.
We should always watch out for complacency. Something is not 'good enough' just because it works. We should instead take pride in making OUR source the very best possible. Paraphrasing Apple for a moment, we really *do* need to 'think different'. A copy of Windows is not likely to be 'best'. A copy of UNIX is not likely to be 'best' either.
If we want to evolve further, a point will come where we need to help other people onto our platform. These other people are not users perse, but rather creators of content like us. Unlike us, they create music, pictures, video, etc. We need to provide them with tools to do what *they* do best, and we need to educate them towards our mindset (Open), and they in turn will reward us by sharing *their* work.
Although this is not scratching a personal itch, the job can still be rewarding because these are generally nice and interesting people to work with.
Right, off the soapbox I go...
I've been thinking about this for a while. One of the things that is so powerful about Open Source is that it pools the abilities of so many people. Unfortunately, the mechanisms currently in place in the Open Source world are great for pooling the skills, observations and ideas of programmers and geeks, but they're not good at all at pooling them from end users.
What i'd like to see is a kind of buzilla application completely focused on feedback from end-users. Somewhere where end users could make observations about OSS applications, and perhaps other users could vote or comment on them.
I brought a digital camera the other day. Before buying it, I could go to several web sites (for instance, amazon and cnet) and see lots of comments from other people who had used it. It was really useful to me, but of course could also be of use to the manufacturers themselves to get raw comments from end users. Open source software needs something like this so that developers can see first had what people like and dislike about their software.
... but not necessarily in areas suits would like. It is worth it to remember that Clay Shirky (who submitted this article) is a well-known VC, so from his viewpoint it is understandable that Python would not seem very innovative.
:-))
Open source uber-successes (innovation + usability): Apache, Sendmail, Perl, Python, PHP, emacs, vim (vim adds sufficiently to vi to justify it being innovative imho)
Open source successes (usability): Nautilus, Gnome, KDE, Evolution, the Linux kernel, GPG, glibc, Mozilla, OpenSSL, OpenSSH
Open source failures*: Directory Servers, Calendaring/Groupware servers, Office software, desktop publishing tools, graphics/prepress tools (the Gimp isn't a prepress tool), message queueing systems, heavy duty databases (despite SAP/DB).
I see a pattern here: Open source does pretty well at stock protocols that fulfil community/individual needs, it has even done reasonably well at end-user desktops (Nautilus being the crowing example -- if only the rest of the Linux desktop was that good!
Where we have not done well is about stuff that solves suits' needs: directory servers and groupware being a classic example.
I think we'll need some initiative from the industry now to fill these gaps, because it is not obvious that the community is going to scratch those itches anytime soon. Sun's open-sourcing StarOffice was great, OpenOffice has a chance of catching up with MSOffice in ~2 years. I sometimes wonder what would happen if IBM were to walk the talk and open up *any* of the following: DB/2, Domino+Notes, SmartSuite.
* yes, I am aware of OpenLDAP and OpenOffice, thank you.
Excellently put. I agree completely.
On the other hand I also feel there are a few Open Source projects that, after years struggling meet the current state of the art in that particular type of application, have finally caught up with it and even surpassed it. A shining example of this would be Mozilla. For years it was a huge pile of bloat, barely useable. Now it is a stable and relatively fast browser, feature-complete by even the most rigid modern standards (at least on Windows, for linux there are still a few crucial plugins missing) and provides features that i sorely miss when, for one reason or the other, I end up using IE.
The point I'm trying to make is that much is dependent on maturity. Maturity of the code that makes up the application, but also of the team that develops it and of the community of users as a whole. And I have the distinct feeling that this maturity has been rapidly increasing that last few months.
Open Source development models do not prevent innovation. They merely provide unusual challenges. Only now are we starting to learn how to deal with those challenges effectively, especially in the arena of desktop/user software.
Pathman, Free (as in GPL) 3D Pac Man
They sit there, with their lightweight psychology, graphics design, philosophy, women's studies, physical education, etc degrees, and contemplate and measure cost/benefits of how the software should work. When appropriate, they poll users. They perform usability tests on actual users. They monitor how actual users use existing versions of the software in order to both spot common errors and figure out exactly what features are actually being used. They document what they do. They read books and articles on nominally the flimsiest of things, like "the psychological implications of color choice in menu design." Their feature requests sometimes border on the really difficult to implement, and sometimes quite simple.
Theirs is a full-time job. Even, if in your arrogance, you don't believe that it takes particular skill, at least grant them that it takes time to go and set up actual usability testing sessions and so forth.
The implication is that PROGRAMMERS DON'T DO DESIGN (at least, interface / features design). Or, at least, anything a programmer might do is reviewed and analyzed.
Needless to say, this is totally different than in oss/linux, where programers are really the only actors in the whole software development cycle. "design" is accomplished largely by copying other products, whcih inevitably leads to a lack of appreciation for the subtleties that make up for good interface and usability design.
Gimp and Xfig - my two favorite whipping boys, are examples number one and two of programs that nominally have the features, but in practice are painful to use compared to their closed-source equivalents (Photoshop and Visio).
The problem is as well is that there is no plausible way to get designers and similar 'soft skill' people into the OSS cycle today:
- culturally, the OSS 3l337 reject anybody without super-skills. don't even pretend that this isnt true.
- Technically, there are no mechanisms in place for this. CVS is about code. The development model is essentially about continuous 'patching' of the software rather than grand rearchitecting, which design considerations often require.
- economically, there's little hope of getting quality designers involved. Programmers barely get recognition in OSS (blowing to hell ESR's naive theories, btw). Who would care who designed what? How do you get street cred as a designer? I mean, it could happen, but it would take a pretty big mental shift.
Design = customer focus. OSS too often has this not. Profit drive causes customer focus. Alas.That's odd. I started programming in the 70's and a lot of what we're working on now is exactly what we were dreaming of then. If anything, we'd have been pretty shocked and horrified to know that we'd still be working on them after <heavenly choir>the Year 2000<heavenly choir> (which once seemed as distant and hallowed, yet imminent and all-conquering as any Messiah or deity)
We cursed the limitations of our hardware and software then, but we worked with them because they were all we had. We'd spend weeks trying to trim a kilobyte or a few cycles out of a loop, not because we were virtuous (we weren't) but because we had to. Those whirring disks ran 24/7 to do crude payroll jobs that would run minutes on a desktop today.
It's been well documented, here and elsewhere, that most of our routine computing doesn't really save us much time, if any, but is simply chasing that elusive 0.1% "better output" (documents, biz data, presentations, etc.) A high school frosh today would be ashamed to hand in a report that looked like the most painstakingly prepared reports prepared for President Nixon. The addition of kerning hasn't vastly improved the content of a high school report.
Among the early computers I worked on was a triple CDC Cyber7600, which (in its initial configuration as a mere dual Cyber6600) exceeded the total computing capacity of the Soviet Union, but had a fraction of the CPU power, storage, etc. of my laptop so where's my accurate voice transcription, much less my intelligent editing and automated personal research assistant? Text recognition is just about usable for some major applications, but handwriting recognition is still barely usable
Yes, many are happy with their PDA hand-print inputs (or whatever) but it's a singing pig, we've learned not to expect Pavarotti; we're amazed it's workable at all. In ten years, we *may* achieve our 1975 dreams, and call the current state 'crap', not because of added features, but because 95% success (if one can even achieve it) *is* crap performance that we'd never accept in other daily technologies. Contrast this with, say word processing: load a PDA with 1979's AppleWriter ][ word processor, (a 190K package) and *truly competent* speech- , text- or handwriting recognition (pick one) and you'll have something that'll fly off the shelf at a kilobuck a pop, and make the cover of Time.
I don't know your age, but I assume you recall the 70's. If so, you'll recognize that these functions that we expected "any year now" back then (and are still waiting for) are just a handfull of hundreds of basic functions that are clunking along today. Our standards have dropped. We haven't even implemented the feature set we were promised then
With the Web (hypertext), wireless networking, huge storage, and a thousand other technologies, we're finally getting close to the Dynabook, which in 1974, we were promised "by 1984 at the latest". In 1994, a dynabook would still have been a total world-changing breakthrough. in 2004, it may finally be here. Added features are nothing compared to competent basic functionality. We've had programs that claim to do all this stuff for 25 years, but we're still waiting for solid human-free performance (which is largely the point of having a computer do the job) on our 1975 dreams
Before you say creeping featurism pulled us out of the Dark Ages, just give me reliable, competent worry-free versions of the stuff that was outlined in the magazines in the 70's -- software that was being written on mainframes, knowing the PC and laptop were coming, someday, and is still only in what we'd have call "commercially released beta" form 30 years later
See Von Hippel's papers here.
Enjoy,
Frank Ruscica
Adding features is easy. But I think often less is more. As an example, I compare Apple's iTunes software with MusicMatch Jukebox. I have both and I much prefer Apple's offering.
When you compare what's under the menus, MusicMatch looks like a mess. In comparison, iTunes seems clean and well designed. I think the ratio of useful features to features follows the 80-20 rule.
It's probably also the reason that I have stuck with Palm handhelds (actually a Handera) when the PocketPC's seemingly have much more to offer.
It is very difficult to make something simpler without losing any essential functionality. And of course what is essential is very subjective. But in the case of iTunes, I think Apple has done a very good job.
I've done as you suggest multiple times to no avail. The request is either not "sexy" enough or somehow not important enough. For instance, THE reason that openoffice/staroffice will NOT replace M$ Word or Wordperfect in academia/science is a total lack of ability to handle references and citations. Word and Wordperfect do this beautifully and professionally (good enough for submission and publication in professional journals) through their designed ability to work well with 3rd party apps like EndNote. I have made the request/statement to Stardivision, to Sun when they took it over (twice!), and to Openoffice.org several times over the years to no avail. Thus, these packages are mere toys for use by people who either plagerize (failing to cite references) or people who don't do serious research/writing and thus don't need to give attribution or support their ideas/claims. These tools are for letter writers and memo passers, not college students or researchers or professors.
I have also asked several groups to consider moving away from the should-be-extinct Motif interface to a modern, user friendly, SYSTEM friendly widget set (QT or Gtk). Motif IS UGLY, CLUNKY, AND NEEDS TO DIE. It has NO respect for screen size, insisting of shooting beyond your window borders such that you lose the ability (in some cases) of manipulating buttons and menus. If something happens such that you move the app interface beyond your window borders, you're screwed because it will happily reside in no mans land beyond the reach or your mouse cursor. Simple requests like this (FRIENDLY and REASONED requests, not rants) are ignored or quickly blown off. Why? Because developers don't understand nor care about endusers and app usability. As long as THEY can use the app just fine, tough shit for anyone else. My way or the highway.
This is the area that OSS cannot compete with commercial. In commercial software development, it REALLY DOES MATTER what the interface looks like (psychology is important) and principles of interface design are known and adhered to. They don't just willy-nilly toss together an interface and call it good enough.
In Bushworld, they struggle to keep church and state separate in Iraq as they increasingly merge the two in America.
The worst you'll do is just ignore me. I've been waiting two years for one additional IDE drive to be added to the "quirks" list in pdc202xx.c of the Linux kernel. The architecture of the file has changed, so someone's looking at it, but no one has bothered to listen to my very simple request.
The work is done. My request was to have a specific line added; I sent the actual line, my reasoning, and my configuration to both the individual maintainer and the appropriate list. I verified that the patch works before submitting it. This would be 30 seconds of work for the maintainer, with very little risk to anything else.
I never received any response at all after multiple submissions from multiple accounts. Nothing.
This has prevented me from really going with Linux for two years. My options are:
- Add the line myself and recompile the kernel, after getting Linux up and running without the disk in question (in essence, spend 3+ hours installing, only to end up with a non-standard configuration (which some software refuses to install on)).
- Stop using my Ultra66 controller, losing that performance gain.
- Dump the problematic disk, and use a different one.
None of these options are attractive. I want to be able to just install Linux on my computer, as is, from a CD, without hassle. I still cannot do that, and no one is interested.This is my experience with the open source "process".
Good Software is not about more features! Good Software is about doing it safely, reliably and securely. Good software is about doing it well, not doing more.
Adding more features will only make the software worse. More bloat, less easy to understand and use, needs more hardware, and the documentation usually lags behind as well.
Writing software is an art form. It is an exercise in restraint and thinking before you do it, not in gluttony or adding more crap to already crappy software. The world is full of bad software with hundreds of little understood and under-documented features. I'd rather have small, well-documented and reliable software, thank you very much.
We ALSO need to maintain the Unix philosphy of SMALL TOOLS THAT DO ONE THING WELL that you can combine - and perhaps develop a GUI paradigm for that, not throw the kitchen sink into every package.
Note prior to my comment - I am an "end user" (for the most part) not a developer. As such my comment may be ignorant about how things are working behind the scenes, whats possible with the existing techniques (but not actually showing up in the software...) etc. etc. etc. Anyway, on with the comment...
Amen! It seems once you start using a GUI there is a tendancy to build monolithic programs that do *everything* themselves even though all the other programs also do many of those same things. But I don't see why this has to be that way. It seems you could have some standards that organise and identify independent components (plug-ins?) by function. Perhpas something like mime types that has both a general and specific classification - like "text/spelling" for a spell checker/dictionary or "text/search" for a gui for grep etc.) Application developers would figure out which classes of plug-ins make sense to load into their program and where the particular classes and sub-classes show up in their gui (rather than putting them all under "services" like OS X).
From an end-users perspective it is *bad* that there are slightly different tools with slightly different capablities doing much the same thing from app to app to app. Every word processor or text editor needs/has a "search" function - why don't they all use the same one implemented in the same way and why can't that way be determined by me, the user, by installing the search utility of my choice that would be used by every program that needs one. For that matter lots of applications need text-editting (or word-processing, or image editing etc.) themselves - why don't they all use the same full featured application as a component rather than having their own lame version of that functionality or forcing me to launch another application; do what I need to do; copy or save the result and import/paste it into where I need it. If you really did this right there would even really be distinct "apps" you'd open a document and the components for any type(s) of data in that doc would load so you could do whatever needed to be done with whatever was in the document.
It never became a reality (at least for the end user) but it seems that Apple was working on something like this with OpenDoc.
You know, one thing I've noticed is that many people tend to have a erroneous perception of the complexity of their jobs. I mean, after working (or studying) in a given field for a couple of years, you start forgetting how much you know and understand it. Many tasks seem utterly moronic to you even though a very bright specialist in another field would be completely helpless in front of them.
/. are proficient in CS/CE. Many threads on /. are totally obscure to me. Sometimes, I don't even know what is the subject. Yet, I'm often appalled by the complete lack of understanding of financial basics of the /. crowd. One example? The RIAA/P2P "debate". How many times have I read "reasonable" arguments like "A CD costs $1 to burn and the majors charge 20 bucks. They are overpricing!!!"? Yet a quick look on yahoo! finance will provide you with most majors' annual reports and you'll see that their operational profit (whoops, I do it myself, do you know what operational profit is?) represent around 5% of their sales. Does this mean that a CD costs $19 to be brought to the store? No, of course. First, the 20 bucks is not the major's revenue it's the retailer's revenue, and he keeps a margin. Then, there's sales tax in most states. Then there are fixed costs, investments that are not proportionnal to the number of CDs you sell. Then there's price elasticity: volume increases if prices decrease. Then there's competition etc... Sounds complex? It's not. For me, it's like 6th grade common sense. Yet, many educated and clever people (though unfamiliar with accounting/finance) suck at that.
The reasons are many. Language: read any doc in any field and you'll see countless acronyms, names, references that an outsider has no chance to understand. Mindset/methodology: in each field, there's a "best practice" way to present problems and solve them; and you get (grok) it only through experience. Information: every field (not only IT) is evolving quickly and most people don't know where to look for useful info; in many cases, they don't even realize that information is available.
This holds true in every field, not only scientific or technical fields. It's true in finance, sales, marketing, journalism, politics, you name it. That's the reason why it's so hard to make different kind of people to work together (like R&D and marketing). Very few people are able to reduce the level of previous expertise needed to (really) understand what they are talking about.
One illustration: My field is finance, i guess most people on
It's the same with programming. I want to code something, what should I do? Wow, lots of questions come to my mind. Which langage, which platform, which IDE, which compiler, which database? How do I use any of these things? What do these words really mean in the first place? What is their syntax? Should I write the whole thing or use an existing GUI? Which one? Does this question even make sense? I'm confident that I could do it eventually if I commit enough time. It's not worth it!!! It's far more efficient if I outsource that task to a tech-savvy person.
To conclude: No, everybody is not a developer. And even in the future, most people won't ever do something that you would call developping. The problem goes far beyond GUIs and user-friendliness. You just grossly under-estimate the amount of investment needed to develop even the simplest piece of code.
It would be nice to be sure of anything the way some people are of everything.