Slashdot Mirror
Systems Research Is Dead?
Manoj writes "Rob Pike of Bell Labs (Yes, that [Rob Pike]) says that systems
software research is irrelevant. At a time when computing is almost the definition of innovation, research in both software and hardware at universities and much of industry is becoming insular, ossified, and irrelevant. Where is innovation? At Microsoft, mostly. Exercise: compare MS software in 1990 vs MS software today. He states that Microsoft has been working hard, and that on many (not all) dimensions, their products are superior technically. Linux is the hot new thing, but it is merely a copy of the same old stuff. And anyway, the exciting thing about Linux is the development model, and researchers contributed little to that. He states that the excitement, the art, of systems research is gone. "
Mr. Pike is just upset because his beloved innovative Plan 9 is being completely and totally eclipsed by Unix-like operating systems like Linux and FreeBSD.
He bemoans the fact that computer scientists still tend to use such crufty old tools as Unix, Emacs, and TeX, despite the fact that these tools are still very capable and that they grow more capable every day. The Novelty that Mr. Pike is so obsessed with is only useful when it constitutes an improvement over what is currently in use.
Perhaps the most absurd part of Mr. Pike's lament was his comments on Standards. He states that "With so much externally imposed structure, there is little slop left for novelty." Clearly, however this is both untrue and unfair. Mr. Pike could easily create his own little completely novel computer world. He undoubtedly has the talents. However, unless this new system allows the end user to do the things that they traditionally use their computer for what's the point? Why would I want an innovative new system that can't send email or surf the web (or run Emacs for that matter)? I wouldn't, no matter how novel it might be.
More importantly it is certainly possible to innovate within the standards that he complains about. Napster, Gnutella, and FreeNet are all innovative ways of sharing files, yet all of them rely on existing standards. Mr. Pike also totally belittles the success of scripting languages like Perl, Tcl, and Python. I certainly don't use C or C++ on my new development projects. These languages all represent innovation, but they probably aren't available for Plan 9, so they don't count.
If Plan 9 were in Linux's position you can bet that Mr. Pike would be singing a different tune, even if all Plan 9 were being used for was to run Emacs, TeX, and Netscape.
Here are some random features I want to see in an operating system.
- You turn on the computer and it works instantly.
- Whenever you feel like stopping you turn off the computer.
- You never have to save your files, you always work with up-to-date information
- You never have to drag or resize a window
- My mom can store her weaving project ideas on it without any help
- You never have to remember obscure names, everything is built out of a small set of simple blocks
- You never think about "connecting to the internet", you just work with data that happens to be located somewhere else
- The kernel continually decides how to configure itself for maximum efficiency for the task at hand
- There are never any library problems -- library specifications are written in a formal mathematical language, which can be verified against the library code
nojwFirst of all, MacOS X Server has been shipping for, I believe, a year now. If you regard Jobs as being Apple, then Jobs has been shipping preemptive multitasking since before OS/2 had a GUI (which is before Windows had preemptive multitasking too), and he's been working on bringing that back to Apple since '96.
Further, Apple hasn't delivered yet because they're more ambitious. They've gone through several attempts that simply didn't live up to their standards. If you want to look at places that innovate with backwards compatibility, look at Apple -- architecture change, no problem. Complete replacement of everything from the kernel to the userland, your old programs will still work. MacOS X is really innovative; it's more modern than UNIX, more stable than Windows NT or OS/2, and easier to use than anything.
--Matthew
Get off your stupid soapbox about amiga!
;-) /usr/dict/linux.words' will do!
If anyone should Be on a Soapbox bragging about innovative OSes, it should Be someone who's talking about a high-caliBer OS. Behold the one true Multimedia Befitting OS. Benchmarking shows that this OS Belongs among the Best OSes in performance. Of course, I'm speaking on Behalf of...
darn I forgot!
Of course, the company essentially took a briBe by ditching their desktop platform. Now we have to scriBe on a emBedded version if we even want to Begin to use this OS.
(Please don't laBel me a BeOS fan, I just thought this was funny
It's amazing what: 'grep be
My biggest beef is that he treats Unix (and Linux) as if they were a static entity that never changes, but nothing could be further from the truth. Linux (and most Unices) have embraced the results of all new operating systems research. In fact, most research is actually done using Unix as a base, even thoutgh the technologies are by no means Unix-specific. SMP, threads, IPv6, journaling and global filesystems, real-time extensions, just to name a few.
The point is that Unix is that it is so modular that you can rip out entire pieces of it and replace it with something better. After all, at the core, what really is unix? Do you define it as the V7 system call set? Or do you define it as the implementation by a specific vendor?
What really gets my goat is the statement comparing MS software in 1990 to 2000. That's trivial because MS completely ignored the fruits of the research of the previous 20 years. It wasn't until the advent of NT that they even began to try to do things right. But even then they blew it by integrating the GUI into the core of their OS. Grrr...
The statement of "twenty years ago, students would be exposed many operating systems each with good and bad points" is pretty much bunk for the same reason. Over the past 20 years, Unix has tried to ammend the bad points, and incorporate much of the best. Is this a bad thing?
Lambasting gcc and emacs in favour of Visual Studio? I'll stick with my current tools, thank-you-very-much! VS may look pretty, but what does it really do that I haven't been able to do with my current tools for the past 10 years? Ok, except lock me into a one very specific platform that I cannot customize. And what's this cor to do with operating system research in the first place?
Statements like Linux's [GUI] interface isn't even as good as Windows are what really get my goat. Rob, what were you thinking when you wrote this? Surely you know that the only interaction with Linux is by system calls to the kernel. What is the GUI? Well, in most cases that would be a glob of X/Windows, and a desktop package running on top of that. But gee, take a look... People are developing all sorts of new interface paradigms that do not use X or look anything like what is currently being used. And where do the run? Yes! Linux and other Unix platforms.
Yeah, sure, I'll agree that the number of OS research breakthroughs has been dwindling on the past decade, but that's the be expected. Much of the easy, obvious, and general stuff has been well researched and documented. What's left is the really hard esoteric stuff; the stuff that most people just do not have enough background to understand.
Maybe Unix (or Windows, or the presentation to the user of each) has become like the automobile of the transportation industy. Most automobiles are pretty similar. Mostly they have four wheels and are powered by an internal combustion engine. In fact, in most general respects, they are remarkably similar to the machines produced at the turn of the century. Does this mean that they have reached the pinnacle of design? Or maybe it just means that the concept has reached a fundamentally usable point.
Now, let's take the automobile analogy to the next level. Most people can learn to drive a car in about an hour. Most cars will run for years with little to no service. Can the same be said of computers? Of course not. Does the user really care that billions of dollars of research have gone into making the car 10% more efficient? Not really.
Are computer's as usable by the general population? Hardly. I'd make a strong case that what is under the hood is pretty much irrelevant at this point if most people can't figure out how to take the machine out of the garage.
And this statement applies equally well regardless of whether you prefer Windows or Linux or anything else.
-p.
(sorry for the ramble... too busy even to be writing this...)
Asking single line questions doesn't really expand anyones understanding of any subject.
To answer your question however:
Though I am not a "purist" as I make my living as a benevolent parasite (ie tech support/analyst) for Microsoft products, I really do enjoy an have a lot of respect for Linux.
Mr. Pike has some good points, but I don't think he is completely right in every respect. First off, stable operating systems are somewhat new (linux/freebsd being among them). For all its "innovation" I'd like to see windows 2000 still run after a hard drive crash (a friend just recently told me how his hard drive crashed but bsd kept going with the processes that were still in memory... my friends, THAT is innovation).
I would also say that the idea of customizability/infite choice in OS is an innovation that even microsoft is picking up. The fact that you can tweak almost everything about an OS is relatively new (thanks to the Open Source Model).
Sure, we have a long way to go, but I think the very open nature of Open Source/Free Software encourages even more research (especially in areas of application) instead of less. You have to consider that Linux (et al) have spent thier time getting the right things done first (like stability and versatility) and now are beginning to branch out.
Yes, GNOME/KDE may be copies of prexisting things, but the wonderful thing is that they have ability to be extended in a myriad of ways by anyone (hello, Eazel). These projects are perfect examples of people accomplishing feats that were previously dismissed. Now, with the education and experience gained from overcoming the hurdles to get these things running, they can now extend those capabilities as far as thier imaginations can take them. I, personally, can't wait to see the new developments in store for KDE and GNOME. I have a feeling that they are going to prove Mr. Pike wrong.
Whenever someone claims that a field is essentially complete, they are nearly always wrong. Consider what people believed in the year 1900: Physics was considered practically a completed science -- and that was right before the explosion of modern physics.
If you read the interviews from the gnome guys, they admit to copying what's good about other people's designs. They say that first we need to get a good free software base that is consistent with the current market, and then we can start to innovate from that. So the current GNOME efforts have been just that - copying from others to get a good free software system to innovate from.
Engineering and the Ultimate
The problem with a lot of CS programs is that they teach Computer Science. What does that mean? It means that they teach the big, complex problems of fundamental compuation and not the practical details of coding in modern languages. The Honest Truth of it is that you need both. There seems to be no middle ground right now between superficial programs that teach how to write a GUI in a certain language and leave you high and dry in the area of general knowledge of principles of programming and the theories that underly the field, and Big Science programs that work from the assumption that everyone is going to spend the rest of their life writing compilers, designing languages, and proving problems NP complete.
The program that I am in right now leans towards the latter, and there are some advantages to that. Yeah, they won't teach me Visual Basic, but frankly, there are all kinds of books out there to teach you visual basic. I learned C on my own from a book, I figure I can do it again with most other languages. True, I didn't really understand C until I started writing projects in it, but I think that is really the primary value of programs that teach language specific classes comes in. It isn't important to learn the concept of a particular language in order to know it; there are very few conceptual programming models. What makes you learn a language is using it, and playing with it. So frankly, I'd rather learnt he general principles behind various Things in the computer field (general ideas about computation and complexity, models and abstractions, network concepts etc.) and then learn what I need to know in the world on the fly than be locked into taking classes that teach me a language and not the underpinings of the field.
The problem that a lot of theory programs have, however, is that they do a poor job of making the study of theory relevant to the lives of students, and they let themselves get a little too out of date in their obsession with 'big ideas'. A widely used (and poorly written) book on Computational Theory written by a dean at my school makes it seem that the only reason to study theories of intractability is that someday, your boss might ask you to solve an NP-complete problem effeciently, and you want to be able to show that it won't work. Yeah, a lot of people I know lie awake at night worrying that their next web page design is going to be for a guy who wants a web page that solves the satisfiability problem. At the same time, my entire school has not a single class that acknowledges the existance of a graphical interface. Yes, opening and closing windows on a desktop is not a timeless model of computing, but it is the dominant paradigm for user interface at the moment, and has been for a while. There is one class on 'Human/Computer Interaction', but it deals with speach recognition and optical recognition, because those are more 'algorythmically interesting.'
Bottom line is only this: we need neither more trade schools, nor more ivory towers. Producing a lot of 'programmers' that only know how to throw up a webpage with blinking mouseovers or how to plug in values in VisBac 6.0 isn't going to advance the field very much. At the same time, where the new pressures and problems are is in the real world, the world of business and commerce. Despite the resurgance of fundamental number theory as a job skill in lite of renewed interest in cryptography, more theoreticians that don't know what the hell #include means isn't going to give much of a boost either. If anything though, I think that a lot of the imbalance right now lies in teh direction of 'Programmer in a Box/Get Rich Quick' programs that emphasize bare competancy in a single area or language as being all you need to be a succesful programmer. If only for depth of character, I think a little more is called for. In the long term, no single language or program can stay current; only the knowledge of the concepts and theories underlying the field can, but only if they seem relevant enough to keep people in the classes.
Microsoft actually gets kind of bad rap when it comes to innovation, and they really don't get pegged for where they really do screw up.
I would say that Microsoft's goals for their software are very ambitious. Backward compatibility is given a lot of priority, which is what leads to the instability of Win/98. However, the miracle of Win/98 is that it works at all, considering the huge amount of hardware variability that it needs to work with.
Give Microsoft credit for attempting to creating an object-oriented operation system, while trying to maintain compatibility with the past. If you look at the internals, there is a considerable amount of power in their object methods.
Now the bad part: Microsoft is very ambitious is what they are trying to do. The trouble is that they have been poor at actually pulling it off. The unreliability of Windows is directly tied to over-complexity of what they are trying to pull off. I think if Microsoft actually slowed down a little bit and put a little more thought into what they're doing, instead of just "growing" the software and ending up with a rat's nest, they would be a lot better off.
On the hand, an argument can be made that Microsoft has succeeded because they don't dry-dock their software forever. They get it out into the world ASAP so they can get maximal amount of feedback (somewhat similar to "release early, release often"). Granted, releases have often been late (ala Linux, I might point out), but they get into the hands of Beta testers very quickly. And no one listens to their customer base like Microsoft.
I detest working with Microsoft's APIs because there is clearly so little engineering thought put into them. However, it's undeniable that there is a lot of energy and enthusiasm put into them, if you know what I mean. Microsoft is a very ambitious company when it comes to trying to include advanced technology in their products, with a friendly face. Far more than Apple, I might point out, who has taken 16 years to give us preemptive multitasking (technically, they still haven't, of course).
--
Sometimes it's best to just let stupid people be stupid.
With all due respect to Mr. Pike, I think this is a case of old-school mentality. Like when your parents claim that Elvis was the last real music, and complain that that horrible stuff on the radio "isn't really music, it's just a bunch of noise".
Systems software, in the sense of low level API's for controlling disk drives or performing DNS queries, has been solved... it's down to boring implementation and tiny incremental improvements. And I say Hooray for that. Do you want to write that stuff?
But as far as I'm concerned, systems software encompasses everything that supports the top level application. When you look at it that way, there are all sorts of exciting things going on with application servers, Java, object models, and so on.
All of that stuff will be just as critical for tomorrow's applications as the "Read Sector" API.
So I say development of systems software hasn't stagnated or died, it's just moved into new areas and new challenges. Like modern music. And I think that's a good thing.
-b
If I wanted a sig I would have filled in that stupid box.
Microsoft hasn't been innovated since it invented the browser (oops!), no the office suite (oops again!), the GUI, not hardly, the file server, not that one either, ah, e-mail, no, transaction services, nope, ... Ah the Word processor, nope, the spreedsheet, not! That would be Visacalc. How about the presentation application , oh ya Harvard Graphics. Media streamer, no, Real did it first. The mouse, no they didn't do that either. the PDA? Not even! What about DOS? Oh yea, bought that one too didn't they. Is there even one core product on the Microsoft line that is an original Microsoft innovation? How about the flight simulator? Maybe they did that first. No I think the Airforce and Atari beat them to it. So, in exactly what way are they innovative besides finding new ways to bastardize someone elses innovative ideas? And before you go claiming NT is some fresh new innovation, you might what to dig around a bit. I seem to recall that it is actually built on chunks of old VMS leftovers.
"He who sacrifices beauty for efficiency gets what he deserves." - Bernard Mickey Wrangler a.k.a. the Woodpecker
I'll drink to that. Linux's supposed crowning achievement -- GNOME -- is very nice but deep down it's just a clone. It just looks prettier than what it's cloning. I got all excited about Evolution a few months ago; then I went and looked at it when Helix did the PR. I'm thinking, ``wow, look, Helix is porting Outlook to GNOME.''. Not exactly exciting or innovative.
I wondered in a thread back when the Beanie Awards were announced -- where's the category for best new thing in the open source world? All I see are reimplementations. Then again, I guess that's what GNU always did -- take existing stuff, rewrite it, and bloat it.
Firstly, what is a system? It is a definable entity, with a definable boundary.
Using this (loose but sufficient) definition, let us look at Linux and see how it fits in. First, of course, there's CODA. This allows you to have many computers as part of a single system, in a far more flexible way than, say, for NFS. How to get such a hybrid, networked system to work in any kind of efficient way? That's the research part.
Then, there's Beowulf. How to modify network drivers (and, if necessary, protocols) to create an efficient, high-performance distributed virtual machine? Again, sounds like research to me.
Is there more? Yes! Linux is the ideal platform to experiment with newer technologies, both where you have one physical machine and many processors, or where you have many machines networked. Because of the kernel's modular structure, you can add and remove different modules on-the-fly, testing ideas in a way that would be impossible under Windows.
Is there anything new that Linux has added, beyond the model? (Which it just borrowed from the FSF, anyway!) Yes! The aforementioned CODA is a good example. Hot-loading and hot-unloading of kernel modules is another. Linux' IPv6 stack was one of the very first for any OS, first appearing for the 2.0.20 kernel, as an extra patch.
Virtual consoles, one of the delights of many a Linux user, don't exist for DOS, Windows, or many flavours of Unix.
ReiserFS, a completely tree-based FS, is unique in it's architecture. No other FS works on that principle. Oh, did I mention it was for Linux?
I'm currently working on IGMPv3, for Linux, which will be one of the first v3 implementations for a mainstrean Unix. (I know of a few others, but not many. I doubt you'll see v3 supported in Solaris anytime in the next few weeks.)
What about other areas of Systems Research, though? Networking is dead, surely! Tell that to people working on robust Anycasting, PIMv2, queue shaping (such as CBQ, RED, ECN, RSVP, etc), any of the experimental IP protocols (IPv7 - IPv9, I think), robust MobileIP (even when ISPs are, themselves, mobile), etc.
Non-networked stuff? System Interfaces are a vital part of a system, and there's plenty of work going on there, with VR, zero-buton mice, eye-tracking, speech synthesis/recognition, OCR, etc.
Internally, there's work on automatic parallelizing of code, using Critical Path Analysis. Neural nets and genetic algorithms get a fair amount of attention, as do Artificial Life-forms. Then, there's always Quantum Computing, Optronics (such as the purely optical router), Chaos Computing (based on non-linear systems), low-temperature circuits (such as the Crusoe, which also deserves a mention as an intelligently adapting circuit, for it's ability to handle non-native instruction sets, natively), etc.
WRT kernels, we now have exokernels, microkernels, monolithic kernels and distributed kernels.
Now, you tell me that ExoPC is old-hat stuff!
Lastly, but by no means least, SETI@Home, distributed.net and Cosm are moving Systems Research forwards in a way that no other projects in the history of computing have even dreamed of. And this is supposed to be the end of the road?
(Only if "road" means a bumpy cart track, in a rickety wagon pushed by Ivory Tower professors and paid for by Big Businesses eager to NOT get results before their cash-cows were milked dry.)
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Again, I must go in my usual rant: is there any reason why we insist in still writing operating systems in low-level programming languages like C (or C++, which I insist on considering as a low-level language)? It may be that low-level languages are better suited for writing the low-level stuff like the bootstrap code and the immediate hardware drivers (but even then, I don't see why a high-level language couldn't be extended with the appropriate functions to do the task), but some things in modern operating systems are definitely archaic.
Why is it for example that we still maintain the memory/filesystem dichotomy? It is about as absurd as requiring that a programmer have to handle the mainboard-level cache by hand. Why is it that whenever any program wants to save data, it must painstakingly convert it into a binary representation? Why is it that subroutines and programs still have to be distinguished? Why is it so painful to have reflexivity (e.g. for user-mode Linux you have to recompile a whole new kernel, the one already in place is not reflexive / reentrant in that sense)? Why is it that for security measures we rely on hardware control (aka MMU's) rather than formal invariance proofs? Why is it that our processors are so big and bloated and distribution / (asymmetric) multiprocessoring / clustering is so far behind?
For more information about what a high-level operating system might be, I refer to the Hurd (which is high-level, but at the cost of an abstraction inversion, because it is still written in C; notice that the Hurd Really Runs), to Erlang, which deserves to be better known because it's really impressive, and to the all-ambitious Tunes project.
In other words, cars are not being developed to manuever in three independant dimensions. The engine of a car (cpu of a computer) is continually being refined. The console and driving instruments (UI of an OS) are continually being refined. The fact that the car travels on land is not being altered (the fact that an OS coordinates the hardware) is not being changed.
Real innovations in computing are not coming at a systems level, but at the hardware and UI level. Case in point: POSIX.1. There have been no drastic changes in the "standard" no OS adheres to in almost 10 years. The reason? no real research into what a system must do more of to work better.
Look at databases: there are 4 critical components to a database (ACID properties). The real research in databases today are not in implementing these properties, but in OODBMS or in RDBMS multi-dimensional queries, speed optimizations, etc. No real systems work.
Most of these properties of systems can only be improved on trivially and any new innovation will be found to be damn close to the POSIX.1 spec for OSes, have ACID properties of a DB, etc.
I'll give you something to ponder: is there a better way to implement virtual memory other than spilling over into hard drive? You have to be able to index them and swap them in and out--on things called pages.
Give me a faster general purpose index (or for that matter, OS-specific purpose) than a B-tree.
Until there is a new and radical type of hardware (other than storage, input, and output) there is little need to change what and OS does or how it does it radically. Rather, as mentioned way above, the techniques used now are optimized for a particular piece of hardware and/or software.
ReiserFS may be pretty cool stuff, but it hasn't led to really new things. There is the offer that it may allow constructing data structures reasonably efficiently via "hordes of tiny files," but nobody is really using that yet, and the "research" side of that is already reasonably well-understood.
For there to be real research out of something like ReiserFS would require that people start studying different ways of constructing (say) DBMSes by using the abstractions provided by the new FS.
It isn't really systems research for someone to construct a Linux emulation system to run atop EROS; what would be innovative would be to see what kinds of cool things that may have nothing to do with UNIX as we know it can be done with it.
The problem that he doesn't comment on, which seems to be an important flip side to the notion that Microsoft is a source of innovation, is that, during the 1990s, Microsoft did an impressive job of buying up top researchers, virtually closing down major systems software research groups:
If people started doing some substantial work on exploring how to powerfully connect applications together using CORBA, that could represent some new work; unfortunately, the tools are still maturing, and the mappings to C and C++ kind of suck, at least for the purposes of generating dynamic applications.
Remember, Pike's criticisms aren't based on some vague notion that Linux is useless or bad; they are based on the notion that it's not particularly innovative, from a systems software research perspective.
If 90% of your effort represents dealing with the same old ordinary UNIX stuff, that would be largely familar to a UNIX hacker of the 1970s, then whatever you're doing can't be more than 10% innovative. Note his comment that around 90% of the effort in Plan 9, which was one of the more innovative systems of the last decade, represented efforts to honor external standards. That's a problem.
If you're not part of the solution, you're part of the precipitate.