Slashdot Mirror
No More Coding From Scratch?
Susan Elliott Sim asks: "In the science fiction novel, 'A Deepness in the Sky,' Vernor Vinge described a future where software is created by 'programmer archaeologists' who search archives for existing pieces of code, contextualize them, and combine them into new applications. So much complexity and automation has been built into code that it is simply infeasible to build from scratch. While this seems like the ultimate code reuse fantasy (or nightmare), we think it's starting to happen with the availability of Open Source software. We have observed a number of projects where software development is driven by the identification, selection, and combination of working software systems. More often than not, these constituent parts are Open Source software systems and typically not designed to be used as components. These parts are then made to interoperate through wrappers and glue code. We think this trend is a harbinger of things to come. What do you think? How prevalent is this approach to new software development? What do software developers think about their projects being used in such a way?"
Code recycling, and that word was chosen specifically for this case, is a time-honored and proven concept. One just has to look at Microsoft's products to see that.
If you think debugging is a pain NOW...
I certainly do not want to be developing the same common hardware implementation for a FPGA over and over and over again. Give me something the code for something that works and let me get to the stuff that is unique to the project.
Ooo man the floppy drive is broken. No wait. The computer is just upside down.
The promise of SOA says that you won't have to do this. If you believe in that promise then anyone that develops projects in the future will create them in discrete elements that are accessible as a web service. If you want to start a new development project, just utilize those services that you need and ignore the ones you don't. Because the functionality is encapsulated (and therefore, written, debugged and tested) within the service you're good to go.
I see application projects in the future acting like glue that holds many services like this together and makes them more than the sum of their parts.
I'm a big tall mofo.
As the developer of freely available software, I find the prospect of people using my code a mixed bag. Partially, I feel an ownership of the code I write and am somewhat offended by the idea of people using it . However, as a paid engineer, I go through this at regular intervals; older projects get handed to others for support, as I work on new components.
On the other hand, I welcome the idea that my free code would be used by others - it is a flattering prospect, I suppose.
Others profit from this sort of re-use: COM, CORBA, Jars, etc...
Some basic tasks like file io or token processing and such minor things might be reused. But even then porting something so simple like a string tokenizer written for a full fledged virtual memory OS like Unix/WinXP to a fixed memory handheld device is highly non trivial, especially if you want to handle multi-byte i18n char streams
If the author sells what he was smoking while coming up with the article, he stands to make tons of money.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
It's called "Pop Music". Or, more specifically, "Nickelback". :)
ResidntGeek
Code re-use is a great idea. Free software is a great idea. Taken together, and to an extreme, they can cause problems, particularly where security is concerned. What happens when someone finds a security flaw? How can you contact the people who are reusing your code if you have no idea who they are?
To take a personal example, my delta compression code, which I wrote for FreeBSD, is now being used by Apple and Mozilla to distribute updates; I've talked to their developers, and if I find a security flaw in this code (very unlikely, considering how simple it is), I'll be able to inform them and make sure they get the fix. On the other hand, I know developers from several Linux distributions have been looking at using my code, but I'm not sure if they're actually using it; and searching on Google reveals dozens of other people who are using (or at very least talking about using) this code.
Putting together software by scavenging code from all over the Internet is like eating out of a garbage dump: Some of what you get is good and some of what you get is bad; but when there's a nation-wide recall of contaminated spinach, you'll have no idea if what you're eating is going to kill you.
Tarsnap: Online backups for the truly paranoid
Libraries
Some context for people who didn't read the book... or didn't read it carefully enough.
First, Vernor Vinge has a PhD in Computer Science. This obviously doesn't guarantee he can't be wrong, but to those commenters who said something like "these ideas are idiotic"... you've got an uphill battle to convince me that you're that much smarter than Vernor Vinge, especially as most of you saying that don't show me you understood what he was saying in the first place.
Second, A Deepness In The Sky is set in his "zones of thought" universe. In this universe, the fundamental limits of computation vary depending on where you are physically in the galaxy. This is only faintly hinted at in A Deepness In The Sky, it is explicitly spelled out in A Fire Upon the Deep. This limit on computation may or may not be real. One of the effects of this limit on computation is that you can build a system larger than you can really handle, and eventually all such systems come apart in one way or another. This story is set thousands of years in the future and it is explicitly (albeit subtly) pointed out that the software running the ramscoop ships has direct continuity with modern software. (Qeng Ho computers use the UNIX epoch as the most fundamental form of timekeeping; apparently even the relativistic compensation is layered on top of that.) We are at the very, very beginning, where it is still feasible to burn an OS down entirely and start from scratch.... or is that really still feasible? (Perhaps Microsoft will soon find out.)
Those of you posting that "we can always wrap it up in an API or whatever", I'd say two things: First, you get the Law of Leaky Abstractions working against you. The higher up the abstractions you go, the more problems you have. (Look it up if you don't know what that is.) The more sub-systems you make, the larger the state space of the total system becomes (exponentially), and the harder it is to know what's going on. It is entirely plausible that you eventually hit a wall beyond which you can't pass without being smarter, which, per the previous paragraph, you can't be.
In other places in the galaxy, you can be smarter, and Vernor Vinge postulates the existence of stable societies on the order of thousands or tens of thousands of years or beyond, where the society actually outlasts the component species, because the software base that makes up the species does not exceed the ability of the relevant intelligence to understand it.
Both cases (software might exceed intelligence, intelligence might grow with software) are extremely arguable, and I do not think he is advocating either one per se. (Leave that for his Singularity writings.) But you do him a disservice if you think he is not aware of the issues; he's extremely aware of the issues, to the point that he is the reason some of us are aware of the issues.
(Even this is a summary. In isolation, probably the best argument is that it is always possible to create a software system one can not understand or control, but one person can be wise enough to avoid that situation. However, in A Deepness in the Sky Vernor Vinge explicitly talks about how in a societal situation, one can be forced by competitive pressures to over-integrate your system and make it vulnerable. "OK, but the government can be smart enough to realize that's going to happen and step in to stop it." First... smart government? But even if you overcome that objection, now your society faces death-by-surveillance and other automated law enforcement mechanisms, which since they can't be human-intelligent will fail. If you avoid that (and it is a big temptation), then you face the problem of anarchy. And remember that "governance" is anything that governs; even if the "formal government" doesn't regulate you to soceital death, private corporations may do it. Anyhow, upshot, Vernor Vinge has done a lot of thinking on this topic, it shows in his books, it is not showing in the criticisms I've seen posted, and when it gets down to it he really has more questions than answers.)
Component based programming for Linux turned out very different. Since Linux stuff tends to be developed from the bottom up, starting with a library, then a console app, then a GUI, most "components" ended up just being stuff lower on the stack, like libpng, gzip or whatever. That, and file formats tend to be standardised and read with a library, rather than file access being implemented in any specific application that needs to be loaded every time the file needs to be accessed.
.NET, seems to be more of a developer friendly way of providing and using library components as a developer does in Linux. .NET components remind me of how libraries are used in Linux. IMO this is a cleaner way of working and it's nice that Microsoft are starting to think this way.
On DOS and Windows, applications tended to be all-in-one blobs that use proprietary file formats and only depend on the operating system itself. Thus, OLE made sense. There was OLE-type technologies on Linux (such as Bonobo and KParts) but they never took off in any big way due to the general lack of need. In most cases, they were just used for automation, which better technologies such as DBus now exist.
As for "real code reuse" on Linux... try ldd-ing any application and seeing how many other libraries it uses to do it's bidding. Many application's I've written are just sticking together bits and pieces of other libraries, and that's in C++. I'd go as far as saying that code reuse on Linux is currently better than that in Windows. Generating quick applications with Linux tends to be pretty easy using bash, Python and even C, C++ or C#, where libraries have directly translated into "components" and "modules". The thing on Linux slowing down development isn't having to reinvent the wheel every time you write code, but rather, deciding what libraries you want to depend on and the difficulty of the languages favoured in Linux development, like C.
Since you bring it up, OLE seems to be taking a backseat in Windows now, where ActiveX is rarely used by anything other than IE plugins. Now clipboard/d'n'd/DDE and Microsoft Office are the only times I actively see it being used. COM is still heavily in use by libraries such as DirectX, however. It was never really used to it's full potential, such as to open Photoshop documents in Microsoft's preview application. Windows, with
Why does everybody want a "revolution" when evolution does the job? You only have to look at the language popularity charts to see the trend away from low level programming languages to higher level languages with more expressive constructs. Yes they were here years ago but it takes time for things to catch on properly. I program aeronautical systems for a living and never step outside strict ANSI C for flight code. However for the supporting tools around us we use Python. It isn't FAST but it is fast enough to get the job done and we can develop new tools faster and easier than coding them in C/C++. Why becase Python is a more expressive language than C and for a large domain of problems is now more suitable. The current state of programming language usage is more like a sliding window over the domain of currently available and yet to be available languages. There are rarely revolutions but it is dumb to say that what we are doing today is no differnt to 40 years ago. System complexity rises all the times and new ways of expressing the problems and solutions to those problems will evolve over time. Reusing a 20 years old library will not cut it.
The bikini - security through obscurity since 1943