How Mainstream Can Code Scavenging Go?

The time-honored tradition of code scavanging has long been a way for new programmers to "break in" to a new language or task that they may not want to build from the ground up. The re-use of old code, cleaned up and tweaked to a new purpose can help developers learn many useful skills and accomplish tasks quickly, especially for small tasks that aren't of vital importance. One blogger wondered if this process could be formalized and tools could be built to help foster and enable code scavanging on a mass level. Is this a viable option, or are there just too many things to consider?

Comments

IP Laws?

[gambit3]

So, how quickly would you run afoul of Intellectual Property laws doing this?

Re:IP Laws?

[thegrassyknowl]

I want to be the first to welcome our new GPL overlord to the commercial software world.

Re:IP Laws?

[caffeinemessiah]

So, how quickly would you run afoul of Intellectual Property laws doing this?

That's a great knee-jerk reaction. Without understanding the motivation behind the article, you assume that code scavenging means stealing other people's code. What they're really talking about is (legitimately) re-visiting code that you or other people have written, and then picking and modularizing bite-sized chunks. In other words, you would design a large program (mark I) and then go back and pick out useful parts, clean/debug them and have working modules (mark II) for the next project.

Also, for people who haven't read TFA, it's 9 short paragraphs long and barely an article. They talk about a "formal approach to code scavenging" without even coming close to explaining what exactly that MEANS.

Re:IP Laws?

[sootman]

They talk about a "formal approach to code scavenging" without even coming close to explaining what exactly that MEANS.

Agreed. Reading TFS, I thought it was going to be yet another "we can make programming like Lego!" thing. (Which it ain't, and probably never will be. Bonus reference: "Lego" is mentioned in the second paragraph of this article about Steve Jobs/NeXT/WebObjects from Wired. God bless Wired and their eternally fucked-up CMS that can't serve images for any story in the archive and, this week, shows the actual HTML code that should be formatting the Question-and-Answer portion of the article.)

Reading TFA, I really don't know much more than I did before. This is the best I could come up with:

Code scavenging is seen as the most frequent and least complex way of re-using code and has been common practice at an informal level since programming first began. Programming is difficult to teach and most programmers learn their chops by looking at working code and using it as the basis for building their own programs. In other words, they "scavenge" the good bits and tweak them to a new purpose.

The term scavenging appears to have first surfaced as a formal concept in a 1992 paper by Charles Krueger of Carnegie Mellon University. It was tested by academics in the 1990s but rejected because it yielded few gains for a lot of effort.

According to Hackett, code scavenging is worth re-visiting because the Web makes it easier to find code and re-use it. He points to sites where massive amounts of existing code are available for potential scavenging such as Google code search, Sourceforge, Code Project, Microsoft's Codeplex, and O'Reilly's Code Search. Others include the Free Software Foundation (FSF), FreeVBcode.com, Freecountry and Freshmeat.

So, code scavenging is... um, re-use? Can anyone make better sense of that than I can?

"In other words, they 'scavenge' the good bits and tweak them to a new purpose."

Um, no. You scavenge the pieces you need, not necessarily the good bits. Have you ever been looking for some code to do parse phone numbers, and while looking at source, said "Hey! This looks like a great way to compare two lists!" Probably not. You're only looking for formatting code, so that's all you see, so that's all you get. Looking at source is not like looking at produce at the food store, where you can walk by the tomatoes and they catch your eye because they're perfectly ripe and really, really nice-looking.

Rather than searching Google, I think every good programmer should take the time to create a really good library. I don't mean take the time writing great code, I mean take the time to organize it into a proper library: make one, clean, well-commented version; put things into variables, ($tableName in queries instead of the actual table name, etc.) and pull code from that when you need it, rather than just copying-and-pasting from the last place you remember using it and then changing all the variable names, table names, etc.

I plan to make mine Real Soon Now. :-)

>> So, how quickly would you run afoul of Intellectual Property laws doing this?

> That's a great knee-jerk reaction.

No, that's just the first thing that popped into his head. (Pardon me if I'm putting words in your mouth, Mr. Gambit.) With that one sentence, he did not say (or imply) "The only people who would use this are thieves." He just put out that question for people to discuss. That topic came up here just a couple days ago. I highly recommend reading that discussion. There are some very good points; among them, that if you publish something with no licensing info, it is copyrighted to you by default. (In the US at least,

Re:IP Laws?

[alex4u2nv]

These are the ideas that CPAN, PEAR and other code repositories are built on. So instead of trying to reinvent the idea, the author should have poked around a little more, and to learn more about what is available as opposed to trying formalize a "hackup job."

The parent makes a good point here, where you should take the time to build a clean library base to work with. If one should have a well structured infrastructure, and should they need to implement a certain feature, more often is the case, that a generic library to facilitate the need already exists. If it's open source you're in luck, if it's commercial, you could license it. For example, take a peak into the game industry. A lot of those engines libraries are licensed out to many companies.

Now I will assume 'bits and pieces' is nothing more than 1 class, or a function. So, If one should be looking for bits and pieces of codes, as a developer you should be able to develop at least that much.

"Hackett makes some good points, but can scavenging for scraps really turn you into a master programmer, and is it possible - or even desirable - to institutionalize this "make-do" approach?...," TFA

The short answer is No.
This sort of programming will not make you a master programmer, it will make you a master scavenger.

Re:IP Laws?

[jc42]

[I]f you publish something with no licensing info, it is copyrighted to you by default. (In the US at least, and many other countries as well.) So even if you're looking at a site that is, say, clearly marked as a tutorial, that doesn't necessarily mean that you can use that code, unless the guy comes out and says the code is public domain/GPL/etc.

Good point. I think I'll use it the next time someone comments about code of mine that is overly complex and convoluted.

"You see, all the simple ways of doing it have been published in tutorials. This means that they're copyrighted by the author. If I used simple code, I'd almost certainly be violating some author's copyright, and since you hired me to write the code, they'd probably sue both of us. Wouldn't want that, would we?"

(Only half-joking here. This is what the world's coming to. ;-)

I've been thinking of scavenging code from SCO...

[kclittle]

...since they obviously aren't going to be using it for much longer...

Vernor Vinge

[boster]

In A Deepness in the Sky, Vernor Vinge posited Programmer Archaeologists would replace all new development. http://everything2.com/index.pl?node_id=760521

Re:Vernor Vinge

[bcharr2]

Code is basically an algorithm that solves a computational problem. So yes, you can cut and paste algorithms. If you want your application to be maintainable, however, you also have to solve the larger architectural problems, which is something you DON'T get when you cut and paste code.

It is one of the most misunderstood concepts about programming. A programmer fresh out of college knows how to write algorithms really well, but has no idea that there are architectural and design land mines waiting for them just down the development road.

Too many development shops will get an app "working" and think that is all there is to development, because no one has the depth of experience to look a year down the road and see that they will need to rewrite the entire app from scratch in order to make the simplest of changes.

I'm sorry, but if your program is not extensible nor maintainable then you really haven't "succeeded" at anything. You've simply fooled yourself into thinking the process is simpler than it is while screwing your clients out of their development dollars.

Re:Vernor Vinge

[Alphager]

The method of assigning homework is to give 3 lines on what a program needs to do. For example, write an FTP client that you can use to download a file. The method of grading is if you can download a file or not. The decision on how to get the HW done with the least amount of time spent is an architectural challenge.

Which is what the parent hates: people who only think about getting the product out of the door and who sacrifice things like maintainability.

Even firefox and Mozilla did a few complete rewrites of the various parts. Rewrites are part of programming unfortunately. The nice thing about rewrites is that the programmers now have experience on how to do things better and are able to better compartmentalize the code. I wouldn't say rewrites are terrible things - though they do annoy management and people above to no end.

Firefox & Mozilla are prime examples of bad codebases. We all know the disaster that was the last rewrite in-house at Netscape.
_SOME_ re-writes are necesary and usefull, but a company where the usual way to add a new feature is a re-write of the whole software is doing somethign wrong.

code repositories

[drfrog]

like cpan and ruby gems etc

we scavenge code online w/e, find it needs to be used by a lot of people

so we inherit the scavenged and put it in a nice module and tada!

this is nothing new

semi-formalized

[mycall]

Some people are already doing this, such as koders, code fetch, codase, and snippets. Talk to them for formalizing as I'm sure they have some good input.

It's called a "subroutine library"

[Animats]

The Web 2.0 crowd rediscovers subroutine libraries. Film at 11.

Re:It's called a "subroutine library"

[visualight]

The poster you replied to is thinking the "lots-o-books" kind (I'm sure). As in, "a library of subroutines".

Re:It's called a "subroutine library"

[Tablizer]

The Web 2.0 crowd rediscovers subroutine libraries. Film at 11.

You gotta punch it up, PHB-ify it: "Reusable Enabling Action-Oriented Web Object Architecture Patterns".

Re:It's called a "subroutine library"

[Comatose51]

You joke but I've phone interviewed someone and asked him, "What are some ways of reusing code or implementation?" Inheritance or composition would have been okay. In fact, a lot of answers are acceptable.

Instead, he answered "uh... cut and paste?"

Google Code

[TooMuchToDo]

http://code.google.com/

Re:Google Code

[daveb]

for code scavenging try this instead / as-well
http://www.google.com/codesearch

foreheat meet desk

[Yath]

I guess this is slow news day. Using bits of code without writing everything from scratch - how novel! How controversial! Is there anyone who doesn't do this? What kind of skull-shattering boredom do you have to endure before you start writing blog entries about this?

And the first article suggests that trusting the code is an issue, because you didn't write it. Well let's see - it's short, and you just pasted it into your program. But you're not going to bother to read it? You fail. Seriously.

Isn't this a library?

[poppycock]

Isn't this, you know, a library?

"Scavanging"?

[Anonymous Coward]

Why don't you scavenge the dictionary to spell properly?

Unlikely?

[SnoopJeDi]

From TFA:

You are unlikely to find what you want with a simple Web search

Since..when? Recently I've picked up perl again, and I've found more than what I need to scavenge to make my own personal extensions to blosxom through google searches.

I mean, granted, it depends on your definition of a bite-size task, but it's a blanket statement no matter which way you spin it.

This is a great idea!

[AEton]

If only there were some computer programming language that had built-in support for some kind of a Comprehensive Archive Network, that would be the best.

Maybe the C++ language could do it. Then you could just ... hmm ... "import" the things you need from the Comprehensive C++ Archive Network!

Hmm, CC++AN sounds pretty dumb. It'd never catch on. Oh well.

Today Slashdot jumped the shark.

[xxxJonBoyxxx]

Today Slashdot jumped the shark.

Seriously. I'm starting to lose brain cells when I read the "articles" these days.

Re:scavanging turds is mainstream

[YrWrstNtmr]

That sir, was a work of beauty. There should moderation other than -1 Troll for such art.

Indeed...we need a -5 Asshole.

It's already been done.

[etnu]

It's called Google.

Wow

[smitth1276]

That article used a lot of words to say absolutely nothing. But it got me thinking... perhaps we could group related snippets of code into units called "libraries", and then we could easily use those libraries to perform common tasks?

The reuse of old code...

[adamkennedy]

So... the formalised use of old code... you mean like CPAN?

Re:Don't we call that "refactoring"?

[Anonymous Coward]

TFA is not really a FA.

It leaves us only "the". Which is an article. Liar.

This reminds me of another book ...

[James Youngman]

... of an SF book I read a few years ago, where all programs were written by a process of digging into 10,000 years' worth of computer programs in a sort of archaeological way, pulling out something that did more or less what you want and amalgamating it with what you had so far. I thought at first that it was a Vernor Vinge book, but checking the plot summaries on wikipedia, it looks like it was somebody else. Can anybody remember the book I'm thinking of?

Re:This reminds me of another book ...

[stoborrobots]

boster suggested above that it was A Deepness in the Sky ...

oops

[yoprst]

Apparently, slashdot has already scavenged angle brackets - they're gone. Truly the most scavenged code ever.