Slashdot Mirror
Finding New Code
tabandmountaindew writes "Too much time is wasted re-implementing code that someone else has already done, for the sole reason it's faster than finding the other code. Previous source code search engines, such as google codesearch and krugle, only considered individual files on their own, leading to poor quality results, making them only useful when the amount of time to re-implement was extremely high. According to a recent newsforge article a fledgling source-code search engine All The Code is aiming to change all of this. By looking at code, not just on its own, but also how it is used, it is able to return more relevant results. This seems like just what we need to unify the open-source community, leading to an actual common repository of unique code, and ending the cycle of unnecessary reimplementing."
There's an old adage in the racing business: if you're building a parade float, buy your wheels. If you're racing a 300 kph Formula One car, consider building your own. If you place very high demands on a component because it is at the core of what you do, the stock component may not be good enough.
So, in software, if the "wheel" is at the core if your product, you may have to re-invent it to get exactly what you need. This is not because everyone else screwed it up, just that the stock "wheel" serves 90% of the features for cheap. Look at the Mac iPhone's OS, Cisco's move to (cheaper, memory-wise) VxWorks on the WRT54, etc.
If we create this grand, uber code-searching portal, which can search the context of the code, aren't we making it easier for commercial entities to go ahead and and pick and choose those bits of code to use in their products, knowing full well that they're going to violate the GPL (or other OSS licensing models) by doing so?
I've talked to NO LESS THAN a dozen commercial companies in the last 2-3 years where they're actively taking FOSS source and incorporating it into their products, because.. (and I quote) "..Its freeware, so we can use it however we wish."
The licensing differences between "freeware" and "free software" seem to escape them. Just google around and you'll see thousands of FOSS projects listed on sites like TUCOWS, download.com and others, as "freeware" and not the proper "free software" that they are. There are also people who think "free software" means just that (lowercase "F" there).
Let's be sure that if we have a search engine that let's brainless developers look like experts by cutting and pasting bits of OSS code from here and there together to make their software work, that they know what the license is and that they must be in compliance with it to use it.
Please?
I think a system like this might work if it has a user feedback system, where particular authors get a good reputation by positive feedback. So you know that an implementation is probably good if that author has good ratings. Think about system libraries. Nobody (well, mostly nobody) writes their own implementation of system libs, because they trust (usually) the implementation provided by the OS or the compiler. Why do they trust? Why is a Microsoft routine more trustworthy than S00p3rC0d3r's? Just because those functions have been tested by users over and over again. And they're (usually) well-documented.
We shouldn't ignore a good idea just because it makes it easier for someone to do something illegal. There are laws to protect the code. I think the benefits will outweigh any loss from commercial companies stealing the code. I hope this does work out as well as it looks like it could.
http://bgcommonsense.blogspot.com
I think in order to be really useful for not reinventing the wheel, it should allow intelligent searching for licensing. That is, it should allow to restrict your search to codes with certain licenses, or even better, to code under a license compatible with any given license (or set of licenses).
For example, if you are working on code which you want to release as BSD, it's not much help if you find code licensed under the GPL, even if that code on its own is great. Likely, if you are writing GPLed code, you are not interested in code under licenses incompatible with the GPL (like e.g. the MPL).
Of course, the search engine cannot make a guarantee that the license will fit your needs, but then, it cannot guarantee that about the code's functionality either.
The Tao of math: The numbers you can count are not the real numbers.
While the article mentions that too much time is spent re-implementing new code, I disagree that this is necessarily a bad thing (tm). Re-inventing the wheel can often cause evolution of code, as opposed to the stagnation that can occur if something remains static. Now, of course people will say that this is GPL code, and people can then modify it -- this is of course true, but modification on that level seldom equates to evolution per se, sometimes because the changes as specific to the application, sometimes because you are trying to do something with code that simply wasn't designed for (I guess you could equate it to trying to run a a web server from Windows 95).
There is probably a point at which the system complexity of the resued code becomes great enough that the re-use is valuable. But how big, and how mature the reusable codebase, affects this decision.
www.jmagar.com
-
One thing that I find disappointing with all the code search engines is they all treat them as regular text files, more or less.
None of them seem to make an effort at understanding the code syntax.
That's why a few years ago I wrote one for C/C++ code called http://csourcesearch.net/
I just did it as an experiment, and using all open source software and in my spare time, but I think it having the ability to syntactically know the difference between a comment, a function, a structure, etc. makes a big difference.
When Google launched their engine, I was disappointed they didn't take the extra time needed to make their parser/engine smart.
Except that most people have jobs with deadlines. Besides, there's always more code to write. Even if I pull available code out of a repository, I'll still be continuously writing code. Furthermore, there's value in seeing how other people implement a solution because it is probably not exactly the way you would have implemented it and you just might learn something from their solution.
I don't think people are giving programmers enough credit for having common sense, and this project to reduce code re-implementation sounds pretty idealistic. I don't know if I call bullshit on it, but I smell a few flawed assumptions.
First flawed assumption seems to be that the hard part in re-using code is simply to find it. But that's crap. When code is in the form where it can even be re-used, it's called a module, and a big chunk of this code is the module's interface. The interface is what lets you re-use it. But there are huge differences between interfaces. There are different calling conventions, different parameter orderings, different limitations on thread re-entry, different permissions on order of things you're allowed to call, and entirely different approaches to specifying the interface. A streaming library can have a public function Read() or it can have a pair of public functions BeginRead()/EndRead(), and there are many valid reasons for both cases.
Point being, you often have to refactor a module's interface before you can fit it into your project, and depending on the size and purpose of that module, the refactoring might be just as slow as writing a new implementation.
Second flawed assumption is that developers aren't even able to find the code which they need to find today. But that's crap too because there are a lot of great, re-useable libraries that programmers already commonly know about, or can easily find through Google or Sourceforge. First of all, the standard C/C++ libraries give you a lot. Then there's zlib, curl, glut, Allegro, and hundreds (thousands, depending on your standards) more, depending on what you're doing.
Come to think of it, when you really want to re-use code, you look for libraries, not source code. Searching for source code mainly helps people who want to learn programming.
I know that the site linked in the summary only contains Java code right now, and I'm mainly focusing on C/C++ here but I think a lot of what I said applies. (Don't tell me that Java's automatic turning of each class into a monitor solves the thread re-entry problem, because that really just substitutes one problem for another.)
The iPhone isn't really a good example other mobile oses (e.g. Linux) could have done the job just as well if not better. There are only two reasons for apple to use their own OS branding and vendor lockin.
thank God the internet isn't a human right.
Is the code easy to find? Will a quick search of sensible key words take me to a short list of results with high accuracy? No point in spending an hour wading through results that may or may not be useful when I can implement and test it myself in 2 hours.
Is the license clear? I may eventually want to release as open source or commercially use something I write. If I include someone else's code/library I have to make a note (hopefully in the LICENSE file provided with the code or in the top of the code comments) on what the license is. Is it BSD, GPL, public domain, not stated or some commercial license that lets me look at the code but not use it myself?
Is the code self contained? This generally means does it come as a library. I dont like copying and pasting code into my code - especially if its not the same coding practice as my own. (this comes abck to licenses above - if its self contained and with an incompatable license atleast I can rip and replace later if I need to)
Is the code well known? Is if the defacto standard for doing this type of thing (STL, Perl core, glibc)? Or is it one of several well known options for the same thing (gtk, qt, kde)? Or is it an unknown? This will help you know how well this code is field tested already - I don't like signing up to be someone else's beta tester for free!
Is the code still maintained? Is this an active company with a project? Or a group on source forge? Are the developers still around and the forums active? If I need a new feature further down the line is there chance of support? I don't usually want to pick up the whole dead weight of supporting unsupported code that I didn't write if I can avoid it.
Can I use it as is? It frequently takes longer to modify an "almost there" modules to do what you need than it would have done to reimplement the wheel as it were and write it yourself first time, and writing it yourself will atleast make future debugging easier assuming you have a good memory for design and good coding practices.
Is there documentation? The old comparison about documentation and sex, when its good its very very good and when its bad its better than nothing. I dont want to have to read someone elses uncommented un documented code just to evaluate if it might work for me. I want to be able to read a good overview of the library, its functions, methods, attributes, errors and exceptions - CPAN is an excellent (in most cases) example of what I mean.
Thats a pretty hard list of requirements to meet - true it shouldn't be, but this is the real world. If those requires are not met then odds are it will be less effort in the long run for more reward for me to implement it myself.
$_="Slashdotter";$syn="OTT";s;..;;;sub _{print shift||$_};s!ash!Perl !;s=$syn=ack=i;tr+LLEd+BLAH+;_"Just Another ";_
"I often Push on the Pull doors, just to see if they work...."
Yup. Same here.
I once walked up to the doors in a parking garage. The sign above the door knob said turn and push. So I did a pirouette and gave the door a shove.
It opened. You should have seen the looks on the faces of the people around me.
Anyway, back to the main topic.
I suspect the "not invented here" problem is why these code repositories aren't used. Most of us programmers don't want to use someone else's code because we have to first figure it out, and then use/interface with it. We tend to think it's easier to just re-write the thing. You know, just like the complete re-write for version 3.0 of a program.
Not that I agree that's the best way to go, mind you.