Slashdot Mirror

← Back to Stories (view on slashdot.org)

The Great Computer Language Shootout Revived

Posted by timothy on from the sputters-to-life dept.

An anonymous reader writes "Doug Bagley's famous Great computer Language Shootout more or less died in 2001 out of lack of support by its own author. A group of developers have decided to revive it and update it with the latest versions of each compiler and interpreter available on the Debian distribution. The good news is, a wiki has been set up so that people can help improve the shootout, discuss the implemetations of the programs, and suggest optimizations."

17 of 51 comments (clear)

  1. Judging by the lack of comments... by Anonymous Coward · · Score: 2, Funny

    Judging by the lack of comments in here, I'd say it isn't just the author who lost interest.

    1. Re:Judging by the lack of comments... by turgid · · Score: 2, Funny

      No man. We're all flame-warred out from the other day.

      --
      Stick Men
  2. And that'd be...? by Anonymous Coward · · Score: 2, Funny
    ...and update it with the latest versions of each compiler and interpreter available on the Debian distribution.

    I'm fighting the urge to ask how C compiled with gcc 1.3 will compare to COBOL...

  3. gcj vs Sun? by BRSloth · · Score: 3, Informative

    They could have added comparsions between gcj and Java.

    At first, when I saw the Java comparsions, I tought "ok, that will send the last report that says that Sun Java is faster than g++ to space!" but checking the "Implementations" section, it showed that they used gcj for its java comparsion.

  4. Flawed by Jordy · · Score: 4, Informative

    Going through some of these sample programs, I see some serious flaws. The C implementation of regex just calls pcre. The C implementation of Hello World calls fputs instead of write(). The C++ implementation of hash uses sprintf and strdup. The C and C++ implementations of the fibonatchi sequence test are recursive. The tests themselves are so short that you are measuring the time to load the binary into memory and cleanup for half the tests.

    Really there should be some automated way to submit a replacement for some of these tests that gets peer reviewed. That way each language has the "best" implementation for the language.

    Of course, it would have to be considered the "best practices" use of the language as you could always write C in C++ (call write() for Hello World for instance).

    --
    The world is neither black nor white nor good nor evil, only many shades of CowboyNeal.
    1. Re:Flawed by Vaevictis666 · · Score: 3, Informative
      Keep in mind that the purpose is to test a "method"

      The fibonacci test, for example, is not testing fibonacci number generation, it's testing recursion. (Same deal with the Ackerman test) If you were allowed to do it iteratively, you're not testing the right thing.

      Also, if you want to submit a replacement that you think will do better, feel free. Just make sure it's doing what the test spec says.

      Please, do read the methodology page, it's there for a reason.

    2. Re:Flawed by Jordy · · Score: 2, Informative

      Ok, I can understand that one. The rest I'm not sure about. Surely using strdup() and sprintf() in a C++ program to hash is wrong or using fputs() to output a string. Plus, how the heck did they use 2 megs of memory for Hello World?

      Static:

      jordy 21022 0.0 0.0 388 60 pts/1 T 12:39 0:00 /home/jordy/hello

      Dynamic:

      jordy 21038 0.0 0.0 1344 228 pts/1 T 12:40 0:00 /home/jordy/hello

      --
      The world is neither black nor white nor good nor evil, only many shades of CowboyNeal.
    3. Re:Flawed by Sigma+7 · · Score: 4, Informative
      The C implementation of regex just calls pcre.
      Yes. If you have a better regular expression library, you're welcome to submit as a more efficient submission.

      The C implementation of Hello World calls fputs instead of write()
      fputs() is an ANSI C standard function designed to write text to an output stream or file. Write() is not ANSI C compatable, requires knowledge of the length of the string (either truncating if the length parameter is too low, or writes garbage if the length parameter is too high), and is designed for binary output rather than text.

      When writing a portable C application, you should *NEVER* assume that a platform is using anything other than the ANSI C libraries except when absolutly necessairy (e.g. a mutlimedia library.)

      The C++ implementation of hash uses sprintf and strdup.
      Correct it then. The webpage containing the benchamarks has links available to contact the maintainers of the website so that you can submit a more efficient function.

      This, of course, assumes that the HM class is not affected by the buf[] string being overwritten. If it is, then you'll have to find another way to eliminate the sprintf/strdup inefficiency.

      The C and C++ implementations of the fibonatchi sequence test are recursive.
      The fibonacci numbers test is explicitly stated that the procedure must be use recursion. If you want, you can ask the website maintainer to add a new test containg the iterative counterpart.

      Just because there is an absolute best way of doing things does not mean that it is permitted in the test. Read the Testing Methodology for more information.

      The tests themselves are so short that you are measuring the time to load the binary into memory and cleanup for half the tests
      That's why the webpage provides an option to exclude startup time in the performance metrics. The option to do so is located at the top of the table, just below the Title naming the test.
    4. Re:Flawed by Jordy · · Score: 4, Insightful

      fputs() is an ANSI C standard function designed to write text to an output stream or file. Write() is not ANSI C compatable, requires knowledge of the length of the string (either truncating if the length parameter is too low, or writes garbage if the length parameter is too high), and is designed for binary output rather than text.

      write() is POSIX/BSD/SVr4 which is certainly fine for the vast majority of platforms. Even with doing the strlen() and a const, it is cheaper than fputs() due to the locking fputs() puts on stdout (admittedly nothing compared to loading the binary into memory). Further, even if you wanted to use ANSI C, they should have just used puts().

      Correct it then. The webpage containing the benchamarks has links available to contact the maintainers of the website so that you can submit a more efficient function.

      Every single test for every single language is almost certainly flawed in some way. If they wanted to do it right, they should have allowed open submission and peer review of every test so that they could be written and rewritten then tested.

      The point is, the person running this is certainly not an expert in every single language. I doubt very much that they can make a proper decision between four different and seemingly correct implementations of the hash implementation. Just submitting replacements doesn't make much sense because *my* implementation could raise just as many questions as the up right now.

      This, of course, assumes that the HM class is not affected by the buf[] string being overwritten. If it is, then you'll have to find another way to eliminate the sprintf/strdup inefficiency.

      Oddly enough, the "correct" implementation from a best practices standpoint may be slower by using std::string instead. It all depends on what your definition of "correct" is. That is why an open peer review process would be helpful as the feedback would educate people as to why certain design choices were made.

      The question you have to ask yourself is, do the results on this page reflect reality in any way?

      --
      The world is neither black nor white nor good nor evil, only many shades of CowboyNeal.
    5. Re:Flawed by 0x0d0a · · Score: 2, Insightful

      As was pointed out in the many comments concerning Java's speed when compared to C++, that hash code for C++ is seriously screwed up. It leaks memory like crazy, and one of the examples doesn't even do the same thing as the Java example because the code uses hash_map::operator[]() to test for element existence, instead of hash_map::count().

      Funny that you should say that, because hash tables are one of the easiest ways to "leak" memory in Java -- to have stray references floating around.

      --
      May we never see th
  5. If it's true, it's great news by GCP · · Score: 2, Interesting

    I find it quite frustrating that there is such huge inertia in programming languages. Even when languages have some remarkable advantage, programmers won't use it because "nobody uses it."

    I don't deny that popularity provides some huge advantages. I just hope that there can be mechanisms to bootstrap popularity, and this Great Language Shootout is (potentially) the sort of thing the could do that.

    Some languages, for example, have significant advantages in some specialty area: Erlang in parallelism, Lisp as a language for writing specialty languages, bug minimization approaches taken by Eiffel or the functional languages, etc. They may be awful for other types of work, but the fact that a hammer is a terrible saw doesn't make it any less of a great tool for the right problem.

    If there were an active, busy site frequented by developers at which there were a wide variety of benchmarks that allowed uncommon languages to show off their specialty advantages, more developers would do some of their work in those languages, and the increasing popularity would result in better implementations and a flourishing of new languages.

    Right now, the popular languages don't come close to taking full advantage of what we have learned about programming, and yet you would have to be either an academic or a fool to put a lot of effort into designing a great new language. The way things are right now, a great language designer probably has a better chance of being hit by lightning than he has of ever making his great new language popular.

    Anything that reduces this language inertia (and YES that includes .Net) is a Good Thing.

    --
    "Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
    1. Re:If it's true, it's great news by Brandybuck · · Score: 2, Insightful

      Probably because most of us want to be expert in a few languages than mediocre in many. It's like human language. How many people do you know are fluent in more than one language? Two languages? By the time you get to fluency in three languages, they're getting rare.

      You can't pick the language for the task, because in all probability you don't have the time to learn a new language for each task. There are many times when I have to write a piece of code in an "inappropriate" language simply because I don't have time to learn the appropriate language. I know what it's like to be in the middle of some C++ code thinking, "gee, [Java|Lisp|Ruby|etc] would be perfect here...".

      As with human language, some people are naturals when it comes to being fluent in multiple programming languages. Unfortunately, I am not one of their number.

      --
      Don't blame me, I didn't vote for either of them!
    2. Re:If it's true, it's great news by GCP · · Score: 2, Insightful

      Can YOU become an expert in Lisp in one week?

      I've actually spent a number of years using Lisp. I've been paid to write software in somewhere between a dozen and two dozen languages, depending on your definition of language, and I assure you that your "acquire complete mastery of a new language in a week" test is completely irrelevant.

      If you haven't even started to look at a language until a week before you need to be a master, it's not the fault of the project. You simply aren't qualified for that particular job.

      But, unless you never do anything until someone else forces you to do it, you usually have years in which to get to know a new language. Over that time, you will either determine that it is not useful enough to add to your toolbox or you will have added it to the extent you feel it warrants.

      So all I'm talking about is the advantages of a site like the Great Language Shootout that gives you a lot of information with which to make your decisions over that period of years that I'm referring to.

      --
      "Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
    3. Re:If it's true, it's great news by LardBrattish · · Score: 2, Insightful

      It's more than that.

      In the real world someone is paying to get software done. That someone does not want to pay us to learn a new language just so a function doesn't need to be written because in language X it's a one liner. Write the equivalent function or find a suitable existing library. If you program in C/C++ the library probably does exist. In Haskell or Eiffel...

      Also real commercial products do not exist in a vacuum. You will probably have existing code that does at least part of the job. The person paying the bills will not want to pay to port all of that code. They will want to leverage it.

      That's why the new cool language du jour does not rule the world. It's frustrating if the language you want to use is demonstrably better but factor in the cost of finding more progammers that can use the language effectvely & most of the advantages you perceive evaporate.

      --
      What are you listening to? (http://megamanic.blogetery.com/)
  6. Oh Boy! by Rhesus+Piece · · Score: 2, Funny

    Oh boy! There is a "Manufacture your own results" cgi! Now we can all play the "change the weighting so your language of choice wins" game!

    I changed everything to '1', and Ocaml won.

  7. System Calls by Detritus · · Score: 2, Insightful
    The C implementation of Hello World calls fputs instead of write().

    write(2) is a system call on UNIX systems, not a part of the C language or the C standard library. It has no place in a portable C program. You might as well argue that the program should have use DosWrite() (OS/2 system call) or $QIO (VMS system call).

    --
    Mea navis aericumbens anguillis abundat
  8. The Python results seem suspicious by Colonel+Panic · · Score: 2, Insightful

    Apparently they used Psycho to get the Python numbers. This is why the Python numbers look so good. However, Psycho is not the standard distribution of Python and it doesn't allow the full flexibility of standard Python. This should be noted in the results. They should create a set of results using the 'standard' Python distro as well. ...otherwise, why not just let us use Inline::C in Perl (or in Ruby) - basically this would allow you to put C code into your Perl/Ruby script (which wouldn't be fair).