The Great Computer Language Shootout Revived

An anonymous reader writes "Doug Bagley's famous Great computer Language Shootout more or less died in 2001 out of lack of support by its own author. A group of developers have decided to revive it and update it with the latest versions of each compiler and interpreter available on the Debian distribution. The good news is, a wiki has been set up so that people can help improve the shootout, discuss the implemetations of the programs, and suggest optimizations."

Comments

gcj vs Sun?

[BRSloth]

They could have added comparsions between gcj and Java.

At first, when I saw the Java comparsions, I tought "ok, that will send the last report that says that Sun Java is faster than g++ to space!" but checking the "Implementations" section, it showed that they used gcj for its java comparsion.

Flawed

[Jordy]

Going through some of these sample programs, I see some serious flaws. The C implementation of regex just calls pcre. The C implementation of Hello World calls fputs instead of write(). The C++ implementation of hash uses sprintf and strdup. The C and C++ implementations of the fibonatchi sequence test are recursive. The tests themselves are so short that you are measuring the time to load the binary into memory and cleanup for half the tests.

Really there should be some automated way to submit a replacement for some of these tests that gets peer reviewed. That way each language has the "best" implementation for the language.

Of course, it would have to be considered the "best practices" use of the language as you could always write C in C++ (call write() for Hello World for instance).

Re:Flawed

[Vaevictis666]

Keep in mind that the purpose is to test a "method"

The fibonacci test, for example, is not testing fibonacci number generation, it's testing recursion. (Same deal with the Ackerman test) If you were allowed to do it iteratively, you're not testing the right thing.

Also, if you want to submit a replacement that you think will do better, feel free. Just make sure it's doing what the test spec says.

Please, do read the methodology page, it's there for a reason.

Re:Flawed

[Sigma 7]

The C implementation of regex just calls pcre.

Yes. If you have a better regular expression library, you're welcome to submit as a more efficient submission.

The C implementation of Hello World calls fputs instead of write()

fputs() is an ANSI C standard function designed to write text to an output stream or file. Write() is not ANSI C compatable, requires knowledge of the length of the string (either truncating if the length parameter is too low, or writes garbage if the length parameter is too high), and is designed for binary output rather than text.

When writing a portable C application, you should *NEVER* assume that a platform is using anything other than the ANSI C libraries except when absolutly necessairy (e.g. a mutlimedia library.)

The C++ implementation of hash uses sprintf and strdup.

Correct it then. The webpage containing the benchamarks has links available to contact the maintainers of the website so that you can submit a more efficient function.

This, of course, assumes that the HM class is not affected by the buf[] string being overwritten. If it is, then you'll have to find another way to eliminate the sprintf/strdup inefficiency.

The C and C++ implementations of the fibonatchi sequence test are recursive.

The fibonacci numbers test is explicitly stated that the procedure must be use recursion. If you want, you can ask the website maintainer to add a new test containg the iterative counterpart.

Just because there is an absolute best way of doing things does not mean that it is permitted in the test. Read the Testing Methodology for more information.

The tests themselves are so short that you are measuring the time to load the binary into memory and cleanup for half the tests

That's why the webpage provides an option to exclude startup time in the performance metrics. The option to do so is located at the top of the table, just below the Title naming the test.

Re:Flawed

[Jordy]

fputs() is an ANSI C standard function designed to write text to an output stream or file. Write() is not ANSI C compatable, requires knowledge of the length of the string (either truncating if the length parameter is too low, or writes garbage if the length parameter is too high), and is designed for binary output rather than text.

write() is POSIX/BSD/SVr4 which is certainly fine for the vast majority of platforms. Even with doing the strlen() and a const, it is cheaper than fputs() due to the locking fputs() puts on stdout (admittedly nothing compared to loading the binary into memory). Further, even if you wanted to use ANSI C, they should have just used puts().

Correct it then. The webpage containing the benchamarks has links available to contact the maintainers of the website so that you can submit a more efficient function.

Every single test for every single language is almost certainly flawed in some way. If they wanted to do it right, they should have allowed open submission and peer review of every test so that they could be written and rewritten then tested.

The point is, the person running this is certainly not an expert in every single language. I doubt very much that they can make a proper decision between four different and seemingly correct implementations of the hash implementation. Just submitting replacements doesn't make much sense because *my* implementation could raise just as many questions as the up right now.

This, of course, assumes that the HM class is not affected by the buf[] string being overwritten. If it is, then you'll have to find another way to eliminate the sprintf/strdup inefficiency.

Oddly enough, the "correct" implementation from a best practices standpoint may be slower by using std::string instead. It all depends on what your definition of "correct" is. That is why an open peer review process would be helpful as the feedback would educate people as to why certain design choices were made.

The question you have to ask yourself is, do the results on this page reflect reality in any way?