sincos() is not an ANSI/ISO Standard function. I am writing the applications to Standards for maximum portability; therefore, using sincos() is not an option.
Heck, I could just drop some inline assembler code into the C program -- but then, I'd be benchmarking hand-tuned assembler language, not Standard C code.
And if I use a language-specific timing loop, people will complain that the comparison is unfair. Timing routines are language-specific; C, for example, doesn't include a Standard function for obtaining time on the millisecond level. If the cross-language comparison is to have any meaning, the timing method must be consistent across all languages.
Import statements are only relevant at compile time - if you meant to talk about class use why did you bring up import statements?
Because, my dear fellow, the import statement implies usage; there is no point in importing something you do not use, so the existence of an import indicates the need to load classes at run-time. You are correct that import, in and of itself, is moot in terms of run-time performance -- but the specification of an import implies (almost always correctly) that something is being loaded at run-time.
The "technology testbed" probe, Deep Space 1, was programmed in Common LISP, I'm told. As I understand it, NASA engineers can (or could; I believe the probe is shut-down now) bring up a prompt and actively write/debug code while the probe was in space. Quite impressive; perhaps I need to brush-up on my LISP?;)
The Space Shuttle is programmed in Ada -- not terribly surprising given the origin of the shuttles in the late-70s, early 80s. Ada is a pretty good language, from what I know of it.
NASA uses a lot of different tools and languages -- which proves that intelligent life exists on Earth. No one programming language is the ultimate tool, and people who cling to one language for all solutions are ignorant at best and foolish as worst.
It is not valid to require that people rewrite parts of the run-time library in order to obtain effective speed. Run-time libraries are part of the product, just as much as code generators -- and if the library is slow, the product is slow. The best code generator in the world is of no avail if the runtime library is ineffective.
I will be adding an accuracy benchmark in the near future, as well as test that do not invoke transcendental functions.
I realize the trig functions are the limiter, and have said so since the benchmark was first released. That is one of the things I wanted to test, because the trig functions are intergral to the work I'm doing. I would be a fool to test without any trig functions, assume that Java is fast, and then have my app slow substantially when I apply real code. This is not an abstract benchmark like Whetstone; the code is actually trying to do something.
No it doesn't: import statements only affect compilation. Can you consult someone with some Java knowledge next time?
Okay, to be very specific: "import" does, indeed, increase run time if it brings in classes used by the program. Interpretted Java loads packages dyanmically at run-time -- or didn't you know that? Of course, an import for an unused package has no overhead.
Please don't harangue me about minor grammatical errors in the above; I'm tired, and pasted an old buffer (which I thought was a new buffer that I'd proofed. Damn these modern conveniences like cut-and-paste!;}
I'm going to watch Reign of Fire with my lovely wife now.
The original article generated an exceptional collection of
interesting and helpful of responses. In this section, I'll run down the
important points people made.
Note: This is a duplicate of a new section I added to the review; I'm posting it here for posterity.
An overwhelming number of people suggested that I include results for IBM
JDKs -- and I have. In fact, I've added results for Sun JDK 1.3.1_06, IBM JDK
1.4.0, and IBM 1.3.1 RC3. Adding these JVMs made a significant difference in
the results, cutting runtimes in half. On the other hand, almabench runs
as an infinite loop with Sun's 1.3.1 JVM (it starts, but never finishes). Note
that I recompiled almabench with the corresponding javac for each
JDK, so the JVMs were executing code generated by their corresponding compiler.
The problem with Java's performance is not my code or my lack of
Java skills -- what real problem is that Java 1.4
is slow. Both the Sun and IBM JVMs lost significant
performance in the move from 1.3.1 to 1.4, whether due to a new language
requirement or other factors. My faith in Java is severely shaken when
applications lose significant performance by upgrading to the
current release of Java.
Java 1.4 added many new features to the language and packages; however,
changing from version 1.3 to 1.4 should not double run-times! Nor am I comforted
by the problems of Sun's 1.3.1 server JVM.
Given the nature of the problem, my conclusions about Java stand (albeit slightly softened). By Sun's own definition, JDK 1.3.1 is obsolete; the fact that it performs better
than the most current JDK is indicative of a serious problem with Sun's
improvements to their language. Since Java 1.4.1 is what Sun is
promoting, so that is what I base my conclusion on. I can say that IBM's product
is superior, and have already set it is my default JDK. It's no wonder Sun is
upset about IBM usurping Java -- IBM is producing better tools.
Some people asked if my Java results were biased by the amount of time it
takes to load the JVM. I've tested several empty and near-empty applications; a
Hello, world program, for example loads and runs in less than
2/10ths of a second -- hardly significant. The start-up time increases with the
number of imports -- but almabench.java imports only one small class
package, java.math.*, which (in my tests) does not impose measurable
overhead.
I did not include any commercial Java compilers. Most, like the oft-cited Excelsior
JET, are Windows-only; this article is about Linux. I don't benchmark Visual
C++ or C# for the same reason (although I will look at Mono
and C# some time in the future). The free version of Borland C++ does not
include a complete optimizer, so I don't think it fits in this
review.
How do I know that the programs are producing the correct output? Each
program includes code to display results; I run the programs with I/O to ensure
that all calculations are being performed, then I comment out any header
inclusions, imports, and print statements for actual testing.
How am I timing the results? With the Linux time command. Table 3
reports the real value reported by time (the elapsed time of
execution.) Embedding timers in the actual code is fraught with problems; for
example, each language implements different time scales and abilities. I'm sure
someone will tell me that time is full of problems too, but it works for
me and is consistent across all programming languages.
Amid the barrage of Java-related comments, a few people actually noticed the
Fortran code. I am looking at other Fortran compilers for future updates. As for
GNU g77 -- I wrote the code in Fortran 95 because I find Fortran 77 to be
annoying. I wrote piles of Fortran 77 back in my CDC and VAX days, but these
days I'm writing for environments where Fortran 95 is more appropriate. Believe
it or not, Fortran 95 is a very clean, orderly language that eliminates many
Fortran 77 idiosyncrasies while adding features important for high-performance
coding.
Thanks for the pointer; however, I looked through the gcc documentation (on the gcc web site) for any options related to intrinsic functions. I didn't see what you allude to; which specific options did you use?
The end result is that I'd rather write in C or Java. C has the honesty of being strongly procedural, and if you wish to use some OO ideas where it makes sense, you can build up some limited OO constructs. Java is strongly OO, and if I don't like some of the limitations that come with Java (such as the floating point performance hit), I can link in C code with JNI. I'm fine with that. Each of these two approaches lets you build the tools you need when you need them.
Your comment matches one of the core points of my article. To quote from the article:
For a long time, I strongly advocated changing Java to support numerical work. In recent months, I've decided that I was mistaken. We already have several good general-purpose programming languages; programmers would best be served by the refinement of existing tools for their featured environments. Rather than create a vast gray-area of similar languages, we need tools for creating powerful and focused solutions.
Another of your comments:
C++, to me, lacks this overall paradigm, other than maybe "anything goes...no kidding".
I carry a Swiss Army knife with me everywhere (I saw too many episodes of MacGuyver, I think). It's a pretty good tool for most things, and it can do an awful lot, but it isn't as strong or useful as task-specific objects. The little scissors don't compare well to "real" scissors for serious paper-cutting; the saw works, but a chainsaw is better; the can opener functions but makes my hand shurt. None of the knife's tools are optimal -- but it's easier to carry one small red knife than it is for me to laod down with optimized gadgets.
Of course, building a house with a Swiss Army Knife is somewhat insane as well. (At least one guy did this, but I can't find a web reference at the moment).
In the end, it all comes down to a judgment call, which is likely to be different for different people.
I really wish Slashdot allowed embedded sounds, so my message could laugh like I just did.
On the other hand, I'm not sure I want to hear the audio for certain sites that appear frequently on Slashdot...;}
Ahem... back on topic. Do you actually expect Intel to produce a compiler that optimizes for processors produced by their biggest competitor?
AMD appears to have a tight relationship with the gcc team; purportedly, gcc has excellent code generation for Athlons. If and when I get an Athlon system, I'll look into this.
Looking at it briefly, the benchmark pretty much seems to test only straight line numerical code, and it seems to be dominated by calls to trigonometric functions.
Well, considering that what I was interested in, I must have done a good job on the benchmark!;) I think the title was pretty explicit about my intent.
All the differences between the results might be explained just by how trigonometric functions are handled. GNU C always translates them into function calls, and there are good reasons for that.
Indeed, I think this is the root of gcc's difficulties. It doesn't invalidate the test by any means; in fact, I've started discussing this very issue on the gcc mailing list. I'll be looking at some generated machine code this evening, which leads to your final comment...
I didn't see any analysis of the generated assembly language.
The point of using a high-level language is to avoid the need to play in assembly-land. I like programming for assemblers, but it isn't practical. The trick is picking the right level of hardware abstraction -- do you write for an assembler, a traditional programming language, or a high-level tool like Java or Python? Making decisions involves analyzing trade-offs between development time, run-time, and knowledge.
The g++ compiler is a pile of crap - it gave execution times more than double the times given by Intel C++. IMHO the difference is more than can reasonably be brushed off by saying that Intel know their own CPU better, or g++ has more features, or whatever the g++ fans will say. A difference as big as this means that - as an optimising compiler - g++ is broken.
The statement above is a load of dingo's kidneys, to paraphrase Douglas Adams.
Read my comparison of Intel and gcc for a broader view. No tool -- be it Fortran, or Java, or C++ -- can be accurately judged through the lens of a single test or review. Indeed, I have some applications where Intel performs poorly, due to a weak implementation of some STL classes. However, if I publish those results, I'll be beat up by the same nitwits who complain that a mathematical benchmark uses sin and cosine functions.
I like the Intel compilers for most tasks -- but I am also a very strong supporter of the GNU gcc project, which produces a high-quality, portable compiler that is the true backbone of Free Software. Without gcc, there would be no Linux, or KDE, or Mono, or whatever else people find so fascinating. I may have my quibbles with gcc, but it is NOT a "pile of crap."
What is the point of a portable programming language if you tie it to a specific platform? I don't see a Linux (or Mac, or Sparc, or...) version of Jet on their web site.
I've been writing and selling Java since it was first released, so I guess I have a bit of experience as well.
The HotJava VM has received an official "End of Life" notice from Sun; its technology is incorporated into the current "client" and "server" VMs (so I'm told).
Portability does conflict with native code generation. I agree that it is possible to be portable and support processor-specific JIT optimizations -- but as a practical matter, such is unlikely. Sun has X resource spread across at least 3 platforms (and their Linux JDK is not, IMHO, as impressive as it could be).
Intel's C++ and Fortran compilers produce very good code because Intel only cares about Intel processors. The gcc team, on the other hand, has to focus on dozens of architectures, including many within the so-called "ia32" realm. It is possible for gcc to produce code that matches Intel's -- but is it unlikely because of a lack of focus.
On the other hand, a broad view makes gcc (and Java) portable. And gcc can focus on optimizations for platforms (AMD, anyone?) that doe not have the luxury of vendor-created compilers.
As you point out, the "ia32" world is very fragmented -- a problem that will only grow as AMD and Intel continue to try and distinguish themselves with new processors. Supporting all these variations would bloat the JVM considerably, or require many different JVMs that could be a support nightmare for Sun.
...the conclusions you draw from this are that Java is a poor tool for a much broader set of applications and circumstances than your benchmark accurately measures.
If you read the conclusion in my article, you'll see the following paragraph:
This benchmark does not imply that Java is useless, or that Fortran is the world's best programming language, or that compiled languages are superior in all cases to byte-code interpreters. I would no more write a sophisticated graphical application in Fortran than I would analyze physics experiments in Java. Java is a terrific tool for building user interfaces; I enjoy working with Swing, and am in the midst of developing a large vertical-market database application in Java. Be that as it may, I'm exploring number crunching in this article, not buttons and mouse movements.
Back when I wrote reviews for print magazines (back when there were print magazines, that is), it was standard policy to limit reviews to the actually, shipping, commercial product. Demos often lack critical features (like optimizers) or are tuned for benchmark tests, so I've kept to that policy now that I write reviews for my own web site.
I own licenses for the Intel compilers, for example -- and, of course, gcc and Sun Java don't cost anything in the first place. I'm considering my options in this case; long gone are the days when a dozen Fortran or C compilers would arrive at my door for a magazine review. Heck, there aren't a dozen compiler vendors left...;)
My biggest surprise review was the excellent performance of the C++ code. In the past, C++ compilers introduced overhead -- even in code that isn't object-oriented.
Just because Java is a great tool doesn't mean it is the best tool for every job.
I only included the -server option because people would've bitched if I hadn't. I'm well-aware that the server JVM is optimized for persistent processes.
Everyone wants me to include the IBM JDK, so I'll do that later today.
I'm very interested in investigating the efficacy of functional programming for numerics. I've dabbled in Erlang and Haskell a bit, but haven't had an opportuntiy to study them in depth.
So many tools, so little time to understand them...
The title of the article CLEARLY states the limits of the tests:
Linux Number Crunching: Benchmarking Compilers and Languages for ia32
That doesn't leave much room for doubt as to what I'm testing.
I can add Solaris/SPARC numbers if anyone is interested.
Considering that I make a rather healthy living from Java development, I must be getting something right.;) And I've written at least one successful book on Java development, Java Algorithms (JDK 1.1-based and a bit old now). Ignore the J++ book I wrote for Microsoft; it was butchered by political concerns in Redmond.
For my clients, I create Swing-based GUI apps that have no numerical code whatsoever; the current one's running about 50K lines at the moment. Performance on such applications is database-bound, so Java's performance is a non-issue.
Almabench was not a standard Java program; it is a translation directly from the Fortran, and, as such, it has a few inconsistencies with the usual Java style. Here's the Catch-22: Had I customized the code to be very "Java-like", people would have complained that I was comparing apples and organges -- yet when I keep the code consistent across languages, people such as yourself argue that I should be more language-specific.
It's a good thing I put on my asbetstos underware this morning. I hadn't expected this to hit/.'s front page....
Given that numerical code often uses intrinsic functions like sin and atan2, testing those functions is quite valid, given the inability to chose my floating-point library under Java (byond changing the entire VM).
Using Object-Oriented constructs is no guarantee that a program is maintainable or even readable. I have seen some horrifying OOP code in my life, written by people so enamoured of syntax that they drown theircode in it.
In numerical applications, and extra 10% can be the difference between success and failure. I'm corresponding with a fellow who works in meteorology; his company uses commodity boxes to compete with government-funded monopolies. For him, the ability to gain 10% is crucial.
I am all in favor of object-oriented programming -- but my philosophy matches that of Bjarne Stroustrup, who refers to his language as a having "multiple paradigms." Use OO when it makes sense -- but use the right tool for the task at hand. C++ does not force you to use OOP when it doesn't make sense.
Many numerical applications make mroe sense when using short variable names (that match formulas in texts) and a function-based approach (again, matching mathematical idiom).
...the tools I have at hand. I have nothing against TowerJ, but can't test it if I don't own it.
As for Java, I made note of a lack of flags, which is different from complaining. The -O flag is no longer supported by Sun's JDK according to the documentation.
Slashdot Mirror
sincos() is not an ANSI/ISO Standard function. I am writing the applications to Standards for maximum portability; therefore, using sincos() is not an option.
Heck, I could just drop some inline assembler code into the C program -- but then, I'd be benchmarking hand-tuned assembler language, not Standard C code.
And if I use a language-specific timing loop, people will complain that the comparison is unfair. Timing routines are language-specific; C, for example, doesn't include a Standard function for obtaining time on the millisecond level. If the cross-language comparison is to have any meaning, the timing method must be consistent across all languages.
Because, my dear fellow, the import statement implies usage; there is no point in importing something you do not use, so the existence of an import indicates the need to load classes at run-time. You are correct that import, in and of itself, is moot in terms of run-time performance -- but the specification of an import implies (almost always correctly) that something is being loaded at run-time.
The "technology testbed" probe, Deep Space 1, was programmed in Common LISP, I'm told. As I understand it, NASA engineers can (or could; I believe the probe is shut-down now) bring up a prompt and actively write/debug code while the probe was in space. Quite impressive; perhaps I need to brush-up on my LISP? ;)
The Space Shuttle is programmed in Ada -- not terribly surprising given the origin of the shuttles in the late-70s, early 80s. Ada is a pretty good language, from what I know of it.
NASA uses a lot of different tools and languages -- which proves that intelligent life exists on Earth. No one programming language is the ultimate tool, and people who cling to one language for all solutions are ignorant at best and foolish as worst.
It is not valid to require that people rewrite parts of the run-time library in order to obtain effective speed. Run-time libraries are part of the product, just as much as code generators -- and if the library is slow, the product is slow. The best code generator in the world is of no avail if the runtime library is ineffective.
I will be adding an accuracy benchmark in the near future, as well as test that do not invoke transcendental functions.
I realize the trig functions are the limiter, and have said so since the benchmark was first released. That is one of the things I wanted to test, because the trig functions are intergral to the work I'm doing. I would be a fool to test without any trig functions, assume that Java is fast, and then have my app slow substantially when I apply real code. This is not an abstract benchmark like Whetstone; the code is actually trying to do something.
Okay, to be very specific: "import" does, indeed, increase run time if it brings in classes used by the program. Interpretted Java loads packages dyanmically at run-time -- or didn't you know that? Of course, an import for an unused package has no overhead.
Please don't harangue me about minor grammatical errors in the above; I'm tired, and pasted an old buffer (which I thought was a new buffer that I'd proofed. Damn these modern conveniences like cut-and-paste! ;}
I'm going to watch Reign of Fire with my lovely wife now.
The original article generated an exceptional collection of interesting and helpful of responses. In this section, I'll run down the important points people made.
Note: This is a duplicate of a new section I added to the review; I'm posting it here for posterity.
An overwhelming number of people suggested that I include results for IBM JDKs -- and I have. In fact, I've added results for Sun JDK 1.3.1_06, IBM JDK 1.4.0, and IBM 1.3.1 RC3. Adding these JVMs made a significant difference in the results, cutting runtimes in half. On the other hand, almabench runs as an infinite loop with Sun's 1.3.1 JVM (it starts, but never finishes). Note that I recompiled almabench with the corresponding javac for each JDK, so the JVMs were executing code generated by their corresponding compiler.
The problem with Java's performance is not my code or my lack of Java skills -- what real problem is that Java 1.4 is slow. Both the Sun and IBM JVMs lost significant performance in the move from 1.3.1 to 1.4, whether due to a new language requirement or other factors. My faith in Java is severely shaken when applications lose significant performance by upgrading to the current release of Java.
Java 1.4 added many new features to the language and packages; however, changing from version 1.3 to 1.4 should not double run-times! Nor am I comforted by the problems of Sun's 1.3.1 server JVM.
Given the nature of the problem, my conclusions about Java stand (albeit slightly softened). By Sun's own definition, JDK 1.3.1 is obsolete; the fact that it performs better than the most current JDK is indicative of a serious problem with Sun's improvements to their language. Since Java 1.4.1 is what Sun is promoting, so that is what I base my conclusion on. I can say that IBM's product is superior, and have already set it is my default JDK. It's no wonder Sun is upset about IBM usurping Java -- IBM is producing better tools.
Some people asked if my Java results were biased by the amount of time it takes to load the JVM. I've tested several empty and near-empty applications; a Hello, world program, for example loads and runs in less than 2/10ths of a second -- hardly significant. The start-up time increases with the number of imports -- but almabench.java imports only one small class package, java.math.*, which (in my tests) does not impose measurable overhead.
I did not include any commercial Java compilers. Most, like the oft-cited Excelsior JET, are Windows-only; this article is about Linux. I don't benchmark Visual C++ or C# for the same reason (although I will look at Mono and C# some time in the future). The free version of Borland C++ does not include a complete optimizer, so I don't think it fits in this review.
How do I know that the programs are producing the correct output? Each program includes code to display results; I run the programs with I/O to ensure that all calculations are being performed, then I comment out any header inclusions, imports, and print statements for actual testing.
How am I timing the results? With the Linux time command. Table 3 reports the real value reported by time (the elapsed time of execution.) Embedding timers in the actual code is fraught with problems; for example, each language implements different time scales and abilities. I'm sure someone will tell me that time is full of problems too, but it works for me and is consistent across all programming languages.
Amid the barrage of Java-related comments, a few people actually noticed the Fortran code. I am looking at other Fortran compilers for future updates. As for GNU g77 -- I wrote the code in Fortran 95 because I find Fortran 77 to be annoying. I wrote piles of Fortran 77 back in my CDC and VAX days, but these days I'm writing for environments where Fortran 95 is more appropriate. Believe it or not, Fortran 95 is a very clean, orderly language that eliminates many Fortran 77 idiosyncrasies while adding features important for high-performance coding.
Thanks for the pointer; however, I looked through the gcc documentation (on the gcc web site) for any options related to intrinsic functions. I didn't see what you allude to; which specific options did you use?
Your comment matches one of the core points of my article. To quote from the article:
Another of your comments:
I carry a Swiss Army knife with me everywhere (I saw too many episodes of MacGuyver, I think). It's a pretty good tool for most things, and it can do an awful lot, but it isn't as strong or useful as task-specific objects. The little scissors don't compare well to "real" scissors for serious paper-cutting; the saw works, but a chainsaw is better; the can opener functions but makes my hand shurt. None of the knife's tools are optimal -- but it's easier to carry one small red knife than it is for me to laod down with optimized gadgets.
Of course, building a house with a Swiss Army Knife is somewhat insane as well. (At least one guy did this, but I can't find a web reference at the moment).
In the end, it all comes down to a judgment call, which is likely to be different for different people.
I really wish Slashdot allowed embedded sounds, so my message could laugh like I just did.
On the other hand, I'm not sure I want to hear the audio for certain sites that appear frequently on Slashdot... ;}
Ahem... back on topic. Do you actually expect Intel to produce a compiler that optimizes for processors produced by their biggest competitor?
AMD appears to have a tight relationship with the gcc team; purportedly, gcc has excellent code generation for Athlons. If and when I get an Athlon system, I'll look into this.
Well, considering that what I was interested in, I must have done a good job on the benchmark! ;) I think the title was pretty explicit about my intent.
Indeed, I think this is the root of gcc's difficulties. It doesn't invalidate the test by any means; in fact, I've started discussing this very issue on the gcc mailing list. I'll be looking at some generated machine code this evening, which leads to your final comment...
The point of using a high-level language is to avoid the need to play in assembly-land. I like programming for assemblers, but it isn't practical. The trick is picking the right level of hardware abstraction -- do you write for an assembler, a traditional programming language, or a high-level tool like Java or Python? Making decisions involves analyzing trade-offs between development time, run-time, and knowledge.
The statement above is a load of dingo's kidneys, to paraphrase Douglas Adams.
Read my comparison of Intel and gcc for a broader view. No tool -- be it Fortran, or Java, or C++ -- can be accurately judged through the lens of a single test or review. Indeed, I have some applications where Intel performs poorly, due to a weak implementation of some STL classes. However, if I publish those results, I'll be beat up by the same nitwits who complain that a mathematical benchmark uses sin and cosine functions.
I like the Intel compilers for most tasks -- but I am also a very strong supporter of the GNU gcc project, which produces a high-quality, portable compiler that is the true backbone of Free Software. Without gcc, there would be no Linux, or KDE, or Mono, or whatever else people find so fascinating. I may have my quibbles with gcc, but it is NOT a "pile of crap."
What is the point of a portable programming language if you tie it to a specific platform? I don't see a Linux (or Mac, or Sparc, or...) version of Jet on their web site.
I've been writing and selling Java since it was first released, so I guess I have a bit of experience as well.
The HotJava VM has received an official "End of Life" notice from Sun; its technology is incorporated into the current "client" and "server" VMs (so I'm told).
Portability does conflict with native code generation. I agree that it is possible to be portable and support processor-specific JIT optimizations -- but as a practical matter, such is unlikely. Sun has X resource spread across at least 3 platforms (and their Linux JDK is not, IMHO, as impressive as it could be).
Intel's C++ and Fortran compilers produce very good code because Intel only cares about Intel processors. The gcc team, on the other hand, has to focus on dozens of architectures, including many within the so-called "ia32" realm. It is possible for gcc to produce code that matches Intel's -- but is it unlikely because of a lack of focus.
On the other hand, a broad view makes gcc (and Java) portable. And gcc can focus on optimizations for platforms (AMD, anyone?) that doe not have the luxury of vendor-created compilers.
As you point out, the "ia32" world is very fragmented -- a problem that will only grow as AMD and Intel continue to try and distinguish themselves with new processors. Supporting all these variations would bloat the JVM considerably, or require many different JVMs that could be a support nightmare for Sun.
You incorrectly assert:
If you read the conclusion in my article, you'll see the following paragraph:
Back when I wrote reviews for print magazines (back when there were print magazines, that is), it was standard policy to limit reviews to the actually, shipping, commercial product. Demos often lack critical features (like optimizers) or are tuned for benchmark tests, so I've kept to that policy now that I write reviews for my own web site.
I own licenses for the Intel compilers, for example -- and, of course, gcc and Sun Java don't cost anything in the first place. I'm considering my options in this case; long gone are the days when a dozen Fortran or C compilers would arrive at my door for a magazine review. Heck, there aren't a dozen compiler vendors left... ;)
Thank you.
My biggest surprise review was the excellent performance of the C++ code. In the past, C++ compilers introduced overhead -- even in code that isn't object-oriented.
Just because Java is a great tool doesn't mean it is the best tool for every job.
I only included the -server option because people would've bitched if I hadn't. I'm well-aware that the server JVM is optimized for persistent processes.
Everyone wants me to include the IBM JDK, so I'll do that later today.
I'm very interested in investigating the efficacy of functional programming for numerics. I've dabbled in Erlang and Haskell a bit, but haven't had an opportuntiy to study them in depth.
So many tools, so little time to understand them...
The title of the article CLEARLY states the limits of the tests: Linux Number Crunching: Benchmarking Compilers and Languages for ia32 That doesn't leave much room for doubt as to what I'm testing. I can add Solaris/SPARC numbers if anyone is interested.
Considering that I make a rather healthy living from Java development, I must be getting something right. ;) And I've written at least one successful book on Java development, Java Algorithms (JDK 1.1-based and a bit old now). Ignore the J++ book I wrote for Microsoft; it was butchered by political concerns in Redmond.
For my clients, I create Swing-based GUI apps that have no numerical code whatsoever; the current one's running about 50K lines at the moment. Performance on such applications is database-bound, so Java's performance is a non-issue.
Almabench was not a standard Java program; it is a translation directly from the Fortran, and, as such, it has a few inconsistencies with the usual Java style. Here's the Catch-22: Had I customized the code to be very "Java-like", people would have complained that I was comparing apples and organges -- yet when I keep the code consistent across languages, people such as yourself argue that I should be more language-specific.
It's a good thing I put on my asbetstos underware this morning. I hadn't expected this to hit /.'s front page....
Given that numerical code often uses intrinsic functions like sin and atan2, testing those functions is quite valid, given the inability to chose my floating-point library under Java (byond changing the entire VM).
Using Object-Oriented constructs is no guarantee that a program is maintainable or even readable. I have seen some horrifying OOP code in my life, written by people so enamoured of syntax that they drown theircode in it.
In numerical applications, and extra 10% can be the difference between success and failure. I'm corresponding with a fellow who works in meteorology; his company uses commodity boxes to compete with government-funded monopolies. For him, the ability to gain 10% is crucial.
I am all in favor of object-oriented programming -- but my philosophy matches that of Bjarne Stroustrup, who refers to his language as a having "multiple paradigms." Use OO when it makes sense -- but use the right tool for the task at hand. C++ does not force you to use OOP when it doesn't make sense.
Many numerical applications make mroe sense when using short variable names (that match formulas in texts) and a function-based approach (again, matching mathematical idiom).
...the tools I have at hand. I have nothing against TowerJ, but can't test it if I don't own it. As for Java, I made note of a lack of flags, which is different from complaining. The -O flag is no longer supported by Sun's JDK according to the documentation.