Java Faster Than C++?

jg21 writes "The Java platform has a stigma of being a poor performer, but these new performance benchmark tests suggest otherwise. CS major Keith Lea took time out from his studies at student at Rensselaer Polytechnic Institute in upstate New York's Tech Valley to take the benchmark code for C++ and Java from Doug Bagley's now outdated (Fall 2001) "Great Computer Language Shootout" and run the tests himself. His conclusions include 'no one should ever run the client JVM when given the choice,' and 'Java is significantly faster than optimized C++ in many cases.' Very enterprising performance benchmarking work. Lea is planning next on updating the benchmarks with VC++ compiler on Windows, with JDK 1.5 beta, and might also test with Intel C++ Compiler. This is all great - the more people who know about present-day Java performance, the better.""

Comments

Um, it's online

[JoshuaDFranklin]

If you want, you can read the actual author's piece instead of a news story about it:

The Java is Faster than C++ and C++ Sucks Unbiased Benchmark

Re:Um, it's online

[Anonymous Coward]

It also does stupid things. Like this:

int c = 0;
for (int i=n; i>0; i--) {
sprintf(buf, "%d", i);
if (X[strdup(buf)]) c++;
}

When this would have worked just fine:

int c = 0;
for (int i=n; i>0; i--) {
if (X[atoi(i)]) c++;
}

The code is dumb, yes, but you are wrong, nonetheless. That code won't even compile. I think you meant itoa(), which would be about the same as sprintf in terms of functionality.

That for() loop is not equivalent to the Java code's for loop, either. In the java code, he used

if (ht.containsKey(Integer.toString(i, 10))) c++;

which means that he should have used

if (X.count(somestringrepofi)) c++;

X[somestringrepofi] will create an entry for the key if it is not found, making it very different from containsKey().

Re:Um, it's online

[GooberToo]

The more I read the code, it sure is starting to look more and more like an apples and oranges comparison; which is usually what happens when people do java benchmarks.

As typically observed, I'm seeing the benchmarks take serious advantage of java's GC mechanism, whereby, they never pay the piper. With C++, the piper is constantly being paid. All of the benchmarks which allocate objects and then delete them, are therefore, invalid. To be fair, I think, you either need to add a System.gc() line in the java code where C++ is doing its deletes or you need to implement your own new and delete operators to function more in line with what java is doing. Until you do either one of those things, the comparisons where objects are being allocated and deallocated are invalid. And frankly, I'm still not sure adding System.gc() is even fair, on either side. The reason is, calling System.gc() simply hints that it's a good idea to collect. There is nothing which requires the collection to take place. So, technically, the call could still be many times faster. On the other hand, I don't know enough about how they handle their gc-hint logic nor am I aware of exactly how much overhead is involved in the actual collection process. If it occurs too often, the shift in workload may be too unfair. Nonetheless, it's a point of very serious contention.

Just for kicks, I modified objinst.java with, "if( i%(n/1000) == 0 ) System.gc() ;", on the lines that the C++ code had it's delete. When I timed it, it was over twice as slow as the C++ code (24+s vs 55+s). Worse, when I ran it with a 1:1 ratio of delete:System.gc(), I simply got tired of waiting, having waited over 5 minutes.

So, basically, I'm not nearly as impressed as I first was. Simplistically, it's starting to look like a serious apples and oranges comparison. Elsewhere, you can find other examples of just plain bad code. Where again, with correct C++ code, came in about twice as fast at the Java code, whereby, more optimizations were still possible with the C++ code.

So, it looks like we're seeing a combination of things here. Looks like a combination of bad code, ideal corner cases for java's hot spot, and invalid comparisons with memory allocation.

Sadly, I'm once again seriously disappointed in java.

Re:my arse

[kaffiene]

*sigh* have you people never heard of runtime optimisations? There are some things you can optimise at runtime (like runtime constants) which are *impossible* to optimise at compile time.

This whole "x is written in y, so x can't be faster than y" rubbish is just that - rubbish.

been there, done that

[Anonymous Coward]

1) javac (Sun's Java compiler) is written in Java. You can even access it programmatically at runtime if you really want to.

2) While it's not an id game, IL2 Sturmovik is a critically-acclaimed fight simulator that was written almost entirely in Java.

Re:He used g++ to compare C++ with Java...

[ky11x]

g++'s goal is modularity for ease of porting in cross-platform cross-compiling. aggressive optimization is not one of its strengths. the point of such benchmarks is really not a language comparison, but a comparison between the code generated by the most optimized compilers for that language on a specific platform. Using g++ for this simply causes the study to lose credibility

Re:He used g++ to compare C++ with Java...

[mwillis]

Intel gives their c++ compiler away free for non-commercial hobbyist use on linux.

The windows version has a free trial that runs for 30 days.

Try it. See if it makes a difference. If it doesn't, torch it. If you find it makes your critical code run 2x faster, then... have a look at what a computer that runs 2x faster will cost you, and then decide what to do.

Java performance "truths" change over time

[eduardodude]

Check out this recent IBM Developerworks article which looks at how modern JVMs handle allocation and garbage collection.

Some very surprising tidbits there. For instance:

"Performance advice often has a short shelf life; while it was once true that allocation was expensive, it is now no longer the case. In fact, it is downright cheap, and with a few very compute-intensive exceptions, performance considerations are generally no longer a good reason to avoid allocation. Sun estimates allocation costs at approximately ten machine instructions. That's pretty much free -- certainly no reason to complicate the structure of your program or incur additional maintenance risks for the sake of eliminating a few object creations."

Read the article for an excellent nuts-and-bolts analysis of many current performance considerations. From the posts in this thread, it's pretty clear a lot of people haven't looked into what's actually done in a server JVM these days.