Slashdot Mirror
Java Faster Than C++?
jg21 writes "The Java platform has a stigma of being a poor performer, but these new performance benchmark tests suggest otherwise. CS major Keith Lea took time out from his studies at student at Rensselaer Polytechnic Institute in upstate New York's Tech Valley to take the benchmark code for C++ and Java from Doug Bagley's now outdated (Fall 2001) "Great Computer Language Shootout" and run the tests himself. His conclusions include 'no one should ever run the client JVM when given the choice,' and 'Java is significantly faster than optimized C++ in many cases.' Very enterprising performance benchmarking work. Lea is planning next on updating the benchmarks with VC++ compiler on Windows, with JDK 1.5 beta, and might also test with Intel C++ Compiler. This is all great - the more people who know about present-day Java performance, the better.""
The Java is Faster than C++ and C++ Sucks Unbiased Benchmark
Correct link
Here's some kindling...
vi is better than emacs
bsd is better than linux
gnome is better than kde
.
.
.
anything else?
oh yeah...
my dad can beat up your dad.
And you smell funny.
Rhymes that keep their secrets will unfold behind the clouds.There upon the rainbow is the answer to a neverending story
I just happened across Sun's FAQ about the -client and -server settings .
...on x86? Please! Wake me up when someone who knows enough about C++ to pick a decent x86 compiler runs some benchmarks.
Doesn't it make you feel good to know that our freedoms are protected by politicans, lawyers and journalists.
He claims you should use the server JVM instead, stating that it is faster but slower to startup and consumes more memory.
Java and C++ are language. Languages aren't "faster" or "slower", but compilers for them might be. I find it somewhat underhanded to put the languages in the header when it's really comparing compilers.
Not to mention, inter-language compiler benchmark[et]ing is notoriously difficult to get 'right'. The programs tested are often stupid (doesn't do anything meaningful), or constructed by a person with more skill/bias for one language than the other.
It's been ages since I've programmed in C++, but it's good to know see these favorable comparisons. I think about the Struts/Hibernate/Oracle applications I write today and shudder when I imagine what how difficult it would be to have to develop web applications in C++. C++ will be around forever and certainly has its place, but long live Java.
I think you're missing the point. I bet 19 seconds of that execution time was the start-up and shutdown of the virtual machine. As the program gets bigger and bigger, this becomes less and less significant.
Oh... and as of Java1.5, Swing apps can now be skinned to look however you'd like them to.
File under 'M' for 'Manic ranting'
There seem to be some unanswered questions here..
- How equivalent were the benchmarks? Where they programmed in an optimum way for their respective compilers and libraries? I'm sure the java ones were.. what about the C++ ones? The author states he doesn't understand G++ very well.
G++ is also known to not produce the best results.
"I rant it with -O2"
My guess is many of the tests were not implemented properly in c++.
The main clue would be this... I can understand java having better than expected performance.. but there is no way I can accept that java is that much FASTER than properly done C++... it doesn't make any sense.
I looked at his results page quite extensively, but failed to find a good analysis/justification of the results. Just saying that the Server JVM is better than the Client JVM is *not* enough.
I want to know where the C++ overhead comes from, which Java manages to avoid - does the JVM do better optimization because it is given a better intermediate code (bytecode)? Is it better at doing back/front end optimizations (unlikely given gcc's maturity).
I tried to look for possible discrepancies in the results, but the analysis will definitely take more time - and I think it's the job of the experimenter to do a proper analysis of the results. Liked his choice of benchmarks though.
An Indian-American Hindu committed to non-violent thought/speech/action alarmed by the global explosion of radical Islam
I’m sorry but someone who says "I've never been very good at decoding GCC's error messages" is not competent enough to perform performance comparison. This performance test is a shame and shouldn’t be taken seriously.
-----
One is born into aristocracy, but mediocrity can only be achieved through hard work.
That's Great! I can't figure out GCC's error messages, but I offer definitive proof that Java is faster than C++. Nice.
Java isn't "emulation". Modern JVMs use a JIT (just-in-time compiler) to translate bytecode instructions into pure binary assembled object code just before it is reached in the program (hence "just in time"). This is cached, so the next time that particular code is executed, it will run at full assembler speed.
Something I've often wondered is whether this caching could be persistent, i.e. be kept between runs of the JVM. Eventually, the entire program would be translated to pure assembler with the cost of translation largely amortised across many sessions. You still keep the safety, cross platform compatibility and ease-of-programming of a bytecode language (i.e. Java, C#) but you get the bonus of the cached object code being just as fast, even during startup and shutdown.
I care that Java is an inconvenient pain to develop in and use. I care that I have to start a mini-OS just to run a Java program. I care that the language is under the control of one vendor. I care that the 'intialization == resource allocation' model doesn't work in Java. I care that the type system is too anemic to support some of the more powerful generic programming constructs. I care that I don't get a choice about garbage collection. I care that I don't get to fiddle bits in particular memory locations, even if I want to.
I think Java is highly overrated. I would prefer that a better C++ (a C-like memory model, powerful generic programming, inheritance, and polymorphism) that lacked C++'s current nightmare of strangely interacting features and syntax.
I use Python when I don't need C++s speed or low-level memory model, and I'm happier for it. It's more flexible than Java, much quicker to develop in, and faster for running small programs. Java doesn't play well with others, and it was seemingly designed not to.
Besides, I suspect that someone who knew and like C++ really well could tweak his benchmarks to make C++ come out faster again anyway. That's something I've noticed about several benchmarks that compare languages in various ways.
Need a Python, C++, Unix, Linux develop
From methcall.cpp:
int
main(int argc, char *argv[]) {
int n = ((argc == 2) ? atoi(argv[1]) : 1);
bool val = true;
>> Toggle *toggle = new Toggle(val);
for (int i=0; i<n; i++) {
val = toggle->activate().value();
}
cout << ((val) ? "true" : "false") << endl;
delete toggle;
val = true;
NthToggle *ntoggle = new NthToggle(val, 3);
for (int i=0; i<n; i++) {
val = ntoggle->activate().value();
}
cout << ((val) ? "true" : "false") << endl;
>> delete ntoggle;
return 0;
}
Why allocate and deallocate an object within the scope of a function? Well, in C++, there's no reason, so this is bad code. You can just declare it as a non-pointer and it lives in stack space. But guess what? You can't do that in Java: all objects are allocated on the heap.
That, and using cout instead of printf, are probably why this is slower than the "equivalent" Java.
-_-_-
There are 0x40000000 types of people: those who understand 32-bit IEEE 754 floating point, and those who don't.
*sigh* have you people never heard of runtime optimisations? There are some things you can optimise at runtime (like runtime constants) which are *impossible* to optimise at compile time.
This whole "x is written in y, so x can't be faster than y" rubbish is just that - rubbish.
First, it's been known for awhile that Java is a poor performer when writing to the console, for whatever reason. Second, your Java timing probably include the time to startup the VM (not that this is wrong).
If you have a program that runs for awhile (so the startup time is small compared to the time the program takes to run), and does not do intensive output to the console, then Java is a reasonable choice in my opinion. Combined with SWT, Java applications can be quite snappy (see Eclipse, Azureus), and the end user will probably never know the difference.
- shadowmatter
Now, regarding java performance ... Java isn't slow per se, JVMs and some apis (most notably swing) are. Furthermore, JVMs usually have a slow startup, which gave java a bad name (for desktop apps startup matters a lot, for servers it's hardly a big deal). Java can be interpreted, but it doesn't have to be so (all "modern" JVMs compile to binary code on the fly)
Bytecode-based environments will, IMNSHO, eventually lead to faster execution than with pre-compilation. The reason is profiling and specialized code generation. With today's processors, profiling can lead sometimes to spectacular improvements - as much as 30% performance improvements on Itanium for instance. Although Itanium is arguably dead, other future architectures will likely rely on profiling as well. If you don't believe me, check the research in processor architecture and compiling.
The big issue with profiling is that the developper has to do it, and on a dataset that's not necessarily similar to the user's input data. Bytecode environments can do this on-the-fly, and with very accurate data.
The Raven
This would be much more meaninful if you had used fputs() instead of write() in the C version.
As for "several orders of magnitude," I call bullshit. There's no way in hell the standard C++ IO functions are hundreds of times slower unless they're extremely badly written. Which leads me to another reason why this example sucks: there can be different implementations of the standard libraries.
In conclusion, this "comparison" is a stinky pile of shit, and should be ignored. And it's not even on topic, since it doesn't have a Java version.
1) javac (Sun's Java compiler) is written in Java. You can even access it programmatically at runtime if you really want to.
2) While it's not an id game, IL2 Sturmovik is a critically-acclaimed fight simulator that was written almost entirely in Java.
Out of the box Swing is amazingly ugly. The people choosing default colors at Sun could well be substituted by a randomizer without a difference in results. I mean, who was the genius who thought purple bars in a menu were cute?
Now, when you need to change that quickly and without much overload, there are ways. A little known global HashTable called UIDefaults lets you change just about everything on the visual interface without having to write your own LookAndFeel (which you obviously can do too, for very large projects). You can have your scrollbars, menus, etc in any colour, size and shape, using any font. You can easily change all default colours without having to set every control. After a while the ugliness ceases to be a problem.
I'd be interested in comparing the speed of the native code generated by gjc to the that of JVM.
-josh
I am starting on a new standalone server now doing something different, but I am going to stick with Java, and will be happy to see what 1.5 does for me.
But I have seen Java run slow before, and I will tell you this: in every instance it is due to someone writing some needlessly complicated J2EE application with layer upon bloaty layer of indirection. All the wishing in the world won't make one of those behemoths run fast, but it's not fair to blame Java. Maybe blame Sun for EJB's and their best practices, or blame BEA for selling such a pig.
Stuff I like in the Java world:
All you programmers that say you can do anything in Java/C#/etc are terrible.
Actually you can do most anything in those languages. Although for performance, and desgin reasons you may wish to use something else depending on the application.
You have no respect for code. Learn assembly and then we'll talk.
I know assembly, and fun as it is, it isn't well suited for high level projects where code reuse and mantainability are important. By the way, I have no respect for someone who knows assembly and thinks it is difficult. It isn't. And it certainly isn't graceful or elegant, but I love it all the same.
Seeing things like this:is hurting my eyes.
This page has more information about this horrible malfeature.
________
Entranced by anime since late summer 2001 and loving it ^_^
The article says he used GCC to compile the C++ versions, but GCC produces code that isn't nearly as good as the Intel compiler for example. (Here, but no good if you don't subscribe)
A lot of the test results are close, and I think a different compiler would change the outcome.
I believe that Sun's javac bootstraps itself just like gcc does. That would be your java compiler written in Java.
_Jikes_, OTOH, is written in C++. But that's not really the official Java compiler by a long shot.
Your second requirement is absolutely bizarre. Does this mean you're not taking languages like Lisp, Prolog, Python, and Perl seriously, too? Those are all very nice languages for doing stuff in, but I'm pretty sure id never wrote a 3D engine in them. In fact, I was under the impression that id has never written a 3D engine in C++, either. Should we not take C++ seriously?
IMHO: The measure of a language is not how easy it is to write an arbitrary application in it. It's how easy it is to write something for which the language was designed to do.
-Erwos
Plausible conjecture should not be misrepresented as proof positive.
Here is my experience with C++ vs. Java: At my company, we had a specialized image viewing program. The original program was written in C++ years ago, and performance sucked even on modern machines. It probably had a dozen man-years of time in it. We decided to re-write it in java.
We knew java in theory should be worse than C++ at manipulating large blocks of raw data, so we spent some time architecting, prototyping, and profiling java. We quickly learned the limitations and strengths.
The result? After 4 engineers worked for 6 months, we had a program that was rock solid, had more features, had a modern UI, and was WAY faster. Night and day; the old program felt like a hog, and the new program was zippy as anything. And the new code is fewer lines, and (in our opinion) way more maintainable. Since the original release, we've added severeal new features after day or two of work; the same features never would have happened on the old version, because they would have been too risky.
So the question is this? Could we have re-written or refactored the C++ program and gotten the same speed benefits? No doubt, such a thing is possible. But we are all convinced there is NO WAY we could have done it with as little effort. The C++ version would have taken longer to write and debug.
So, if the JIT computes Hot/Cold Paths, and optimizes the Hot paths, then it should work better and better on successive runs (as more and more profiling information is gathered). On the other hand, there will be cases where it performs worse, as profiles are gathered for specific inputs.
That means that if an average of say 5 runs (on the same input) is taken, it will have an unfair advantage (since gcc did NOT have the advantage of profiling information (see man gprof or similar)). Using Profiling as an optimization tool is *always* unfair unless both tools are provided with the advantage of the same profiling information. This is a valid question for the author then: if the JIT/javac/JVM uses profiling information, gcc should too, for fair comparison.
PS: I have seen this argument being made by my Professor and audiences at compiler conferences.
An Indian-American Hindu committed to non-violent thought/speech/action alarmed by the global explosion of radical Islam
A few examples
1) Java has bounds checking for arrays, C++ doesn't. This is specified in the language. This affects performance.
2) Java has garbage collection, C++ doesn't. This is specified in the language. This affects performance.
Also, the specification of Java says that it should be compiled to byte code and executed in a JVM.
So the "language" certainly affects performance.
The Internet is full. Go Away!!!
Why did he use only -O2?
-O3 adds function inlining and register renaming.
Also, some of the code doesn't look too much of a test of the language, but more of a test of the libraries. Both versions of hash rely on the library implementations, and it looks like hash.cpp does an extra strdup that the java version doesn't. I don't know either of the hash libraries well enough, but I don't see why this significant slowdown would be necessary in the gcc version.
HIV Crosses Species Barrier... into Muppets
Gcc is designed for compatibility with a wide range of architectures, and is not optimized for a single one. He also (apparantly) used stock glibc from Red Hat. And only one "test", the method call test, showed java to be a real winner. And even then, it's server-side Java, which is meaning less when you talk about it as a day-to-day dev language (ie; creating standalone client-side apps).
Intel's (heavily optimized) C++ compiler should be a damn sight faster, and so should VC++.
This "comparison" is so limited in scope and meaning, that this writeup should be considered a troll.
Hell, read his lead-in:
Ie; I set out to prove Java is teh awesome and c++ is teh suck!
If anything it proves something I've known intuitively for a long time. gcc does not produce x86 code that's as fast as it could be. That's a trade-off for it being able to compile for every friggin cpu under the sun.
I can't wait till RMS takes personal offense and goes on the attack.
I don't need no instructions to know how to rock!!!!
You might think that the two are the same, but the two settings actually make a visible impact if you're running on a multi-processor system. Most notably, the garbage collector and locking primitives are implemented differently.
The Raven
When C/C++ uses profiling it will only ever produce one "best case" compilation for a given function.
With any JIT system you have the opertunity to use the profiling information from a given "window" of the execution so there is the possibility of having more than one compilation for a function.
Now, I do not know how sophisticated JAVA JIT compilers have become but this is one area where JIT will have an upper hand over a static compiler.
OTOH, these tests do not look like there is enough significant variation in the execution path for profiling to make a large difference.
anything else?
Yeah, Kuro5hin is better than Slashdot.
No, I'm not being ironic.
I'm tired of some programmers expecting to be worshipped because they know assembly.
Assembly isn't all that.
For some uses, it is the right tool. For 99.9%+ it most definitely isn't.
The Internet is full. Go Away!!!
You are once again spouting the tired old line that Sun is the master of Java. Not at all true, Java's fate is controlled by a whole host of companies - including IBM. Take a look at the reality of Java platform evolution at the Java Community Process web site.
It's a standards body just like any other, just more open.
P.S. - Aside from that gripe being wrong, I agree with the other poster that you should look into Objective-C to address other issues. Look for "GnuSTEP" for cross-platform objective C GUI work. It's just nicer to use on a Mac as they have very good tools (though in fairness I have never looked at what GnuSTEP tools might be around, I just can't imagine them being quite as good as the tools Apple has sunk so much effort into!).
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Reviewing the console log, we find that when java programs were tested with a large number of iterations, Java only performed better on one test.
I know that Java has many strengths, but speed isn't one of them. Looking at the results, we see the g++ runtimes are much more consistent than those of Java - on some tests, the Java Server is faster than the client by a factor of 20!? How could a programmer code without having any realistic expectation of the speed of his code. How embarrassed would you be to find that your "blazingly fast" app ran slower than molasses on the client machine, for reasons yet unknown?
When it comes to speed, compiled languages will always run faster than interpreted ones, especially in real-world applications.
But discussions of language speed are a moot point. What this really tested was the implementation, not the language. Speed is never a criteria upon which languages are judged - a "slow" language can always be brought up to speed with compiler optimizations (with a few exceptions). I suspect that if C++ was interpreted, and Java compiled, we'd see exactly the opposite results.
In short, the value of a language consists not in how fast it runs, but in what it enables the programmer to do.
The society for a thought-free internet welcomes you.
I just wrote two programs to count to 1 billion. The one written in C took 2.4 seconds, the one written in assembly took 0.85 seconds. Wow, assembly is so much faster. My in-depth analysis of these two languages has shown once and for all that all us high-level language suckers need to get back to coding in assembly and quit this HLL foolishness.
"No one likes working in a hamster wheel, and your shop smells of cedar shavings from here." - TaleSpinner
Whew, there's seems to be a lot of denial running through this thread. "An interpreted language just can't possibly be as fast or faster as a compiled language! So I just don't care what the empirical results say, no matter how badly or well done they are, it just can't possibly be!"
I think some of you are overlooking the fact that a VM running byte code is capable of doing optimizations that a compiled language just can't possibly do. A compiled language can only be optimized at compile time. Those optimizations may be very sophisticated, but they can never be any better than an educated guess about what's going to happen at runtime.
But a VM is capable of determining exactly what is happening at runtime; it doesn't have to guess. And thus it is able to optimize those sections of code that really are, in true fact, impacting performance most severely. In can do this by compiling those sections to machine code, thus exploiting precisely the advantage that a compiled language is alleged to have by its very nature. And other kinds of optimizations, the kind that a compiler traditionally does, can be performed on those sections as well.
Of course there are scenarios where runtime optimization doesn't win much, for example in a program that is run once on a small amount of data and then stopped, so that the profiler doesn't get much useful info to work with. This is why Java is more likely to have benefits like this in long-running server processes.
And of course a conscientious C++ programmer will run a profiler on his program on a lot of sample data, and go about optimizing the slowest parts. A conscientious Java programmer should do that too. But an interpreted language has the advantage that the VM can do a lot of that work for you, and always does it at runtime, which is when it really counts.
Always keep a sapphire in your mind
Why does the example use a recursive fibonnaci sequence algorithm? It's so slow, and the runtime is dominated by the function call time.
./fib_recurse 40
./fib_for_loop 40
For example:
[bdr@arthurdent tests]$ time
165580141
real 0m3.709s
user 0m3.608s
sys 0m0.005s
time
165580141
real 0m0.006s
user 0m0.002s
sys 0m0.002s
I think a lot of these benchmarks are showing that the Hotspot optimiser is very good at avoiding function call overheads.
Here is an excerpt from the article for this story: Lea used G++ (GCC) 3.3.1 20030930 (with glibc 2.3.2-98) for the C++, with the -O2 flag (for both i386 and i686). He compiled the Java code normally with the Sun Java 1.4.2_01 compiler, and ran it with the Sun 1.4.2_01 JVM. He ran the tests on Red Hat Linux 9 / Fedora Test1 with the 2.4.20-20.9 kernel on a T30 laptop. The laptop "has a Pentium 4 mobile chip, 512MB of memory, a sort of slow disk," he notes.
What this shows is that GCC's implementation of C++ is slower than an interpreted language like Java. This does not show that C++ is slower than Java.
----
Notes on Stuff
I just went through and tested the hash2 benchmark and found that I was correct. The C++ version slaughters the Java version (even in "server" mode). This is completely different than what this dude's page shows.
Here is the "correct" code for hash2.cpp:
#include <stdio.h>
#include <iostream>
#include <ext/hash_map>
using namespace std;
using namespace __gnu_cxx;
struct eqstr {
bool operator()(const char* s1, const char* s2) const {
return strcmp(s1, s2) == 0;
}
};
struct hashme
{
size_t operator()(const char* s) const
{
size_t i;
for (i = 0; *s; s++)
i = 31 * i + *s;
return i;
}
};
int
main(int argc, char *argv[]) {
int n = ((argc == 2) ? atoi(argv[1]) : 1);
char buf[16];
typedef hash_map<const char*, int, hashme, eqstr> HM;
HM hash1, hash2;
for (int i=0; i<10000; i++) {
sprintf(buf, "foo_%d", i);
hash1[strdup(buf)] = i;
}
for (int i=0; i<n; i++) {
for (HM::iterator k = hash1.begin(); k != hash1.end(); ++k) {
hash2[(*k).first] += k->second;
}
}
cout << hash1["foo_1"] << " " << hash1["foo_9999"] << " "
<< hash2["foo_1"] << " " << hash2["foo_9999"] << endl;
}
The ratio of people to cake is too big
I could not run the test for 10M, but ran it for up to 1M. 541 milliseconds in one case, 280 in the other. Here's the code I used (I had to modify the timing cause I'm running XP):The only difference in the class Strcat besides the class name is the instantiation of StringBuffer.
NB: I'm not accusing the author of bias against Java, nor am I ignorant of the fact a bunch of
Information: "I want to be anthropomorphized"
erm ... I only checked the fibonacci routine, but it's actually quite funny - he's branching recursive calls, a clear case when a smart-enough runtime optimization would work better. I mean, any reasonably smart optimizer would eventually figure out that there are too many calls to the same function with the same argument to just stand by and watch. I'd say that given this difference c++ did quite alright in that one.
This is known as the "halting problem". No, the compiler cannot guarantee the ability to transform a recursive solution to a non-recursive one. The case of the fibonacci algorithm is a particularly difficult one to transform properly if the compiler hasn't special cased it.
That said -- Ack and Fib are call overhead limited. They examples of poor quality code whose performance is not inner loop based.
Hash will be C-string (specifically strcmp and sprintf) limited in performance. The performance is therefore very data dependent (since Java uses length delimited strings.) Using a fast string class such as "The Better String Library" (http://bstring.sf.net) would have yielded C++ far better performance. A similar comment applies to the strcat test.
The Heapsort is a particularly bad implementation. In good implementations, the Intel compiler really takes gcc to town. See: http://www.azillionmonkeys.com/qed/sort.html
Integer Matrix multiplying is an extremely rare application. So I wouldn't put too much stock in the results here -- though, I would be surprised if there was much differentiation between either Java of C++ on this test.
The method calling, I think, will be very much limited by the compiler's ability to inline past method calls. I think Intel C/C++ differentiates itself on such things.
The Nestedloop and random tests are interesting -- I don't see how Java is supposed to beat C++ on it, but its possible to be equal.
I don't know enough about the Java object system and barely enough about C++ object system to comment on sieve or objinst.
It seems to me that sumcol and wc are going to be IO limited.
I don't think this test is exactly fair, as the code is not representative for tasks where performance really matters.
Check out this recent IBM Developerworks article which looks at how modern JVMs handle allocation and garbage collection.
Some very surprising tidbits there. For instance:
"Performance advice often has a short shelf life; while it was once true that allocation was expensive, it is now no longer the case. In fact, it is downright cheap, and with a few very compute-intensive exceptions, performance considerations are generally no longer a good reason to avoid allocation. Sun estimates allocation costs at approximately ten machine instructions. That's pretty much free -- certainly no reason to complicate the structure of your program or incur additional maintenance risks for the sake of eliminating a few object creations."
Read the article for an excellent nuts-and-bolts analysis of many current performance considerations. From the posts in this thread, it's pretty clear a lot of people haven't looked into what's actually done in a server JVM these days.
I'm futzing around with the other hash benchmark, and sure enough, making only a trivial change to the code (eliminating the unnecessary strdup in the second hash lookup), gets me about a 30% improvement in performnace.
This guy is a tool.
Let's try not to let fact interfere with our speculation here, OK?