Slashdot Mirror
Java Faster Than C++?
jg21 writes "The Java platform has a stigma of being a poor performer, but these new performance benchmark tests suggest otherwise. CS major Keith Lea took time out from his studies at student at Rensselaer Polytechnic Institute in upstate New York's Tech Valley to take the benchmark code for C++ and Java from Doug Bagley's now outdated (Fall 2001) "Great Computer Language Shootout" and run the tests himself. His conclusions include 'no one should ever run the client JVM when given the choice,' and 'Java is significantly faster than optimized C++ in many cases.' Very enterprising performance benchmarking work. Lea is planning next on updating the benchmarks with VC++ compiler on Windows, with JDK 1.5 beta, and might also test with Intel C++ Compiler. This is all great - the more people who know about present-day Java performance, the better.""
...on x86? Please! Wake me up when someone who knows enough about C++ to pick a decent x86 compiler runs some benchmarks.
Doesn't it make you feel good to know that our freedoms are protected by politicans, lawyers and journalists.
Java and C++ are language. Languages aren't "faster" or "slower", but compilers for them might be. I find it somewhat underhanded to put the languages in the header when it's really comparing compilers.
Not to mention, inter-language compiler benchmark[et]ing is notoriously difficult to get 'right'. The programs tested are often stupid (doesn't do anything meaningful), or constructed by a person with more skill/bias for one language than the other.
It's been ages since I've programmed in C++, but it's good to know see these favorable comparisons. I think about the Struts/Hibernate/Oracle applications I write today and shudder when I imagine what how difficult it would be to have to develop web applications in C++. C++ will be around forever and certainly has its place, but long live Java.
There seem to be some unanswered questions here..
- How equivalent were the benchmarks? Where they programmed in an optimum way for their respective compilers and libraries? I'm sure the java ones were.. what about the C++ ones? The author states he doesn't understand G++ very well.
G++ is also known to not produce the best results.
"I rant it with -O2"
My guess is many of the tests were not implemented properly in c++.
The main clue would be this... I can understand java having better than expected performance.. but there is no way I can accept that java is that much FASTER than properly done C++... it doesn't make any sense.
I’m sorry but someone who says "I've never been very good at decoding GCC's error messages" is not competent enough to perform performance comparison. This performance test is a shame and shouldn’t be taken seriously.
-----
One is born into aristocracy, but mediocrity can only be achieved through hard work.
That's Great! I can't figure out GCC's error messages, but I offer definitive proof that Java is faster than C++. Nice.
Comparing one C++ compiler (gcc) against the Java JVM on one operating system is not much of a test. I love Java, but this is almost something like microsoft would do. Test one specific OS, compiler, and configuration, and then make a blind, far-reaching statement. A fair test would include several platforms and compilers.
From methcall.cpp:
int
main(int argc, char *argv[]) {
int n = ((argc == 2) ? atoi(argv[1]) : 1);
bool val = true;
>> Toggle *toggle = new Toggle(val);
for (int i=0; i<n; i++) {
val = toggle->activate().value();
}
cout << ((val) ? "true" : "false") << endl;
delete toggle;
val = true;
NthToggle *ntoggle = new NthToggle(val, 3);
for (int i=0; i<n; i++) {
val = ntoggle->activate().value();
}
cout << ((val) ? "true" : "false") << endl;
>> delete ntoggle;
return 0;
}
Why allocate and deallocate an object within the scope of a function? Well, in C++, there's no reason, so this is bad code. You can just declare it as a non-pointer and it lives in stack space. But guess what? You can't do that in Java: all objects are allocated on the heap.
That, and using cout instead of printf, are probably why this is slower than the "equivalent" Java.
-_-_-
There are 0x40000000 types of people: those who understand 32-bit IEEE 754 floating point, and those who don't.
This would be much more meaninful if you had used fputs() instead of write() in the C version.
As for "several orders of magnitude," I call bullshit. There's no way in hell the standard C++ IO functions are hundreds of times slower unless they're extremely badly written. Which leads me to another reason why this example sucks: there can be different implementations of the standard libraries.
In conclusion, this "comparison" is a stinky pile of shit, and should be ignored. And it's not even on topic, since it doesn't have a Java version.
All you programmers that say you can do anything in Java/C#/etc are terrible.
Actually you can do most anything in those languages. Although for performance, and desgin reasons you may wish to use something else depending on the application.
You have no respect for code. Learn assembly and then we'll talk.
I know assembly, and fun as it is, it isn't well suited for high level projects where code reuse and mantainability are important. By the way, I have no respect for someone who knows assembly and thinks it is difficult. It isn't. And it certainly isn't graceful or elegant, but I love it all the same.
I believe that Sun's javac bootstraps itself just like gcc does. That would be your java compiler written in Java.
_Jikes_, OTOH, is written in C++. But that's not really the official Java compiler by a long shot.
Your second requirement is absolutely bizarre. Does this mean you're not taking languages like Lisp, Prolog, Python, and Perl seriously, too? Those are all very nice languages for doing stuff in, but I'm pretty sure id never wrote a 3D engine in them. In fact, I was under the impression that id has never written a 3D engine in C++, either. Should we not take C++ seriously?
IMHO: The measure of a language is not how easy it is to write an arbitrary application in it. It's how easy it is to write something for which the language was designed to do.
-Erwos
Plausible conjecture should not be misrepresented as proof positive.
A few examples
1) Java has bounds checking for arrays, C++ doesn't. This is specified in the language. This affects performance.
2) Java has garbage collection, C++ doesn't. This is specified in the language. This affects performance.
Also, the specification of Java says that it should be compiled to byte code and executed in a JVM.
So the "language" certainly affects performance.
The Internet is full. Go Away!!!
I can say after using SWT in the Pocket PC platform that it sucks. The widgets are primitive, lack any real model implementation, they brake compatibility between minor versions, the most advanced things are done in the Eclipse UI packages not in the widget toolkit and the code you end up writting is ugly.
Elaboration:
No model: with swt you get the widget as a UI object with a field of type Object that you may or may not use as a reference to the object dislpayed. You ahve to write the code that updates th view when the underlying object changes and hende there is not real MVC pattern there. You can do it yourself. Imagine the huge pile of repeated effort in many projects around the world.
Primitive widgets: the table widget is just a string grid. No masked text input, and it goes on and on.
Ugly code: they use public fields for setting state to widgets not constructors or factory methods or even setters. They have integer constants for decribing widgets and you have to use those and see them. Creating a label requires using new Label(SWT.LABEL) and creating a horizontal Line is new Label(SWT.LABEL | SWT.HORIZONTAL). So much for hiding complexity.
There are 4 type of layout managers, the initialization is verbose and in fact only 1 are relevant, GridLayout, the other 3 are special cases of the former.
The more advanced widgets are not in SWT, only in the Eclipse centered custom libraried. The SWT mantainers say those classes are for use in the eclipse UI and they don't mean thme to be general purpose, use thej if they fit, but don't ask for improvements.
The only way SWT is justified is when you have very low resources (a pocket pc) or need to compile with gjc. Otherwise, Swing is way better even when it is far from pefect.
Why did he use only -O2?
-O3 adds function inlining and register renaming.
Also, some of the code doesn't look too much of a test of the language, but more of a test of the libraries. Both versions of hash rely on the library implementations, and it looks like hash.cpp does an extra strdup that the java version doesn't. I don't know either of the hash libraries well enough, but I don't see why this significant slowdown would be necessary in the gcc version.
HIV Crosses Species Barrier... into Muppets
Just becuase you can't write a kernel in it doesn't mean the language is worthless. There are many things that you can do very easily in Java that would be more difficult in other languages, and Java makes it impossible to write many security bugs that plague other languages. You can't do EVERYTHING in Java, but you can do quite a bit.
Gcc is designed for compatibility with a wide range of architectures, and is not optimized for a single one. He also (apparantly) used stock glibc from Red Hat. And only one "test", the method call test, showed java to be a real winner. And even then, it's server-side Java, which is meaning less when you talk about it as a day-to-day dev language (ie; creating standalone client-side apps).
Intel's (heavily optimized) C++ compiler should be a damn sight faster, and so should VC++.
This "comparison" is so limited in scope and meaning, that this writeup should be considered a troll.
Hell, read his lead-in:
Ie; I set out to prove Java is teh awesome and c++ is teh suck!
If anything it proves something I've known intuitively for a long time. gcc does not produce x86 code that's as fast as it could be. That's a trade-off for it being able to compile for every friggin cpu under the sun.
I can't wait till RMS takes personal offense and goes on the attack.
I don't need no instructions to know how to rock!!!!
Okay, so how could I make a blanket statement like that? In this case, the author of the paper merely used a compiler switch in gcc (-o2). That doesn't mean his c++ was highly optimized. It just means he told the compiler to do its best. If you really wanted to highly optimize c++, you would study the compiler and how it works, and you would profile the actual assembly that the compiler generates to make sure that it didn't do anything unexpected. Given *any* algorithm, I can come up with a c++ implementation that is faster than a Java implementation. Period.
The java compiler actually compiles to a virtual opcode format, which is then interpreted by the java virtual machine at runtime. Imagine if you needed to talk to someone on the phone, but instead of talking to them, you had to talk through an intermediary. Is there any possible way that it could be faster than talking to the person directly?
Now, I'll be the first to point out that a badly implemented c++ algorithm could be much slower than a well implemented Java algorithm, but I'll take the pepsi challenge with well written code any time, and win.
Relying on benchmarks and code somebody else wrote doesn't prove anything. Did he get down and dirty with the compiler and look at the generated assembly code? No, he did not.
Move along, there's nothing to see here.
WWJD? JWRTFA!
until one can write an OS kernel in it, Java is still not what I would personally look to for the future of software development.
That's a silly thing to say.
How many OS kernels are going to be written in the next ten years? And how many business applications? And the ratio of the latter to the former is what, 10,000 to 1?
When C/C++ uses profiling it will only ever produce one "best case" compilation for a given function.
With any JIT system you have the opertunity to use the profiling information from a given "window" of the execution so there is the possibility of having more than one compilation for a function.
Now, I do not know how sophisticated JAVA JIT compilers have become but this is one area where JIT will have an upper hand over a static compiler.
OTOH, these tests do not look like there is enough significant variation in the execution path for profiling to make a large difference.
No, I'm not being ironic.
I'm tired of some programmers expecting to be worshipped because they know assembly.
Assembly isn't all that.
For some uses, it is the right tool. For 99.9%+ it most definitely isn't.
The Internet is full. Go Away!!!
Unix GUIs are far easier to implement with Java than with C++. Athena/Xlib are a huge fargin' hassle and no-one ever gets the widgets right. Motif is too expensive to license.
You really need to look into Qt. It's much easier to use than Swing.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
You are once again spouting the tired old line that Sun is the master of Java. Not at all true, Java's fate is controlled by a whole host of companies - including IBM. Take a look at the reality of Java platform evolution at the Java Community Process web site.
It's a standards body just like any other, just more open.
P.S. - Aside from that gripe being wrong, I agree with the other poster that you should look into Objective-C to address other issues. Look for "GnuSTEP" for cross-platform objective C GUI work. It's just nicer to use on a Mac as they have very good tools (though in fairness I have never looked at what GnuSTEP tools might be around, I just can't imagine them being quite as good as the tools Apple has sunk so much effort into!).
"There is more worth loving than we have strength to love." - Brian Jay Stanley
What concerns me most is the amount of memory it requires. In theory, once the requisite stuff is loaded into memory, Java byte code can be processed at nearly the same rate that C++ code is. Depending on the bytecode and assembly that are generated in each case, Java or C++ could end up being faster. I think it's obvious that Java incurs some overhead in translating the bytecode, which ought to slow it down *some*, but that amount can be minimized.
On the other hand, Java takes a great deal of memory. If C++ had a dedicated server sitting in memory, ready to execute commands for it, it probably would speed up execution, but that wouldn't mean C++ were faster.
After accepting the memory hit for Java, the performance on things like apps servers seems to be pretty decent. I have yet to use a java client application, however, where I didn't feel that it was sluggish (even after loading). There are only two explanations: all java code is written poorly, or Java inherently causes a performance hit.
As we abstract languages more and more, we see performance hits for increased functionality and ease of developing. We also see that, because of the easier development, it is easier to improve scalability and use more efficient algorithms. It is rare that a program cannot be sped up by hand-optimizing the assembly, but it is also rare that anyone has time to design the much more critical optimized algorithms at such a low level. Therefore, I predict that eventually Java (or something like it) will be embraced as programmer time matters more than speed of execution.
The one thing that disturbs me about Java is that, while in C++, it is easy to change the assembly while maintaining the C/C++ code, in Java, you are tied to platform-independent code, which prevents you from doing platform specific optimizations. You have to depend on the native java implementations and/or widget toolkits for those kind of things. And so far, although the situation is improving, I've been pretty unhappy with the speed and my ability to improve it.
-Dan
Whew, there's seems to be a lot of denial running through this thread. "An interpreted language just can't possibly be as fast or faster as a compiled language! So I just don't care what the empirical results say, no matter how badly or well done they are, it just can't possibly be!"
I think some of you are overlooking the fact that a VM running byte code is capable of doing optimizations that a compiled language just can't possibly do. A compiled language can only be optimized at compile time. Those optimizations may be very sophisticated, but they can never be any better than an educated guess about what's going to happen at runtime.
But a VM is capable of determining exactly what is happening at runtime; it doesn't have to guess. And thus it is able to optimize those sections of code that really are, in true fact, impacting performance most severely. In can do this by compiling those sections to machine code, thus exploiting precisely the advantage that a compiled language is alleged to have by its very nature. And other kinds of optimizations, the kind that a compiler traditionally does, can be performed on those sections as well.
Of course there are scenarios where runtime optimization doesn't win much, for example in a program that is run once on a small amount of data and then stopped, so that the profiler doesn't get much useful info to work with. This is why Java is more likely to have benefits like this in long-running server processes.
And of course a conscientious C++ programmer will run a profiler on his program on a lot of sample data, and go about optimizing the slowest parts. A conscientious Java programmer should do that too. But an interpreted language has the advantage that the VM can do a lot of that work for you, and always does it at runtime, which is when it really counts.
Always keep a sapphire in your mind
>A VM otoh does get this capability for free. ;)
TANSTAAFL
> Java enforces garbage collection, and thus optionally gets the particular performance gain
Err... whether garbage collection gains or looses you performance depends very much on your application. I am not convinced that any garbage collector can even be theoretically faster than automatic variables.
>As was pointed out, one of the strenghts of C/C++ are pass-by-value....
After programming in C++ for 14 years, I don't think I've heard that one before
>The fact that you have to explicity declare a c++ parameter as pass-by-reference suggests that those interested in "good programming practices" (tm) will only make a pass-by-reference if you intend to modify it's contents
const &? Ever heard of const references before? I don't feel qualified to judge your Java programming, but it's evident you're not particularly familiar with C++.
I'd be interested to know how you have more control over the design of 3rd party libraries in Java than in C++.
no taxation without representation!
Here is an excerpt from the article for this story: Lea used G++ (GCC) 3.3.1 20030930 (with glibc 2.3.2-98) for the C++, with the -O2 flag (for both i386 and i686). He compiled the Java code normally with the Sun Java 1.4.2_01 compiler, and ran it with the Sun 1.4.2_01 JVM. He ran the tests on Red Hat Linux 9 / Fedora Test1 with the 2.4.20-20.9 kernel on a T30 laptop. The laptop "has a Pentium 4 mobile chip, 512MB of memory, a sort of slow disk," he notes.
What this shows is that GCC's implementation of C++ is slower than an interpreted language like Java. This does not show that C++ is slower than Java.
----
Notes on Stuff
I just went through and tested the hash2 benchmark and found that I was correct. The C++ version slaughters the Java version (even in "server" mode). This is completely different than what this dude's page shows.
Here is the "correct" code for hash2.cpp:
#include <stdio.h>
#include <iostream>
#include <ext/hash_map>
using namespace std;
using namespace __gnu_cxx;
struct eqstr {
bool operator()(const char* s1, const char* s2) const {
return strcmp(s1, s2) == 0;
}
};
struct hashme
{
size_t operator()(const char* s) const
{
size_t i;
for (i = 0; *s; s++)
i = 31 * i + *s;
return i;
}
};
int
main(int argc, char *argv[]) {
int n = ((argc == 2) ? atoi(argv[1]) : 1);
char buf[16];
typedef hash_map<const char*, int, hashme, eqstr> HM;
HM hash1, hash2;
for (int i=0; i<10000; i++) {
sprintf(buf, "foo_%d", i);
hash1[strdup(buf)] = i;
}
for (int i=0; i<n; i++) {
for (HM::iterator k = hash1.begin(); k != hash1.end(); ++k) {
hash2[(*k).first] += k->second;
}
}
cout << hash1["foo_1"] << " " << hash1["foo_9999"] << " "
<< hash2["foo_1"] << " " << hash2["foo_9999"] << endl;
}
The ratio of people to cake is too big
Thank you! I am not the only one that noticed that these are not even equivilant code. Using sprintf when you expect speed is stupid. sprintf is for simplicity. Making 5 function calls in the middle of a for loop when once would have been fine is also stupid. Also Gxx type compilers are KNOWN to be slower. They are getting better. But the 3.3 branch is more of a staging for better things to come. I would be interested in the same thing done with the intel compiler. WITH all the sub libraries compiled with the intel compiler.
.net could catch up to C performance is interesting though. The code in these enviroents could be self tuning themselves. Did he prove that? I do not think so. NOT with that code.
Sure the code does the same thing. But the few code bits I looked at are not identical. To be a 'fair' test. All the functions need to be written in the same file. Then with *VERY* minimal changes work in both java and C. It wasn't even C++ code. Also if he used the G++ compiler? That is *KNOWN* to be very slow. They rewrote the sucker for 3.4. All the functions need to be called in the same way. All the loops need to be walked the same way. Etc...
This test is bogus. I was willing to give the dude a listen. He even tries to defend what he does wrong. Its almost pathetic. For example he talks about GCC vs Visual studio. Then goes on to use G++. Interesting... Another good thing he does to 'obscure' the issues is to use overloads in C++ yet use method calls in Java. overloads are much slower. Usually firing off all sorts of copy constructors. Method calls are much faster. He even admits it C++ is faster in one case. Well that ONE case is about the only one where he has apples to apples. The rest are not. Also his 'method call' test is not the same thing. He did it in such a way that fires at least 2 copy constructors in the C++ way yet returns direct objects in the java way. C++ is slower in those cases. Well DUH.
The idea that java or
It is like I have always said. C/C++ does not produce awsome code. It lets you produce awsome code. But if your not carefull it lets you produce extreemly BAD code. It does not hold your hand. It has never claimed to. I could with a little bit of work (not as hard as some people would assume) produce the EXACT oposite results of what he got.
A fun one:
Java:
C++:
The C++ version could have been written IDENTICALLY (except for the 'public' modifier on the definition) to the Java version, but it was not. I'm not sure what the compiled difference might be, but there is a difference between these two bits of code, notably that in the C++ version there is a tertiary operator evaluated as an argument to a call to Ack, where this is not the case in the Java version. I would guess that this would be a more difficult thing for a compiler to figure out.
The differences in the methcall sources are even worse; in the C++ version of NthToggle, there are unnecessary dereferences of the this pointer that will kill performance, as well as in the call to new NthToggle(val, 3) in the C++ version is written with the coded constant new NthToggle(true,3) in the java version! It's hardly fair to compare things of this nature.
The trouble with benchmarking different languages is hard enough due to inherent differences between languages; it's not really enlightening to introduce artificial differences such as these.
"There are a dozen opinions on a matter until you know the truth. Then there is only one." - CS Lewis (paraprhase)
sprintf(buf, "%x", i);
It must parse the "%x" and determine what it's trying to do. So it decides, at runtime, you want to translate an integer value into a hexidecimal string. Of course if there's an error at runtime in the string "..." it must generate an error. How 'bout using strtol?
Now let's look at the java version:
Integer.toString(i, 16)
Ok, here we have something that is strongly typed so of course it will be faster. At runtime the java compiler _knows_ what it's dealing with and it knows it must invoke the hexadecimal conversion code.
Note that these statements are within loops.
Just one example, that was the first file I looked at. I don't think they have quite optimized the C++ code IMO. Plus the C++ library is notoriously slow, I would recommend rogue wave or your homegrown C++ classes.
I think the lesson here is it's very easy to write slow C++ code while it's very easy to write fast Java code.
Gimme any C++ code here and I'll profile it/speed it up and get it as fast if not faster than java.
2 years and no mod points. Join reddit. Because openness is good.
It should also be mentioned that the java language requires specific overhead to be included that C++ and C do not. Even if compiled down to sleak assembly, Java is still saddled with doing bounds checking.
The rest of the performance improvements are in the compiler optimizations and libraries which are mostly tangential to the language itself. If the compiler is clever enough to take "for (x=0, i=0; i<100; ++i) x+=5;" and substitute this for "x=500;", then great, but it should not be confused with an endorsement of the language itself.
Furthermore, I had no difficulty modifying the C++ code to outperform or at least meet the results of the server-side JVM using G++. In the cases where Java had any lead whatsoever, the code was so trivial that the JVM could virtually precompute the result. I don't see this as being useful because in the real-world, nothing I write is so trivial that this is possible. If it was, I would have done it myself. I believe this largely explains the discrepency between these "tests" and my actual experience.
-Hope
It's neither a bug, nor is a feature. It's a difference, and possibly a testing methodology flaw.
And on certain workloads it can decrease performance. Coders that know their languages should recognise the difference between the two memory allocation methods, and adjust their code accordingly. I seriously doubt you'd claim that java's garbage collector is always faster than manual memory management.
Perhaps, but it's also unfair to have a 'benchmark' that always pays the penalty of C++ memory allocation and deallocation, but never pays the penalty of Java memory allocation and deallocation.
Just because the memory is freed back to the system after termination doesn't mean it isn't a memory leak. It's a bad practice to allocation memory and not free it when you're writing C or C++ code. A longer running process would probably be more fair, or even a garbage collect at the end of execution. After all, on 'real' programs, a garbage collection is quite likely.
I used up all my sick days, so I'm calling in dead.
"Any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp." -- Philip Greenspun
/ took way too long to google
I notice that much of the overhead is simply in making function calls.
Ackermann.cpp, for example, spends very little time actually calculating anything. Much of it's work includes all the overhead associated with making a function call.
Included in this overhead is management of the frame pointer. By using -fomit-frame-pointer, you avoid pushing the old ebp on the stack and a store of the current esp into ebp.
Ackermann runs about twice as fast with this simple optimization.
I saw this test a few days ago and wanted to check it out. The first thing I realize is that the source code is somewhat different even if Java has almost the same syntax as C++.
I understand that System.out.println(); is not in C++ but why have
instead ofI made the C++ code look like Java and got a 15% save. Problably even more if I had increased the number you call the program with. I looked at some of the other program and they have different code in them as well. So this test is bullshit it only shows that you can make slow programs in any language.
I'm sorry, but I am afraid you are wrong.
Well, I know the halting problem and I don't see how it relates to what you're saying. The conversion from recursive to iterative isn't arbitrarily complex, it's simple and mechanical. The easiest way is to simply use a stack to maintain the state of what were previously recursive calls.
This is still recursion. You have simply substituted one stack for another.
Anyways, all the halting problem implies is that an optimizer will never be able to find every situation where a particular optimization is applicable.
No, it implies much more than this.
Deciding whether two programs are equivalent(EQTM) is equivalent to ATM (the halting problem). (I'll give the series of proofs if anyone requests it.)
This means that there is no algorithmic way to transform recursive functions into iterative ones, and any substitution must be based upon matching of patterns pre-coded into the compiler.
NOTICE: This notice will appear at the bottom of all my slashdot posts.