Performance Benchmarks of Nine Languages

ikewillis writes "OSnews compares the relative performance of nine languages and variants on Windows: Java 1.3.1, Java 1.4.2, C compiled with gcc 3.3.1, Python 2.3.2, Python compiled with Psyco 1.1.1, Visual Basic, Visual C#, Visual C++, and Visual J#. His conclusion was that Visual C++ was the winner, but in most of the benchmarks Java 1.4 performed on par with native code, even surpassing gcc 3.3.1's performance. I conducted my own tests pitting Java 1.4 against gcc 3.3 and icc 8.0 using his benchmark code, and found Java to perform significantly worse than C on Linux/Athlon."

Comments

About the Python performance

[ultrabot]

Note that Python is pretty easy to extend in C/C++, so that speed critical parts can be rewritten in C if the performance becomes an issue. Writing the whole program in C or C++ is a premature optimization.

Read the OSNews thread

[be-fan]

There were a number of problems with this benchmark, which are addressed in the OSNews thread about the article.

Namely:

- They only test a highly specific case of small numeric loops that is pretty much the best-case scenario for a JIT compiler.

- They don't test anything higher level, like method calls, object allocation, etc.

Concluding "oh, Java is as fast as C++" from these benchmarks would be unwise. You could conclude that Java is as fast as C++ for short numeric loops, of course, but that would be a different bag of cats entirely.

Re:Trig functions...

[Doomdark]

Don't forget that it is also percieved as slow since just about any application anyone has seen for a desktop environment written in Java has a sluggish GUI.

It's in many ways unfortunate that with JDK 1.2 (Swing) and onwards, Sun pretty much dumped fast native support for GUI rendering. It has its benefits -- full control, easier portability -- but the fact is that simple GUI apps felt faster with 1.1 than they have done ever since (or even more). This is, alas, especially noticeable on X-windows, perhaps since often the whole window is rendered as one big component as opposed to normal x app components (in latter case, x-windows can optimize clipping better).

Years ago (in late 90s, 97 or 98), I wrote a full VT-52/100/102/220 terminal emulator with telnet handling (plus for fun plugged in a 3rd party then-open SSH implementation). After optimizing display buffer handling, it was pretty much on par with regular xterm, on P100 (Red hat whatever, 5.2?), as in felt about as fast, and had as extensive vt-emulation (checked with vttest). Back then I wrote the thing mostly to show it can be done, as all telnet clients written in Java back then were horribly naive, doing full screen redraw and other flicker-inducing stupidities... and contributed to the perception that Java is and will be slow. I thought it had more to do with programmers not optimizing things that need to be optimized.

It's been a while since then; last I tried it on JDK 1.4.2... and it still doesn't FEEL as fast, even though technically speaking all java code parts ARE much faster (1.1 didn't have any JIT compiler; HotSpot, as tests show, is rather impressive in optimizing). It's getting closer, but then again, mu machine has almost an order of magnitude more computing power now, as probably does gfx card.

To top off problems, in general Linux implementation has been left with much less attention than windows version (or Solaris, but Solaris is at least done by same company). :-/

Language performance arguments miss the point

[tizzyD]

Consider what was done years ago with assembly. The performance was incredible, and the amount of superfluous garbage in the code was minimal. Hey, if you wrote the assembly, why would you spend time putting it in?

Then, with more and more languages, especially ones with VMs, you get further and further away from the hardware. The end result: you lose performance. It does more and more for you, but at the expense of real optimizations, the kind that only you can do.

Now the zealots will come out and say, "Language X is better than language Y, see!" To me this argument is boring. I tend to use the appropriate tool for the job. So:

• Python for scripts, prototypes, proofs of concept, or components where performance generally is not an issue.
• For desktop apps, Visual Basic (yep, most IT apps are in VB). There is no justifiable reason for an IT department group to write a sales force reporting system in C++! If you want C++, go get a job at a software company. Stop wasting money and time making yourself feel like a hotshot. [I'd consider Kylix here if it was based on Basic. Why? Because honestly, Pascal is just about dead, and Basic is the king of the simple app. Let's just live with it and move on. I do want a cross-platform VB . . . ]
• For web apps, well, I stick around PHP/ASP.NET. Why? Portability! And moreover, the sticking point in a web-based app is not the UI layer; it's usually the underlying data extraction and formatting. Don't waste your time with lower level languages there. IMHO it's just not worth it. JSP and Java stuff, yuck! Too much time for too little bang.
• Java/C# (also consider mono/LISP for most production apps. Why? Portability! I want no vendor holding me by the balls. I want platform independence on the back end, and these are the few ways to achieve it. I'd include Haskell/OCAML here when appropriate. Perl? I'm loathe to use Perl as production, considering most Perl code cannot be understood 2 weeks after it's written. I'd rather take the hit in performance and be able to pass the code to someone else later.
• C++/C for components--just components--where performance is at an absolute premium or there exists some critical library that only has this kind of interface. But this step has to be justified by the team, with considerable explanation why a different architecture could not suffice. Otherwise, the team could waste time checking for dangling pointers when instead it could be doing other things, like finishing up other projects.
• Assembly? Only when there is not a C complier around. Embedded stuff. Nowadays, you just do not have the time to play.

Yes, my teams use many languages, but they also put their effort to where they get the biggest bang for the buck. And in any business approach, that's the key goal. You don't see carpenters use saws to hammer in nails or drive screws. Wise up!