Slashdot Mirror
Performance Benchmarks of Nine Languages
ikewillis writes "OSnews compares the relative performance of nine languages and variants on Windows: Java 1.3.1, Java 1.4.2, C compiled with gcc 3.3.1, Python 2.3.2, Python compiled with Psyco 1.1.1, Visual Basic, Visual C#, Visual C++, and Visual J#. His conclusion was that Visual C++ was the winner, but in most of the benchmarks Java 1.4 performed on par with native code, even surpassing gcc 3.3.1's performance. I conducted my own tests pitting Java 1.4 against gcc 3.3 and icc 8.0 using his benchmark code, and found Java to perform significantly worse than C on Linux/Athlon."
I am not a compiler nerd (IANACN?), so maybe someone else can answer the following simple question:
Why are the Microsoft languages so fast with the Trig functions?
I'm a 2000 man.
Not sure of the accuracy. Benchmark is on a loop:
32-bit integer math: using a 32-bit integer loop counter and 32-bit integer operands, alternate among the four arithmetic functions while working through a loop from one to one billion. That is, calculate the following (while discarding any remainders)....
It also relies on the strength of the compiler, not just the strength of the language.
The Custom Mary
Why did VB do so bad on IO compared to the other .Net benchmarks? They were pretty much equal up until the IO benchmarks? Any chance of getting the code published that was used to test this?
Wow.... according to this gcc C sucks : (
Im not here now... Im out KILLING pepperoni
I conducted my own tests pitting Java 1.4 against gcc 3.3 and icc 8.0 using his benchmark code, and found Java to perform significantly worse than C on Linux/Athlon.
Why is this a suprise? C has been most commonly used for so long because of it's speed and efficiency. I think anyone who has done much work with either developing or running large scale java programs knows that speed can definitely be an issue.
Not everything is analogous to cars. Car analogies rarely work.
I see once again that Eugenia (a supposed pro-Linux pro-BeOS person who doesn't use Windows) has done all her benchmarks [i]under[/i] Windows. I have a feeling that Python would perform a lot better if it was running in a proper POSIX environment (linked against Linux's libraries instead of the Cygwin libs). Probably the C code compiled with GCC would perform a fair bit better too.
Visual C++ runs faster on Windows. Duh
Im dreaming ofa big bndwdth, That can resist the
Not that it would have helped them survive the slashdotting, they run linux. Well, when they were up and running that is....
Serisously? Where is the joke? I didn't find anything funny in your post. I don't get the joke. Nice attempt though.
Why benchmark the various ".NET languages" (those languages whose compilers target the CLR)? Every compiler targeting the CLR produces Intermediate Languages, or more specifically MSIL. The only differences you'd find is in optimizations performed for each compiler, which usually aren't too much (like VB.NET allocates a local variable for the old "Function = ReturnValue" syntax whether you use it or not).
Look at the results for C# and J#. They are almost exactly the same, except for the IO which I highly doubt. Compiler optimizations could squeeze a few more ns or ms out of each procedure, but nothing like that. After all, it's the IL from the mscorlib.dll assembly that's doing most the work for both languages in exactly the same way (it's already compiled and won't differ in execution).
When are people going to get this? I know a lot of people that claim to be ".NET developers" but only know C# and don't realize that the clas libraries can be used by any languages targeting the CLR (and each has their shortcuts).
I can't get to the article because of the slashdotting, so can anybody tell me which Java VMs were used in this test? A friend of mine did a study for a supercomputing course in which he found that IBM's J9 Java VM significantly outperformed Sun's JVM.
What about some string manipulation numbers?
...some analysis of the code generated by Visual C++ and gcc side by side, particularly for those trig calls. If there's that great a discrepancy between the runtimes, that's a good clue that either one of the compilers is under-optimising (i.e. missing a trick), or the other is over-optimising (i.e. applying some transformation that only approximates what the answer should be). I didn't see any mention of the numerical results obtained being checked against what they ought to be (or even against each other).
:)
As any games/DSP programmer will tell you, there are a million ways to speed up trig providing that you don't *really* care after 6dps or so.
OK, maybe I'm just bitter because I was expecting gcc 3.1 to wipe the floor.
These sigs are more interesting tha
This is a meaningless benchmark.
In *realworld* applications, different aspects starts to matter, non of which these benchmark covers:
1. IO speed
2. Allocation/deallocation of large (potentially paged) memory segments. How does the language organize memory (wrt memory access pattern) so that cache-locality is maximized?
3. Global optimization starts to matter bigtime
4. Ability to use native features, such as IO completion ports, AEM on linux, etc, etc.
5. bla, bla, bla.
I have a lot experience with large system-level software and Java is sometimes too slow to be useful. For most business applications, however, Java is good enough.
OSNews is showing the slashdot effect so I can't read the article, but I'm assuming that this was all tested on one version of Windows, like XP. In my personal experience, I found that some programs run faster in other versions, and I found the Java programs ran faster in Windows 98/ME then in XP, so I wonder if this was taken into account and if it wasn't, how much of a difference it would make.
This signature was left intentionally blank.
No question that C is faster than Java, but with a properly tuned JVM, I can get performance pretty close. The biggest penalty I saw was in startup. Besides the standard command line args, if you don't know what jvm.cfg check it out and learn how to use it.
"Look Lois, the two symbols of the Republican Party: an elephant, and a fat white guy who is threatened by change."
Well unfortunately, comparing Java to C# on a Windows machine is like comparing a bird and a dolphins ability to swim in water; Several components of C# are integrated right into the operating system so naturally it's going to run faster on a windows machine. Compare C#, C++ and Java on machines where the components aren't integrated and then we will have a FAIR benchmark.
Oh wait! C# only runs on one operating system. Can you name any other development languages that only run on ONE OS, boys and girls? Neither can I.
This is my sig. There are many like it but this one is mine.
Java is intended to run on a Virtual Machine environment, and C is not.
That is Java code can be run on any platform as long as it has a Java virtual machine. You can not do that with C++.
you are comparing apples and oranges.
Consensus is good, but informed dictatorship is better
According to this Visual C++ is faster than GCC? But what really surpised me was that a managed language was actually faster than a compiled language. Because look at Java 1.4 it kicked arss in all but a couple categories. Also what I don't understand is why VB.Net got such a high time for IO operations when all the .Net languages are compiling to the same IL. Also why didn't they try compiling VC++ into managed code too.
.net kicked butt over GCC.
.Net for a while now about how slow it is, and how slow managed code is. Are there any other sources that do simular comparisons?
.Net and Microsoft getting some high scores because it will be a good article to point some of my compiler geeks too. Because there modo so far has been GCC kicks arss even against VC++.
I would really like to see the code they used for some of these tests because I am sure some of the operations can use some further optimization.
But what really surprised me is that some of the managed code
How can that be, everybody has been complaining about
I don't know how accurate this article is but I am glad to see
One of the performance issues may be which C compiler the JRE was compiled with. I mean think about it, Java is only as good the the application that runs the compiled byte code you hand it.
So, if the JRE was compiled without optimization flags for your CPU type then all java byte code you execute with that JRE is going to perform poorly.
-Rusty
The Master (Angelo Rossitto) in Mad Max Beyond Thunderdome, "Not shit, energy!"
Benchmark code like this does not represent how these languages are used in practice. Idiomatic Java code tends to be full of dynamic classes and indirection galore. Just testing "arithmetic and trigonometric functions [...] and [...] simple file I/O" is not going to tell you anything about how fast these languages are in the real world.
All that being said :-| I think that Java in terms of building bytecode, client execution, and server execution is far behind the pack. Unless you have some multiple processor system with tons of bandwidth and memory it seems as if Java code is dog slow. Not trying to troll, just being honest. I can always pick out a website that delivering Java content by the number of times I keep on moving my mouse to ensure my system's still responding.
Someone should do a study on the time taken to design, implement and debug a resonably complex chunk of code under C++ and Java. I'm pretty sure that the result would show the huge advanatage of Java over C++.
The difference between Canada and the USA is that in Canada healthcare is a right and gun ownership is a privilege.
I think it would be nice to see string functions tested as well - some languages that are particularly poor with the Math are also particularly good with the Strings. IMHO
I'm a 2000 man.
These guys need to learn how to run designed experiments and provide useful data! This seems more like something to placate management...
"There are a dozen opinions on a matter until you know the truth. Then there is only one." - CS Lewis (paraprhase)
Visual C++ (an MS compiler/IDE) comes out on top.
Java1.4.2 works as good as native code.
Benchmarked on the Microsoft operating system!?!?!
Lemmie go ahead and tell you the responses:
They cheated on the benchmark!
I ran my own benchmark and gcc rocked the casbah!
Java is just slow starting and takes up too much memory.
This article, itself, is flamebait.
Good quote, too many chars. Seriously, the slashdot 120 char limit sucks!
either in the original XP based article or the Linux based comparison.
Comparing fully compiled against JIT/interpreted code seems like a fools errand especially when some of the components test aren't native to the platform the test is run on.
code
Given the ever accelerating clockspeed of processors, is the raw performance of langauges that big an issue? Except for CPU-intensive programs (3-D games, high-end video/audio editing), current CPUs offer more than enough horsepower to handle any application. (Even 5-year old CPUs handle almost every task with adequate speed). Thus, code performance is not a big issue for most people.
On the other hand, the time and cost required by the coder is a bigger issue (unless you outsource to India). I would assume that some languages are just easier to design for, easier to write in, and easier to debug. Which of these langauges offers the fastest time to "bug-free" completion for applications of various sizes?
Two wrongs don't make a right, but three lefts do.
The Java performance is best explained by an article by Prof Kahan: "How JAVA's Floating-Point Hurts Everyone Everywhere" http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf also see "Marketing vs. Mathematics" http://www.cs.berkeley.edu/~wkahan/MktgMath.pdf I suspect the relatively poor floating-point performance of gcc is also caused by the desire to acheive accurate results.
It's a pity that the present-day language of choice for high-performance computing, Fortran 90/95/HPF, was not covered in this study. There has been anecdotal evidence that C++ has approached Fortran, performance-wise, in recent years, but I've yet to see a proper comparison of the two languages.
Tubal-Cain smokes the white owl.
Don't forget about the Win32 Compiler Shootout
That's not surprising in either case, in terms of Java's results. It's well known that Sun has spent more time optimizing the Windows version of Java than any other, including the Solaris version.
My mind works like lightning. One brilliant flash and it is gone.
Fact: Windows 98 is dying
Hmm... someone needs to learn how to use search-and-replace better.<g>
Note that Python is pretty easy to extend in C/C++, so that speed critical parts can be rewritten in C if the performance becomes an issue. Writing the whole program in C or C++ is a premature optimization.
Save your wrists today - switch to Dvorak
I think this is a pretty good analysis. The very fact that it is non-partisan is a blessing. As a Java web app developer, I notice that if I take a subset of the graph for int and I/O operations, Java is quite competitive (#1 for the sum of those two metrics) with the other platforms. The trig stuff seems to really skew the results.
"No matter where you go, there you are." -- Buckaroo Banzai
IMO a program should use whatever tools are available and appropreate for the job, and not just worry about what is faster.
gcc -march=pentium4 -msse2 -mfpmath=sse -O3 -s -mno-cygwin
...
no loop unrolling ???
no inline functions ???
In fairness, gcc is not the world's greatest
optimizing compiler. It is the world's
most portable compiler. That code he
compiled for cygwin can uploaded to a
15 year old VAX and compiled and run
or to a beowulf cluster.
Try that with VB.
1. Bulgarian
2. Russian
3. English
4. Korean
5. Japanese
6. Chinese
I conducted my own tests pitting Java 1.4 against gcc 3.3 and icc 8.0 using his benchmark code, and found Java to perform significantly worse than C on Linux/Athlon.
From my experience Windows has always had the edge on the Java VM. But this could all be coming to a change:
Red Hat Linux 9 and Java 2 Platform, Standard Edition 1.4.2: A Winning Combination
A quote from the article states
"Internal tests reveal that performance of NPTL with a client/server application has been impressive. On a 2 X 1.6 Ghz P4 Xeon system, time to completion was 191% faster with J2SE 1.4.2 running on NPTL, compared with J2SE 1.41 running on the original Linux threads."
They should have picked the fastest language for their server.
There were a number of problems with this benchmark, which are addressed in the OSNews thread about the article.
Namely:
- They only test a highly specific case of small numeric loops that is pretty much the best-case scenario for a JIT compiler.
- They don't test anything higher level, like method calls, object allocation, etc.
Concluding "oh, Java is as fast as C++" from these benchmarks would be unwise. You could conclude that Java is as fast as C++ for short numeric loops, of course, but that would be a different bag of cats entirely.
A deep unwavering belief is a sure sign you're missing something...
Site was showing signs of Slashdotting, so I'll quote one of the more important sections...
Results
Here are the benchmark results presented in both table and graph form. The Python and Python/Psyco results are excluded from the graph since the large numbers throw off the graph's scale and render the other results illegible. All scores are given in seconds; lower is better.
int long double trig I/O TOTAL
Visual C++ 9.6 18.8 6.4 3.5 10.5 48.8
Visual C# 9.7 23.9 17.7 4.1 9.9 65.3
gcc C 9.8 28.8 9.5 14.9 10.0 73.0
Visual Basic 9.8 23.7 17.7 4.1 30.7 85.9
Visual J# 9.6 23.9 17.5 4.2 35.1 90.4
Java 1.3.1 14.5 29.6 19.0 22.1 12.3 97.6
Java 1.4.2 9.3 20.2 6.5 57.1 10.1 103.1
Python/Psyco 29.7 615.4 100.4 13.1 10.5 769.1
Python 322.4 891.9 405.7 47.1 11.9 1679.0
Beware: In C++, your friends can see your privates!
The guy's trig. VS. trig. stuff is just
comparing libraries, not code gen.
Using the IBM Java VM, I've been able to achieve consistently cutting my runtimes in half over the Sun VM. Anyone currently using the Sun VM for production work should test the IBM one and consider the switch.
My application that I benchmarked is data and network and memory intensive, although not math intensive, so that's what I can speak for. We consistently use 2 GB of main memory and pump a total of 2.5 TB (yes, TB) of data (doing a whole buch of AI style work inside the app itself) through the application over it's life cycle, and we cut our total runtime from 6 days to 2.8 days by switching to the IBM VM.
Which is true, but missing the point somewhat. Given an half decent optimizing compiler, C/C++ should almost always be faster than Java, but not so much faster that the speed becomes a deciding factor over the advantages that Java has compared to C/C++.
Now, there are always a certain class of programmer who must always have the absolute fastest performance possible. For them, C/C++ is the weapon of choice, if they don't have the time to hand-craft assembler.
For everyone else, it becomes a decision based on personal perference, technical imperative, management decision or dogma.
Please remain calm, there is no reason to pani... wait, where are you all going?
When I worked on Nintendo U64 video games (I mostly did the AI), we used C++ effectively to write fast code that was maintainable.
However, no project that I have worked on since had the requirement for runtime performance over-trumping the huge gain of productivity of languages like Java, or better yet Lisp, Python or Smalltalk.
Writing and maintaining software is a business - it the total cost vs. benefit that counts.
-Mark
http://www.ocf.berkeley.edu/~cowell/research/bench mark/code/
You are not testing the languages, you are testing the compilers. If you test a language with a crummy compiler (gcc sucks compared to commercial optimized C++ compilers) you will think the language is slow, when in fact, the compiler just sucks. The only valid comparisons that can be made are same language, different compilers.
They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.
Witness the huge performance delta between Java 1.0 and 1.4.2, and also realize that the dynamic optimizations performed by Hotspot are still relatively immature compared to the optimizations that are performed by your average C/C++ compiler, and you'll begin to realize that C/C++ is NOT the be-all and end-all of high performance code.
As a rough analogy, how often have you profiled code only to be surprised at where the bottlenecks are? Chances are if you've done any serious profiling, this has happened to you often, perhaps even to the point where you don't bother optimizing code without profiling it first. A similar process occurs with static optimizations - the code is optimized without the benefit of knowing on what the common code paths will be, whereas dynamic compilation allows for code to be optimized based on the way it is currently being used. Sure, there's overhead in performing the statistical analysis and compilation at runtime, but it does NOT logically follow that this will cause dynamic compilation to be any slower.
So, yes, you can construct programs, even some useful compute intensive programs, that perform as well or better on Java than they do in C. But that still doesn't make Java suitable for high-performance computing or building efficient software.
Benchmarks like the one published by OSnews don't test for these limitations. Microbenchmarks like those are still useful: if a language doesn't do well on them, that tells you that it is unsuitable for certain work; for example, based on those microbenchmarks alone, Python is unlikely to be a good language for Fortran-style numerical computing. But those kinds of microbenchmarks are so limited that they give you no guarantees that an implementation is going to be suitable for any real-world programming even if the implementation performs well on all the microbenchmarks.
I suggest you go through the following exercise: write a complex number class, then write an FFT using that complex number class, "void fft(Complex array[])", and then benchmark the resulting code. C, C++, and C# all will perform reasonably well. In Java, on the other hand, you will have to perform memory allocations for every complex number you generate during the computation.
The optimisers in sun's Java VM work on run-time profiling - they identify the most run sections of code and use the more elaborate optimisation steps on these segments alone.
Benchmarks that consist of one small loop will do very well under this scheme, as the critical loop will get all of the optimisation effort, but I suspect that in programs where the CPU time is more distributed over many code sections, this scheme will perform less well.
C doesn't have the benefit of this run-time profiling to aid in optimising critical sections, but it can more afford to apply its optimisations across the entire codebase.
I'd be interested to see results of a benchmark of code where CPU time is more distributed..
Come on guys, carefully choosing that simple benchmarks I can easily prove any language's faster than GCC.
I like java for some things, and the performance has even improved a bit lately. However if I am doing ANYTHING that has to scale and perform well under heavy load that uses cryptographic functions (especially public key encipherment), there is no way I can even seriously consider Java.
Someone (meaning anyone other than me) should do a benchmark of THAT, I'm sure it would be quite telling.
Finkployd
I'm just guessing here, but it sounds to me like somebody has had some less than good experiences with Java.
Well, benchmarks are for testing assumptions exactly like yours. The reason bencharks are interesting is they often find discrepancies that vary from what we all believe. Also interesting to note that the source of many of those beliefs is the marketing behind .NET in the first place. If you've worked cross language .NET development, you know that all things are NOT equal. Ever dealt with variable parameter functions?
The Python 'long' type is not a machine type such as a 32 or 64 or perhaps even 128 bit integer/long.
It is an arbitrary precision decimal type! That's why Python's scores on the Long test are so much higher (slower) than the other languages.
I wonder what Java scores when the benchmark is reimplemented using BigDecimal instead of the 'long' machine type.
Python uses a highly efficient Karatsuba multiplication algorithm for its longs (although that only starts to kick in with very big numbers).
"Horses for courses."
I don't think it is very relevant how fast java can be in these kind of benchmarks for most applications. Usually the processor is not the bottleneck. What matters is the responsiveness of the userinterface. Java (Swing actually) still feels very sluggish even on faster computers. That is what is stopping the adoption of Java on clients, not the fact that triogonomy is slightly slower than when done in C++
- - - - - - - - - -
Sample my google hacks
Ok; due to your insightful explanation and supporting evidence, I now understand the situation completely. Apologies for spreading FUD about Java's acceptable performance.
ps. Can you point me to the section(s) in kernel documentation that explain performance problems of Java compared to C or C++? I seem to have trouble finding it.
I like paying taxes. With them I buy civilization -- Oliver Wendell Holmes
First, if you read the description of the test, and look at the source code on the author's Web site, it is obvious that all of the computational "heavy lifting" is being done by the run-time math library routines (e.g., sin, cos, sqrt). I don't know offhand which languages use which libraries; it is not inconceivable that Python, say, uses the C math library. The only thing we can be sure is different in comparing languages is the function call overhead particular to each language.
The second, and IMO more serious issue, is that the test incorporates no measure of how accurate the results are. This is a real issue, because these functions are generally evaluated using an economized power-series approximation. Caculating fewer terms of the series will results in faster execution, but diminished accuracy.
To take a silly example, I can write a very fast C function to return the cosine of x:
(This uses the first two terms of a Taylor/Maclaurin expansion. It will be fast but not very accurate!)in NINE languages? & still no takers?
how cruel?
dude, that's awful advice. You didn't provide any evidence, you just made fun of people who think differently than you.
What would kernel documentation cover? Since the kernel contains no Java, I would say not a lot. The only thing of value you say is to tell people to test -- that's good, but you've told them what conclusion to reach. Why not provide some of your own answers? And, since there are lots of different kinds of code, you should be open to the fact that some code will run faster on some platforms, while other code performs better on other platforms.
overall, good troll.
p.s. C and C++ optimizations are very different. You shouldn't group them together. But you knew that, of course.
I tested this with the standard "time" command like this:
time java TheTestProgram
I also downloaded the JDK1.5 alpha release and it is faster. I believe the license prevents me from publishing the actual numbers for a beta version but you can download the early access release and try yourselves.
Yes, yes. I know C is much faster. But the point here is that you don't have time to react and notice wether startup takes 0.08 seconds or 0.008.
It's well known that benchmarks aren't the be all and end all. They're often just statistics geared towards a ultra-specific application (remember all those /. stories about benchmark cheating vendors?)
I've seen examples of gcc in a cygwin shell kicking visual-c++ ass at load up times of huge image data on a wintel box. I've also seen java (jdk 1.3) annihilate native c code on console apps calculating complex mathematical formulas on a linux box. This goes for both AMD and Intel chips.
Moral of the story? These languages are all suited to specific uses. Analyze your tasks, your platforms, and your compilers. Learn how to use optimizations properly. Evaluate your need for portability. Do a few tests for performance in different languages and compilers to see which one actually is fastest for your current application.
There is no single "fastest" language.
string handling in all varieties and data structure/collection manipulation. In my experience, C++ is orders of magnitude faster than java in these areas and .net will not catch up until it gets typed collections/templates next release. Java string handling makes ones hair hurt.
If it were done when 'tis done, then t'were well it were done quickly... MacBeth
Actually, IBM Java VM for Linux significantly outperforms Sun's. We have done some research on this topic in Aalborg University in Denmark. Final report is here (if anyone's intrested): http://kom.auc.dk/group/03gr841/pdf/Final_Report.p df
Interesting article, but I take it with a grain of salt.
First, All tests done on Windows. Yes, it's the only platform that all those languages run on, but of course M$ languages are going to have an inside track on optimizing for their OS libraries.
Regardless, programming should be done with the right tool for the job. If you were looking to write a utility that did heavy file i/o, I would stay away from an interpreted language. But if I were trying to write a cross-platform utility, I would certainly consider interpreted languages like Python or Java.
Summary: No news here: Some languages are better at some taks then others. The developer has to figure out which fits the task and his/her own skills the best.
-t
http://unmoldable.com W:"No one of consequence" I:"I must know" W:"Get used to disappointment"
But despite the anti-Microsoft feelings shared by the majority of open-source community, every Windows version of open-source software (Apache, PHP, MySQL, etc.) is coded for Microsoft Visual C++... And, since everyone who wants to compile the sources needs a MS VC++ license, most Open-source software projects are, in practical terms, helping Microsoft get richer and strength its virtual monopoly!
OK, Speed does matter a lot.
But what about type safety? Java has no generic typed containers, like the STL. This means you tend to find errors at runtime instead of at compile time.
I need to know that my code is as safe as possible. I don't want a user to find a bug because my hand tests didn't get 100% code coverage every time.
And how about predictable performance. I would much rather know that this function will tak 200ms all of the time instead of 100ms most of the time a 10 s due to garbage collection occasionally.
True. This kinda invalidates that part of the benchmark, because they are not comparing the same thing.
Save your wrists today - switch to Dvorak
Wasn't sure about that one, which is why I ACed it.
/. the C zealots only want to hear about the speed of the code they produce, not it's utility, robustness, or anything else.
/. can't write anything approximating it. No, most programs do not have the goals an OS kernel has.
Seriously meaning this is the point.
My point still stands, troll or not.
Of course, here on
Yes, Linux is written in C. Yes, it's very good. No, most people on
All I am saying is that performance is a terrible criteria for choosing a language when there are so many more important factors involved.
Conveniently I have the same system configuration as ikewillis (dual 2.0 GHz Athlon MP), but am running Windows XP instead of Linux. I also have Intel C++ 8.0, which he used on Linux to generate his results.
So I ran the same tests that he ran under Linux under Windows. Here are my results from Intel C++ 8.0, with Profile Guided Optimization turned off (comparing to his with PGO on):
Running the same tests under Windows with PGO turned on, the numbers did not change except on the least-significant digits, so I won't bother to list those too. Before running the tests, I set the program to run at high priority on one processor to avoid unnecessary interference from other running applications, or unnecessary processor-jumping--although when I tried it without, there wasn't much of a difference (< 1%).
Conclusions? First, it seems the 64-bit integer performance problem is something that exists only for Intel C++ 8.0 on Linux, not Windows. Second, it seems stdlib I/O performance is significantly higher under Linux than under Windows for this benchmark.
It's hard for thee to kick against the pricks.
I conducted my own tests pitting Java 1.4 against gcc 3.3 and icc 8.0 using his benchmark code, and found Java to perform significantly worse than C on Linux/Athlon.
executed with java -server
I would assume that the HotSpot compiler produces much better SSE2 optimized code than gcc. Without SSE2, Java's performance is worse in all respects tested here than native code compiled with gcc 3.3.
Running short life tests with -server is not a good way to test Java's performance. Adding the -server switch increases the load time of the VM (while offering some long-term memory access speed advantages I think). This is fine on servers, which have a long life, so you don't care about startup time. Using it in a benchmark where the typical runtime is a few seconds is not a valid test.
Stop-Prism.org: Opt Out of Surveillance
Yeah, with the increase in hardware price/performance the performance consideration is becoming less and less of a consideration in _most_ applications. There are still environments where efficiency is of paramount concern (the combination of great speed and low resource drag). Examples I work with are real-time financial trading applications, network back-bone servers (routers firewalls, intelligent switches, etc.), mobile and embedded devices, server daemons, and network applications (packet sniffers, etc.).
For general business processing applications and most web applications, efficiency is less of a concern and cost/time-to-market/maintainability/security are bigger.
I like these benchmarks but would like to see ones that also benchmark the other characteristics of languages (such as lines of code to do a common task, number of tests that need to be performed to validate common functions, memory space, etc. etc.)
Then, with more and more languages, especially ones with VMs, you get further and further away from the hardware. The end result: you lose performance. It does more and more for you, but at the expense of real optimizations, the kind that only you can do.
Now the zealots will come out and say, "Language X is better than language Y, see!" To me this argument is boring. I tend to use the appropriate tool for the job. So:
Yes, my teams use many languages, but they also put their effort to where they get the biggest bang for the buck. And in any business approach, that's the key goal. You don't see carpenters use saws to hammer in nails or drive screws. Wise up!
...tizzyd
I think this is a really salient point. To do simple arithmetic functions in Java is not necessarily to write good java code. For example, if I wanted to do vector calculations in Java, I'd make a vector class and overload operations for addition, cross product, dot product, etc. Then I'd have a really convenient way to do vector calculations. A C implementation would be messier, but might end up being more efficient.
I think everyone understands that different languages are good at different things (platform independence, performance, parallel execution, speed of development, and easy of GUI development are just a few relevant attributes), but what the language is used for will determine how you structure your program, which will in turn affect how efficiently it will execute a certain task. And this is differnt than attempting to compare a program written in C to a program written in Java as if it were C. That will be as much of a determinant of how fast a program in a given language seems, as the low-level implementation details.
It's not wasting time, I'm educating myself.
Benchmarks are just that. Benchmarks. A general guide to general performance. There never has, nor will there ever be, an all encompasing benchmark.
Thing of all of the parms which go into application performance. CPU (speed & quantity), memory (usage, speed, quantity), disk space (speed, defragmentation amount), and cache utilization are just a few of the big ones.
Think of the amount of impact the coder makes. Do they constantly free & release memory as opposed to maintaining internal buffer pools in their application? Do they write code to take advantage of multiple CPUs? Do they constantly read & re-read the same files which almost never change?
How about compiling & linking flags? Do they optimize for the x386 architecture or they optimize for the specific CPU?
I've seen really poorly written programs in C that are slow. I've seen well written programs in Java which perform nicely.
I'm not saying that the results of a benchmark are useless, but use them as a guide rather than a rule. The only real performance you need to worry about is your application's performance.
The Pentium trig instructions are not IEEE compliant (they don't return the correct values for large magnitude arguments). gcc errs on the side of caution and generates slow, software-based wrappers that correct for the limitations of the Pentium instructions by default. Other compilers (e.g., Intel and probably Microsoft) just generate the in-line instructions with no correction. When you look at the claimed superiority of other compilers over gcc, it is usually such tradeoffs that make gcc appear slower.
You can enable inline trig functions in gcc as well, either with a command line flag, or an include file, or by using "asm" statements on a case-by-case basis. Check the documentation. With those enabled, gcc keeps up well with other compilers on trig functions.
Running benchmarks on a laptop is at least strange. Benching simple loops is telling, but telling wxactly what? That java is good for empty loops? A much more interesting comparison would be to see how commonly used idioms scale/perform in different languages. For example, say I extensively use maps, how expensive is map in Java, C#, Perl, etc? Can I insert 200M elements? what is the average time of insertion, etc.
Obviously, anyone who jumps to this conclusion has NEVER ran the Eclipse IDE before.
I like Eclipse, don't get me wrong, but ANY editor that keeps the CPU running at around a 70% load while just editing a file or 2 is just plain wrong.
Ron Gage - Westland, MI
How about you? You care; they only had time to do so much; seems like you would be the obvious choice.
Infuriate left and right
I don't know about anyone else, but most programs in the real application world do alot of string manipulation, and I have seen some pretty shocking results of string manipulation benchmarks showing Java the worst with the C++ class second, and Python actualy leading the pack. It would be usefull to also see the overhead calcs for object management too. Java is so memory heavy we have problems with machines that have 4 Gig of RAM configured.
It's pretty stupid to run benchmarks for a language in a non native environment for the python marks.
Yet again OS News publishes a completely meaningless story.
Everyone is living in a personal delusion, just some are more delusional than others.
8.8 19.0 8.66 4.35 0.7 40.81
compared to his results of
9.8 28.8 9.5 14.9 10.0 73.0
The columns are the same as his; all numbers are times in seconds. Same compiler flags as his gcc test (except for the -fno-cygwin which isn't aplicable). My system is a 1.4GHz Pentium M (compared to the 2.0GHz Pentium M used in his test). No particular precautions against background processes were taken (in fact I was browsing the osnews site while it was running on another desktop).
Things to note:
Yes...I am a rocket scientist.
Lies, damned lies and benchmarks.
It seems quite obvious that you have never written C# code before. :)
.Net framework is tied into Windows?
.Net Framework Class Library. The .Net FCL's source code is freely available, and virtually all the code is non-proprietary. The only considerable portion of that library that is proprietary (ie tied into the OS) would be the GUI library (System.Windows.Forms), understandable since it sits on top of Win32.
Care to offer what parts of the C# language is integrated into Windows? You'll find nothing, because C# is just a language, not a platform, not a library. Perhaps you meant the
The library C# uses (and indeed all managed languages) is the
Microsoft even has an implementation of the common language runtime (and hence C#) on FreeBSD. Additionally, Mono has implementations of the CLR on Debian, RedHat, Mandrake, and SuSE.
Therefore, C# can run on Windows 98,ME,XP,2k, FreeBSD, Debian, RedHat, Mandrake, and SuSE.
Next time, do a bit of research before spoutting your rant.
Wow, and this cat went to Harvard?
The benchmarks are poorly designed microbenchmarks. Why do people pay attention to such things? See Cliff Click's talk on "How NOT To Write A Microbenchmark".
We've done some of our own cross-language benchmarking. The NASA Advanced Supercomputing Parallel Benchmarks are problem statements for serious computation science problems, and solutions can be written in any programming language. We implemented the sparse Conjugate Gradient benchmark, and compared Java against fastest Fortran/MPI implementation on a cluster of 32 linux workstations. Java performed at essentially the same speed as Fortran/MPI (actually a little faster on 16 nodes). Although Fortran was slightly faster at the sparse matrix-vector product, Java communications using Java nio was faster than using the LAM implementation of MPI (the MPICH implementation was much worse than the LAM implementation).
Windows was a good choice for this test, because many of the development languages that were used in this test aren't really mature enough in *nix. (i.e. .Net languages and arguably Java) A better test would be doing both tests on both OS's, because GCC is really more optimized twords Linux, while VC++ is more optimized twords Windows. I would have rather seen VC++ vs. Borderland C++, because that is a more real world business example.
People always rave about the speed of C, etc, etc. But what about fortran. It would have been neat to see how fortran stacked up. Though I am not sure how good MS's Fortran compiler is.
:-)
It is hase been a few years since I did any F77 coding, but half of our shop is F77. We use C/C++ for interfacing, and applications, but the number crunching stuff is done by Fortran.
Every now and then we do a little benchmark b/n F77 and C, and F77, just to remind ourselves why the number crunching is done in such a primitive languge.
I am just happy that I don't do procedural programming, and thus don't have to work with it.
~S
It is amusing that the obsession with raw speed never goes away, even though computers have gotten thousands of times faster since the the days of the original wisdom about how one shouldn't be obsessed with speed. Programmers put down Visual Basic as slow when it was an interpreted language running on a 66MHz 486. It was still put down as slow when it shared the same machine code generating back-end as Visual C++ running on a 3GHz Pentium 4. And still some people--usually people with little commercial experience--continue to insist that speed is everything.
Here's a bombshell: if you have a nice language, and that language doesn't have any hugely glaring drawbacks (such as simple benchmarks filling up hundreds of megabytes of memory), then don't worry about speed. From past experience, I've found it's usually easy to start with what someone considers to be a fast C or C++ program. Then I write a naive version in Python or another language I like. And guess what? My version will be 100x slower. Sometimes this is irrelevant. 100x slower than a couple of microseconds doesn't matter. Other times it does matter. But it usually isn't important to be anywhere near as fast as C, just to speed up the simpler, cleaner Python version by 2-20x. This can usually be done by fiddling around a bit, using a little finesse, trying different approaches. It's all very easy to do, and one of the great secrets is that high-level optimization is a lot of fun and more rewarding than assembly level optimization, because the rewards are so much greater.
This is mostly undiscovered territory, but I found one interesting link.
Note that I'm not talking about diddly high-level tasks in language like Python, but even things like image processing. It doesn't matter. Sticking to C and C++ for performance reasons, even though you know there are better languages out there, is a backward way of thinking.
This benchmark strongly favours languages compiled to native code. A much better place to seek language comparisions is The Great Language Shootout, even if it hasn't been updated for 2 years or so. (There's a more up-to-date version, albeit Windows-oriented, at The Great Win32 Language Shootout.)
When operating on native integer types, Java does in fact perform comparably to compiled code - as it should, since the JIT compilers should produce very very similar code.
However, Java does have some inherent qualities that cause poor performance in typical real-world apps.
-GC is required, and is slower than heap management.
-lack of documented/fixed object layout precludes direct memory manipulation of data - mostly a problem in I/O. This makes block-based I/O virtually impossible without serializing.
-The 'Design Patterns' that have been adopted as Gospel by most in the Java community (at least the J2EE community) are very inefficient - too many layers, too much indirection, too many object copies etc.
- Swing. ugh.
None of these are fatal problems, and I know from experience that it's possible to create very good performing Java apps, but it's not easy. You have to fight the language in some cases, and you can't follow the 'Java Purist' crowd in many cases.
There's no metrics for how these languages perform on various platforms. I think that would be more interesting. Of course, not all languages run on all platforms...maybe that's a disadvantage to certain languages.
I might know what I'm talkin' about, but then again, this is Slashdot...
I've done it, currently my C/C++ is fairly rusty and i'd say it takes me about 10 times as long to develop and debug something in C++ than it would take in Java. If my C++ skills were equal to my Java skills, i'd imagine that this ratio would drop to maybe 5 times as long.
The difference between Canada and the USA is that in Canada healthcare is a right and gun ownership is a privilege.
Python did pretty badly in the tests. The reason is that in Python it takes a long time to translate a variable name into a memory address (It happens at runtime instead of compile time).
The benchmark code has stuff that basically looks like this:
Adding 1 to i takes no time at all but looking up i take a little time. In C this is going to be a lot faster.
Python did really bad when "i" from the example above was a long compared to when it was a long in C. That's because Python has big number support but in C a long is limited to just 4 bytes.
Python did OK in the trig section because the trig functions are implemented in C. It still suffers because it takes a long time to look up variables though.
In real life, variable look up time is sometimes a factor. However, for programs that I've written getting data from the network, or database was the bottleneck.
Tirel is a know karma whore troll. Please do not click on his sig as it is a redirect to goatse.cx. Please look at his history before modding. Do not encourage trolls to post trash like this.
Java and C# is a vendor having you by the balls. How many platforms have C# or java support? How many platforms have C and C++ support? BIG fucking difference there, you have no control over java or C#, sun and microsoft respectively have complete control over them, and as such, over you. What is needed is an OPEN, PORTABLE, safe high-level language.
I know it ties into the GCC libs, but does G++ behave any better/worse than GCC. Comparing VC++ (a C++ compiler) and GCC (a C compiler) is a bit skewed. Also, how about a comparison of GCC in windows VS linux (comparable machines), just to see if the OS has any bearing on things?
Last night I attended a Linux user group meeting in South Florida. One of the speakers, Reinhard Wiesemann from the LinuxHotel (LinuxHotel) in Germany mentioned something very interesting. He said that he'd been using a Unisys system. At some point, when the machine itself became outdated, a colleague showed him Linux. He was able to take his source code, re-run gcc, and *it worked*. He did mention some high-order problem that was fixed by a short program (he didn't know about the gcc flag that would have solved this).
The idea that he could take 15yr old code and have it just work is pretty amazing. Imagine the time and development resources saved on this project alone.
So there's a lot to be said for portable code...
In a similar vein, I've recently been experimenting with the latest Sun Java on an older laptop using Fedora Core 1. I'd always heard that Java was slow, and it was on earlier setups. But it seems to work wonderfully with Fedora. It might be the MPTL or the 2.6.0 kernel I dropped in, but everything seems as good as native QT or GTK+ apps.
Now I don't trust Sun fully after their comments about Linux, but in this case I can separate the technology from the company and say that the portability is a good thing.
Please let this site end up in the Internet dustbin with all the other crappy 'I installed the OS so now I am a reviewer' sites. What value do they actually provide? Deprive this hag of page hits and she'll go away.
I want to delete my account but Slashdot doesn't allow it.
I read large data files into large arrays in memory. (~2 GB files into 2GB of RAM) and for this task Java is unbearable. I am not that stellar of a Java programmer I know, but I am not that stellar of a C programmer either, and C handles this task nicely.
I am doing this on an SGI, so maybe it is their Java interpreter that sucks?
move along, nothing to
...well, actually an acronym - SWT.
I'm curious, why the difference? I presume you are using smart pointers (www.boost.org) and other such additions to the C++ world, and of course you are using STL and the standard library string type. With all of that, you should virtually never need to do pointer arithmetic (though it is still available if you want to) so that whole category of problems is eliminated. So, what else is causing you to be so slow?
Perhaps it is the graphical user interface, though this isn't really fair as C++ doesn't have one. I use Qt and while it costs a substantial amount of money, I find it to be of very high quality. I can write high quality UI in C++ as fast or faster than in Java.
I'm not saying that C++ is better than Java. There are many reasons I'd still want to use Java. However, I just don't see that it is possible that Java could be five times faster to write than C++ for a programmer with the same level of skills in both languages.
Oceania has always been at war with Eastasia.
Third, Java 1.4.2 performed as well as or better than the fully compiled gcc C benchmark, after discounting the odd trigonometry performance. I found this to be the most surprising result of these tests, since it only seems logical that running bytecode within a JVM would introduce some sort of performance penalty relative to native machine code. But for reasons unclear to me, this seems not to be true for these tests.
I dont know why the reasons are not clear to him. Perhaps its because he still thinks the JVM is "running bytecode" and does not understand what JITs did or what HotSpot compilers do. Byte code is only run the first few passes, after which its optimized into native code. Native being whatever the compiler of the c program used to compile the JVM could do. This is fundamental. Which explains his results, and points to a poor HotSpot implementation where trig functions are concerned.
Just look at the assembly. "fsin" and "fcos" (and especially "fsincos") are usually faster than software-based versions, but the latter must be used if you need strict standards compliance. If you don't need that, "-ffast-math" or "-mno-ieee-fp" is needed for gcc (but for MS and intel's compilers, you need to specify an option for strict compliance).
Last time I saw Fortran was on Futurama when Bender order "Ole Fortran" beer http://www.windmeuptoys.com/product.asp?product_id =2600
"Look Lois, the two symbols of the Republican Party: an elephant, and a fat white guy who is threatened by change."
If you use the proper libraries, then C++ becomes as effective as Java. By proper libraries, I basically mean the Qt library. I work in the sector of defense applications, and in our company, the C++ teams are *more productive* than the Java ones from the moment we got Qt!!! It's an incredible library that proves the strength of C++...it even automates memory management.
There are also other good libraries, although not for GUI, like boost.
So, the 'huge advantage' of Java over C++ is only on paper...it's pure hype. Nothing can beat a Qt application in development speed, as you get the best GUI, automated memory management and the flexibility of C++ in one package.
And here some 'gotchas' in Java that you maybe haven't thought of:
In Java, you still have to do memory management!!! despite what the literature says, if you don't make pointers NULL, then objects will not be deleted!!! You still have to check if those pointers are NULL, eitherwise your application will crash...
In Java, you can't use the RAII technique, so you have to explicitely 'shutdown' objects by calling a method of them which is named differently in each object, whereas in C++ the destruction of the stack-based object takes care of the side effects...
In Java, each cast is dynamic, since all methods are virtual by nature (unless declared final, of course). Applications that make heavy use of collections are much slower in Java than in C++, due to the cost of the dynamic cast. The slowness of a Java app rises proportionally to the number of casts the application has...the benchmarks certainly did not show that, did they ?
Quoting the post:
I conducted my own tests pitting Java 1.4 against gcc 3.3 and icc 8.0 using his benchmark code, and found Java to perform significantly worse than C on Linux/Athlon.
Reading the author's analysis of his results, we eventually get to this statement:
Comparing my results to those of the original author, there are three possible explanations for the descrepancies between his findings for gcc vs. Java performance....
So, he's wondering why his benchmark (on Linux, AMD/Athlon) provides different results from the OSNews article (on Windows, Intel/x86). Let's see... different hardware platforms, different JVM implementations... where-oh-where could the differences lie?? He continues a bit later:
The second (possibility) is that the JRE implementation on Windows is much better than on Linux. I would also guess that this is not the case, and would expect them to function with more or less equivalent performance.
Why would he guess that? What rationale supports that view? Somehow I don't think the JVM team at Sun spends gobs of time optimizing their implementation** for AMD/Linux. I'd "guess" that Windows x/86 is a bit higher on their priority list. But maybe I'm missing something.
In short, nothing to see here, move along to a more "insensitive-clod" or "... beowolf cluster"-friendly article.
--Mid
** Lest people forget, the Sun JVM is a reference implementation. You want performance? Try using an alternate clean-room implementation that was built for performance.
I had heard that templates were in the works for Java 1.5...? I certainly hope so. I am addicted to the STL, and generic programming in general.
As for the garbage collection, Java's GC algorithm is quite good, according to execution profiles I've performed - it doesn't block everything to perform the collection.
Why didn't they include ActivePerl?
In the article it rather sounds like they just assumed Python performance would be an indicator of performance for interpreted languages generally, but is there anything to back this up?
This Like That - fun with words!
because the Inferno VOS kicks the crap out of the Java platform.
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
Yay - another microbenchmark. And this tells us what - that the Java JIT guys have the advantage of runtime dynamic performance manipulations of small loops?
I actually use C++ for portability, not speed or generic programming (which are nice to have).
If you avoid platform, compiler, and processor specific features, C++ is even more portable than Java. Java on the other hand tends to drag all platforms down to the least common denominator, then requires the use of contorted logic and platform extensions just to attain acceptable performance.
People seem to have forgotten the original intention of C: portable code.
They should have written their site in one of the higher-performing languages.
RP
I was a bit surprised by this quote in the article:
"Even if C did still enjoy its traditional performance advantage, there are very few cases (I'm hard pressed to come up with a single example from my work) where performance should be the sole criterion when picking a programming language. I"
I can only assume from this that he has never done or known anyone who has done any realtime programming. If you're going to write something
like a car engine management system performance is the ONLY critiria, hence a lot of these sorts of systems are still hand coded in assembler , never
mind C.
Same C benchmark as the OS news one, compiled with GCC 3.2.1 (older than the OS news version), run on an Athlon MP 1666 MHz:
;)
Compile options:
gcc -ffast-math -fomit-frame-pointer -march=athlon-xp -O3 -o Benchmark Benchmark.c -lm
Result:
Int: 7390 ms
Double: 5560 ms
Long: 23070 ms
Trig: 3150 ms
I/O: 1420 ms
Compared to their best I/O (VC#): 9900 ms
On both their and mine systems, the file written and read is only a fraction of the size of the system memory - any decent OS will cache this file completely in memory.
I'd like to know what kind of GCC they used - is that a Cygwin version or something? Their numbers are *extremely* poor compared to my much older GCC here.
But hey, I cheated - I used an OS - probably the "OS" News people didn't think of that... Go figure
The benchmark just runs some math intensive functions in a loop. It really does not test how the compiler deals with complex 'if then' logic, or memory access (this is where Java gets hit pretty bad by array bounds checking). For example if you do something like this:
int x[][] = new int[500][500];
for (int i = 0; i 500; i++) {
for (int j = 0; j 500; j++) {
x[i][j] = i^j;
}
}
You get burned by array bounds checking way more than the following (which is noticably faster).
int x[][] = new int[500][500];
for (int i = 0; i 500; i++) {
int y[] = x[i];
for (int j = 0; j 500; j++) {
y[j] = i^j;
}
}
Such a difference is an example of a 'language difference' between C and Java. Java's semantics for 2D arrays prohibits the compiler from optimizing this the way C or C++ might.
It could have at least filled a 2D array with trig values and maybee done some stencil iteration.
The I/O test is fairly weak as well, and may be reflecting differences in the compilers IO libraries and not the compiler itself.
1) JIT optimizations don't always kick in until a function has been run several times. Since the bechmarks only run once, they are crippled on java.
2) Java's IO function work on UTF-8 or other system dependant character set. So in essence java is doing twice the ammount of work during the IO benchmark.
I'm sure other people will comment as well, but overall these numbers are not that suprising for code that was just copy and pasted from c code. Why do people expect that ANY language will perform well using another languages code.
This has probably been discussed ad infinitum, but let's compare Java to awk and perl.
awk is a fun little language, very portable. awk was supplanted by perl, however, mainly because perl allowed access to a much larger subset of the kernel API than did awk.
I don't use Java much, but it seems to me that Java's great failing it that it doesn't allow (easy) access to system calls. If I want to stat a file, open a fifo, etc., I am forced to either link in C code with JNI, or use something like system(). Perl had no problem in being a little more platform-dependent, and I think it served them well.
I wonder if this aspect of Java will ever change, and we might see kernel-specific sections of their standard library. But then again, I don't keep up with Java as I should, so perhaps we have this already.
Raw performance will ALWAYS be an issue. If you can handle 100,000 hits per day on the same hardware that I can handle 1,000,000 (and these are not made up numbers, we see this kind of discrepency in web applications all the time), then I clearly will be able to do MORE business than you and do it cheaper.
You raise excellent points. For many enterprise and server applications, performance is an issue. But I never said one should care nothing abut performance, only that in many applications the cost of the coder also impacts financial results.
For the price of one software engineer for a year (call it 50k to 100k burdened labor rate), I can buy between 20 to 100 new PCs (at $1000 to $3000 each). If the programmer is more expensive or the machines are less expensive, then the issue is even more in favor of worring about coder performance.
The trade-off between the hardware cost of the code and the wetware cost is not obvious in every case. A small firm that can double its server capacity for less than the price of a coder. or the creators of an infrequently-used application may not need high performance. On the other hand, a large software seller that sells core performance apps might worry more about speed. My only point is that ignoring the cost of the coder is wrong.
These different languages create a choice of whether to throw more hardware at a problem or throw more coders at the problem.
Two wrongs don't make a right, but three lefts do.
Database aside, you're talking Internet -- web servers, ftp servers, and so on. And games (quake) don't count. Contrary to what many of the "developers" that hang out here may think, the computer application world does not revolve around internet. There are many many more applications out there like the software that runs a manufacturing process or the 911 call center. If I work in a steel mill and the process is controlled by Windows based apps, it makes sense to use a Windows optimized language.
"Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
I ran four tests using Portable.net and mono. For lazyness reasons I only ran Int and Trig benchmarks. All tests were performed on a 2.4ghz p4.
First, I compiled Benchmark.cs using cscc (portable.net) and mcs (mono).
I then ran each binary with mono and ilrun (portable.net). Results are interesting.
Portable.net compiler: cscc -O3 Benchmark.cs
$ ilrun Benchmark.portable.exe
Int arithmetic elapsed time: 12996 ms
Trig elapsed time: 28700 ms
$ mono Benchmark.portable.exe
Int arithmetic elapsed time: 16235 ms
Trig elapsed time: 4534 ms
Mono Compiler: mcs Benchmark.cs
$ ilrun Benchmark.exe
Int arithmetic elapsed time: 13784ms
Trig elapsed time: 27939 ms
$ mono Benchmark.exe
Int arithmetic elapsed time: 15994 ms
Trig elapsed time: 4596 ms
As you can see, Portable.net has slightly faster Int math, but crumbles under the trig functions. There is no significant difference between the compilers.
the Portable.net runtime had a serious bug where the time calculated was an order of magnitute out. I used the unix time command to get a more accurate result.
It would be interesting to do this comparison using Microsoft.NET as well. I would assume Microsoft.net would absolutely rape these results.
n.b. Please note this was not a comprehensive benchmark. I disabled some of the tests because I didn't feel like waiting (So sue me), while X, xmms, xchat, etc were running.
Is it any surprise that Java is so slow?
Mac OS X and Windows XP working side by side to fight back the night.
That runtimes less than one second long, and programs that are small enough to sit in cache are probably not good indicators of true compiler performance. Even if you ran those tests 100 times, you are too close to the timer interrupt resolution to get any meaningful results.
Try filtering up to 100M, at least (I can't think of any way to use the cache less efficiently that isn't trite).
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Look at the code for the python benchmark. Notice that the value for intResult will overflow to a long (ie arbitrary precision integer) when the value of i exceeds 100,000. Most of the "int" benchmark is being done with python longs.
We weren't quite ready to release it, but we've been working on a language performance comparison test of our own. It is available at:
http://scutigena.sourceforge.net/
It's designed as a framework that ought to run cross-platform, so you can run it yourself. We haven't added it yet, but I think we really want to divide the tests into two categories. "Get it done" - and each language implements it the best way for that language, and "Basic features comparison" - where each language has to show off features like lists, hash tables, how fast function calls are, and so forth.
It's an ongoing project, so new participants are welcome! I would appreciate it if comments went to the appropriate SF mailing lists instead of here, so that I can better keep track of them.
http://www.welton.it/davidw/
Integer and floating point math are really very similar, between native compiled languages (like C/C++) and managed languages (like Java/C#/VB.Net/etc.).
This benchmark really isn't very useful, at least as a comparison to see if managed languages are "ready" for real development work.
A much better benchmark should include realistic use of dynamic memory allocation (free/malloc vs. new/gc), arrays, etc. This is where managed/unmanaged languages really crucially differ.
I know it's petty, but... ;)
You found your particular combination of compiler and Java Virtual Machine to perform significantly worse.
Native compilers and (to a lesser extent) JIT compilers could eventually be written that make the speed different negligable. Perhaps they already have been; It's been nearly 3 years since I coded in Java. :)
His benchmark isn't fair, he's omitting the fame pointer on VC++ but not gcc. How is that fair?
AWT has terrible architecture and consists of heap of fixes sitting on top of the original JDK 1.0 quick and dirty API.
SWT is AWT done right, has very reasonable architecture and API that does not force to create zillions of objects to perform basic GUI operations. This leads to better performance especially noticeable on less powerful computers.
Actually he used very agressive CFLAGS for VC++ but much more passive (but still agressive) flags for gcc.
Guido van Rossum noted in an interview the following statistic, and I think it bears considerably on appropriateness:
So then, unless you quantify the types of apps you build, the team you use, and the results that are expected, my experience has shown me that most of the time, for business apps, it's overkill. Now, if you're in a dev team at a software company, well then, I could consider the other side.
...tizzyd
When comparing ICC and GCC one has to take into account
that they have very different compilation defaults.
For floating point code ICC defaults to non-IEEE
floating point mode, whereas GCC is IEEF FP compliant
by default. In IEEE mode a lot of FP optimizations
CANNOT be performed, trigonometric x86 instructions
cannot be used, etc. So for a fair comparison
GCC needs the -ffast-math flag added.
Also GCC needs -fomit-frame-pointer, ICC does not
generate a frame pointer by default.
Doing and ICC vs. GCC comparison without taking
the above issues into account is plain silly.
BTW, no version of GCC supports pipeline scheduling
for Pentium4! If anybody at Intel reads this
maybe is time to write that code for GCC, it's
not that hard, a few days of work.
Perhaps it would be more correct to say that OOP is a SB when realized by a language like Python (it definitely isn't w/ C++ and Java).
... but only in very small handful of languages, mostly Python and C++ in the hands of a true expert. (I'd bet ruby qualifies but I've never used it, and Perl can qualify but you end up needing to implement good object semantics on your own, which is both good and bad. Of these languages Python probably has the lowest barrier to entry for the maximal OO power.)
;-), programming in Python is so much less frustrating then other langauges. A lot of people have a lot of unlearning to do before they can realize the benefits.
Actually, over the past year or so I've come to the point where I also believe this. The advantages of OO promised in the 80's are just now materializing...
This fact is largely going unnoticed, but it'll probably penetrate the mainstream here in another five years. I sure hope so
Python with good OO in the hands of a competent person magnifies their power almost beyond belief. I'm building an innovative application in Python, have been over the last year or so, and I wouldn't even be a 10% as far as I am now in C++. I am easily matching the output of a full-time 3 or 4 man C++ team in my spare time with Python. Once this really cranks up in the mainstream I expect to start seeing some real differences in programs.
Perhaps Java 1.4.2 is slow in the trig section not because of a bad trig implementation, but because of how it makes function calls.
Honk if you're horny.
Now, I agree that I would not go back, but realize that you still depend upon someone out there, writing fairly low-level hooks for the DirectX calls in the GPU to get the performance you want. Yep, someone's still hacking assembly out there.
But in the end, for most business apps (the kind that care about whether you're using java, c#, dot net, or the langue du jour), it's not about DirectX. It's about cost.
...tizzyd
That's a feature built into Java 1.5, but you can get a test reference implementation which is about 96% of the features now to try it out. It has a really clean syntax and provides the benefit you seek.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Yes, but if I get mine done today, and you don't, you can't compete with me because you aren't there yet.
I get first crack at the market, by the time you catch up I could already be improving my performance by rewrite critical portions in a higher performance language.
Then not only do I end up with your same performance, but I got into the market first.
That's why your
OS
Word processor
Browser
Mail Client
Terminal emulator
Shell
Spreadsheet
Image Editor
et fucking cetera
are written in C/C++
Speed matters.
It's +5 Funny that *you* don't take it seriously.
Sincerely,
C-Zealot
Reminds me of my 6th grade 'science fair' project.
I took a couple different compilers, languages, did some loops and math and such, timed them all.
"Which computer language is the fastest"
About half way through the project I realized how big of a waste of time it was.
What kinds of things should you be testing?
Speeds of function calls???
Implement various sorting algorithims?
Audio/Video compression/decompression?
When it comes down to it, it's all the same math, and any good compiler is going to come close to making the same darn code.
By now, we all know that you use one language for one thing, and another language for another. For various reasons.
A hammer is your only tool if all your problems are nails, isn't the cliche?
The Great Computer Language Shoot-Out compares many more languages in a variety of benchmarks, although it doesn't have C# yet. The key thing to note is that high-level compiled languages like ML or Scheme perform slower than C, but much much faster than interpreted scripting languages like Perl.
-- Ed Avis ed@membled.com
if written well, swing is a very good toolkit. i did a few (unscientific) tests, and discovered that the better way to use swing was not use multiple subclasses of the widgets rather than having one big class. write anything you need in the constructor, and put the methods i nthe subclass. also, if you use threading with swing, you'll notice a huge increase in performance. but that is not the easiest thing to code, and people would rather let the IDE do it for them. i don't kow.
My problem? I was perfectly gruntled, until some numbnuts came by and dissed me.
- seeds (people who can create new approaches quickly, grasp lots of ideas, but cannot fine tune or produce solid code to save their lives),
- coders (people who need more structure around the problem, but can methodically solve it and produce good code), and
- polishers (people who are so detail focused and methodical that they can structure tests, analyze inputs, and find needles in haystacks)
Sadly, the last group seem to be disappearing, where instead we throw batches of Bangalore and Indian engineers or automated test tools at the app, getting it to meet the thresholds of the business spec, and then we move on.So, for something like Mozilla, the performance to cost is not the main business driver. If it takes 1 sec to show a web page vs. 0.5 sec, most users simply don't care. Thus, the cost relationship.
...tizzyd
Generics, *NOT* templates are coming Tiger (JDK 1.5). There is a difference.
From the poster's benchmark:
The Java code was compiled with J2SE v 1.4.2_03 javac -g:none and executed with java -server
It's my experience that the kind of optimizations that are applied when the -server JVM option is enabled usually slow down the benchmark.
The client VM is usually faster on x86 machines.
But I really don't know why.
Y35! L!n0X T0rV@lDz 1z teh rul3z h4xXh0r!!!1!
Comparison against gcc, gcj and Java 1.4.1 on the same host:I was somwhat surprised on the difference in the trig tests, as both appear to use libm. Not surprised that the IO was slower, the Java IO classes are nifty but do add quite a bit of overhead compared fputs/fgets.
(Sorry about the formatting, it was the best I could do)
Changing this to 'linesToWrite = [myString] * ioMax' dropped time on my system from 2830ms to 1780ms (I'd like to note that I/O on my system was already much faster than his *best* I/O score, thank you very much Linux)
In the trig test, I used numarray to decrease the runtime from 47660.0ms to *6430.0ms*. The original timing matches his pretty closely, which means that numarray would probably beat his gcc timings handily, too. Any time you're working with a billion numbers in Python, it's a safe bet that you should probably use numarray!
I didn't immediately see how to translate his other mathematical tests into numarray, but I noted that his textual explanation in the article doesn't match the (python) source code!
(My system is a 2.4GHz Pentium IV running RedHat 9)
Hate stupid software on freshmeat? Laugh at
The windows version of Python is much slower. Testing with Python2.3 + psyco on a 2.4ghz p4 running Linux 2.4.20 yeilds impressive results
$ python -O Benchmark.py
Int arithmetic elapsed time: 13700.0 ms with
Trig elapsed time: 8160.0 ms
$ java Benchmark
Int arithmetic elapsed time: 13775 ms
$ java -server Benchmark
Int arithmetic elapsed time: 9807 ms
(n.b. this is only a small subset of the tests- I didn't feel like waiting. Trig was not run for java because it took forever.)
To dismiss a few common myths...
1) Python IS compiled to bytecode on it's first run. The bytecode is stored on the filesystem in $PROGNAME.pyc.
2) the -O flag enables runtime optimization, not just faster loading time. On average you get a 10-20% speed boost.
3) Python is a string and list manipulation language, not a math language. It does so significantly faster than your average C coder could do so, with a hell of a lot less effort.
Every one who proclaims a Silver Bullet is doomed to be considered a fool.
Sorry, but Brookes was right, there is no Silver bullet, and Python certainly is not it.
1) BPM in Python
2) Adaptor Strategies and Toolkits in Python
3) Distributed Transactions in Python
4) Clustering and Caching strategies in Python
Java is irrelevant as a language, its the support and tools it has that makes it a good platform choice.
An Eye for an Eye will make the whole world blind - Gandhi
Someone has to be *really* comfortable with C++ to get the benefits of it. Especially, you have to forget all your standard C/Java techniques and embrace modern C++. Once I let go and just started writing code that naturally flowed around the STL, my productivity went up enormously. I constantly see myself doing things that just wouldn't be possible in Java. For example, I was writing a wrapper class for a C data structure. I didn't want to write and maintain a whole bunch of get/set methods, but I didn't want to make the variables public. So I just wrote a dozen-line template to auto-generate the get/set methods for me, and that was that.
On the other hand, C++ is still a very irritating language at times. Its biggest problems are:
- Hairy syntax (a little bit better in Java)
- Complex template semantics
- The need to have to deal with "classic C++" interfaces
- Lack of advanced features like lambdas, type inference, etc.
A deep unwavering belief is a sure sign you're missing something...
I see just one small issue with the benchmarks. Microsoft claims, that all .NET languages are compilled at the runtime. This means, that the first pass of the execution through the function has a compile time added on top of the execution, which falsifies somewhat the .NET execution time benchmark. I did some simple tests that confirm this. To my surprise, .NET languages are actually faster than Visual C++, Borland C++ or GNU C+ for a simple 1/n series calculation without visible loss of accuracy. Don't ask me how it is possible. I don't know, but it is a fact that my benchmark shows. My best guess would be that the just in time compiler is better in getting code optimized for the CPU in the particular machine it runs or maybe it is better in filling the cache. The key of the benchmark is to write software in such a way that it runs through the function at least two times. The first time it runs just to allow just in time compiler to compile the code and then it runs subsequent times to measure performance. Below is the schematics of my benchmark:
// This is to allow .NET "just in time" compiler to compile the benchmark function
// CurrentTime is a placeholder here for a system time function in ticks // lprt is a placeholder for a nice formatting print here
// This is the body of the benchmark
.NET is part of the execution time but I disagree. My position on this is, that in most real life cases the software runs into the particular functions many times thus creating long exectution times. It is rare, that a signle function call creates long exectuion time that is annoying to the user.
double benchmark(int number_of_iterations);
void main (void)
{
Time start,end;
double outcome;
benchmark(1);
for(int i = 1; i < 11; ++i)
{
start = CurrentTime();
outcome = benchmark(i*1000000);
end = CurrentTime();
lprt (i,outcome,end-start);
}
}
double benchmark (int number_of_iterations)
{
double s,t;
s = 0.0;
t = 1.0;
for(int i = 1; i < number_of_iterations; ++i)
{
s += 1.0/t;
t += 1.0;
}
return (s);
}
As you can see above, I run the benchmark function once with counter of 1 and ignore its outcome before starting to measure time. The key is to allow compiler to compile the benchmarking function before running actual benchmark. Once it is done, I run then the benchmark 10 times for succesively larger counter from 1 billion to 10 billion and print number of iterations (in billions), the accuarcy and the time it takes to run. The idea here is that under the assumption that the benchmark time is related to number of iterations as a linear function I can easily find linear best fit function between number of cycles and run time in the form of
time = a * number_of_cycles + b
and then use value of a as a measurement of the benchmark. The value of b is good check, how the benchmark behaves. If it is large, then something went wrong. In my case it was always close to zero. I'm now away from my home computer and I don't have all the compilers, that were tested in this article, so I can't repeat those benchmarks modified to this method at the moment, but you guys might try to do it yourself.
Some people might challenge this by stating that the compile time for
Best regards.
When it's the foundation of Cocoa can anyone tell me how come this wonderful language isn't in the competition?
It saddens me greatly that they failed to consider the Shakespeare Programming Language (/. story here ) worthy of inclusion in their performance tests. *sniff*
Granted, since SPL eventually becomes C, the C scores would apply, but still...
--- Standard disclaimer applies.
I find that with C++ you spend a reasonable amount of time finding buffer overflows, with Java you don't have that problem.
However, you spend much more time configuring BEA, the jar files, misc crap like that.
I've been writing in Java for years, and C,C++, etc. even longer. But I've switching back to C and really liked it.
Now if I need a server I can dumpster dive and get a Pentium 200 MHz with 64 MB ram. After installing Slackware I have a great machine!!!
With Java the overhead and bloat would slow it to a crawl. But with C, my server runs very well- even under heavy load.
On the same machine, with no optimization at all to GCC, I get:
;)
Compilation:
gcc -O0 -o Benchmark Benchmark.c -lm
I/O: 1590 ms
Almost the same. Clearly, the thing that takes time is the system calls.
If he used Cygwin GCC it's no wonder that it sucked. But my results are still about an order of magnitude better than (optimized!) VC++ and C#.
Anyway, this comes as no surprise to me. System calls on Win32 stink - their performance as well as the raw API.
Yes, I spend too much time getting things to run fast on that platform - I have my scars - this is just an opportunity to rant about it
Yea its definately not a comparison on languages. Its more a test on compilers. Problem is it ignores the different ways these languages access memory, which can have a more severe effect on performance. Java and C# have well optimized bounds checking and garbage collection, but there are still circumstances you may come across where these bytecoded languages cannot optimize memory access to the speed of C. Sometimes overhead just can't be avoided.
All fine, and excellent, and wonderful -- and it really surprised me Java could be so fast. Guess I was wrong about it. I took a mental note to compare Java and Python as my next language...
BUT, I was more interested in compile times (because these affect development tremendously). Does anyone know about one such benchmark? Please have in mind I don`t have the slightest interest in execution times... thanks in advance for any hint.
BTW, I may do it one day, but right now I lack the time...
why, when cornered, do Javaphreex start
shouting "bigot" ???
you are like apple and kde zealots.
Not only that, but it helps to have a benchmark that actually tests the things that are claimed at being tested.
It's a Good Thing (tm) this didn't make the front page.
The author states "I am by no means an expert in benchmarking; I launched this project largely as a learning experience" and it shows. The man has an associates degree in computer science, and a Ph.D. in psychology. His list of publications are his dissertation, a single published paper, and excerpts within a 15-year-old travel guide.
Just a cursory glance at the first page of his article shows that he has no clue as to how things work. He states "I first tried to eliminate the CLR from the Visual C++ benchmark by turning off the language's "managed" features with the #pragma unmanaged directive, but I was surprised to see that this didn't lead to any performance gains." If he cannot understand why something that generates a few thousand CPU instructions of initial overhead doesn't change the speed of an I/O- and loop limited program, he isn't skilled enough to interpret his own results.
His benchmarks never actually test his first, second, third, or fifth question. His fourth question is actually addressed better when his contrived test is compared on relative measure with his two Java tests.
Look at his benchmark programs (found here). Some of those test can, should, and will have compiler-specific optimizations, having nothing to do with the language. General 'counting loops', which is the only thing he is using, have long been known to produce bad benchmarks. He claims to be testing 64-bit floating point math, but in fact, many of his examples use 80-bit floating point.
Just for fun, look at his VC and Java 1.4 floating point tests. Now look at his compiler options. It is painfully obvious that the compiler saw "He explicitly said this is a Pentium 4, I can use parallel floating point instructions!" where the other compilers could not. Saying that those languages are inherently faster than the other compiled languages is lunacy.
This is hardly news. This is a BAD example of benchmarking, and would be given a poor grade in a graduate level CS class.
//TODO: Think of witty sig statement
Yes, let's give Torvolds credit for every corny computer industry cliche
So you get the maximum performance out of every program. You want it when?
There is a fine line between raw performance and the time required to code. If I'm first to market with a slower product I'm at an advantage. If it cost you 10X as much to code your much optimized solution I'm still at an advantage provided hardware is fairly inexpensive compared to programmer time.
Perl programs write really fast code because they tend to use hashes more than c/C++ programmer. Perl isn't compilied.
Its not as simple as you make it seem.
This comment alone marks you as a troll, but I'm bored so I'll reply to you anyway.
Java isn't and never will be as fast as C/C++
This is pure nonsense. JIT compiling is one of the most exciting research areas in practical computer science at the moment. With a pre-compiled program, written in a static language such as C or C++ the compiler has only one shot at optimising it and has to do so with no knowledge of the data on which the program will be executing.. With a JIT compiled program, it is possible to continue to optimise the code as it runs. This means that the code will be slower the first time it runs, but (theoretically at least) will get gradually faster over time. An analogy for this would be the difference between defragmenting your hard drive once at boot time, which will give you a one-shot performance boost, and dynamically re-allocating files so that the most used ones are on the fastest parts of the disk, which gives you a performance hit at first but then a gradual performance over time as the system optimises your disk.
I am TheRaven on Soylent News
first, it's a reference implementation. second, sun spends far more time optimizing hotspot for sparc solaris than x86 linux. why? because ibm and bea do x86 linux so much better. this isn't a useful benchmark until it includes either ibm or bea jvms.
Unfortunately, the author compiled python using cygwin, which is extremely *slooooow*. Why on earth he chose to download the python source and compile it under cygwin instead of using the windows binaries provided on python.org that the rest of the world uses remains a mystery. Based on the same false premises that lead gentoo users to compile night and day, perhaps? ;)
It is also misleading that python is often referred to as "fully interpreted" (often several times in one sentence) and compared to "semi-compiled java". Python is byte compiled, just like java.
As a programmer who uses python almost exclusively, I have rarely have need to optimise, although I have written a few C extension modules (which is fairly simple), and occasionally used psyco (which is trivial, and in my experience has almost always resulted in a big increase).
Python is slower than the languages tested in many situations, but the results of this particular benchmark aren't worth the electronic paper they're printed on. Oh well, it's pretty damn hard to compare language speed, I guess Christopher guy did his best.
Almost all my work programming is function calls and string, dictionary, and list manipulations in a single and multi-threaded apps. A little integer, very little float (money), no trig. In this case the only useful benchmark in the article is I/O.
My guess is that a majority of programmers aren't developing apps heavy in integer, float, or trig. So this benchmark article suggest that we not develop the next iteration of Quake in python.
Does anyone know of a multiple language benchmark relevant to the rest of us lumpen proletariat programmers?
news at 11.
Yes, performance might be important, but then, many times it is completely irrelevant. I started out with a C64, Amiga, 286, 486, P60, ... uhh, i actually have to check to find out what processor I am currently running: Athlon 2400. I couldn't care less how fast todays machines are, as it is fast enough for 99% of my tasks.
Now my employer has lots of transactions to process etc., still we are using Java, and not for performance reasons.
But what I, my employer, and 99% of the developers care for is time to market. Because that means money, more free time etc. Always. Humans can do something else while waiting for the machine to spew out some results. If you don't have enough time, most of the time buying faster hardware is an option. A team next to ours bought new hardware for CHF 2 mio as their app was too slow for the original hardware. So who cares, development cost was something like CHF 8 mio. Without having the app ready, an annual CHF 100+ mio business (revenue) would have been severely crippled.
So tell me which language needs the least lines of code and has the most useful libraries for reuse.
My guess is python, ruby, smalltalk and lisp might be some very productive languages. Java, C++, Perl might have huge libraries attached to them.
I am sick of these language performance comparisons! Tell me which language to use to save precious development time.
That would explain a lot then.
Any recent version of Visual C++ supports "intrinsics", i.e., conversions of a function call directly into a specific machine code instruction, or "perfect in-lining" if you prefer.
That means that if you use something like abs(x) in C++, these compilers would convert directly into an instruction to get x to the head of the floating point stack, then a FPU abs opcode, with no overheads at all. Same goes for trig functions and other variations supported by the FPU.
Moreover, IME the VC++ optimiser is quite smart about intrinsics, and in lengthy calculations will often arrange for the right values to be coming to the head of the stack in the correct order as cheaply as possible, even if it means planning ahead a few instructions. If you look at the assembly language output from VC++ for a numerical computation, it tends to have a series of instructions to stack what it needs, with the occasional calculation opcode thrown in between them, and then a whole series of neatly co-ordinated calculation opcodes at the end.
I don't see how any language that has function calling at all could keep up with this low-level, direct-to-FPU approach, and surely any language using software emulation of floating point code won't even come close.
I don't know whether VC# and VB.Net use the same trick, but given how much effort MS put into sharing things across the product range prior to releasing VS.Net, it must be a good bet. There are some advantages to writing platform-specific code. :-)
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
I'd like to see Meastro using SWT, it'd be heaps faster. But... is Java3D "compatible" with SWT?
Why are the two best Borland languages never included in benchmarks? Maybe in just the odd-ball that doesn't use C++, Java, or Micro$oft. - TMK
By replacing the Reader/Writer classes in the java benchmark with their InputStream/OutputStream counterparts I realized a 24% IO performance boost when running the 1.4 JVM with the -server option. The Stream classes don't bother with the unicode conversions that the Reader/Writer classes do. Since the other benchmarks didn't perform unicode conversions (at least the C/C++ ones - can't speak for the other langs), this seemed like a reasonable modification.
Curiously, without the -server option, this resulting in a 78% performance HIT.
- Marty
So you have found one example where Python is more efficient, I could easily provide examples which demonstrate the opposite, it proves nothing.
I'm not even a Java programmer and I can see your problem. You lack a quality testing methodology. If all you're doing are hand tests to assure yourself that your code is correct, I pity your customers, whether you program in C++ or Java or Python.
Type safety alone (especially the faux "safety" you get in C++) is not an adequate replacement for a comprehensive testing methodology.
Jeremy
Looking for a Python IRC bot?
A few posts mention that .NET is for Windows only. That's completely untrue. .NET executables can be run on Linux, FreeBSD, MacOS X, maybe even other platforms in near future. .NET Runtimes:
Mono - http://go-mono.com - C# and VB
DotGNU - http://dotgnu.org - C#
For Windows, the Microsoft's .NET Framework is the winner, being almost twice as fast as Mono, with DotGNU being the slowest, but I've only done synthetic SciMark2 benchmarks.
(Also don't forget you can compile Java to .NET assembly :)
I mean, the functions are so simple, the code generated doesn't stress anything. Not any of the 'advanced' compiler, or even architecture features. None of the good features of a JIT.
I mean seriously, they do math on all the ints from one to one billion. Why even bother? Adding large 32 bit ints takes exactly the same amount of time as adding small ones (but I guess you save one variable by doing math with the counter. Or one extra line of code saved)
I'm sorry, but this is the most pointless compiler benchmark ever.
A good language comparison would be to have a bunch of groups of people try to code up the best implementation they could in whatever language, of some complex problem, and use that as the baseline.
autopr0n is like, down and stuff.
You are completely wrong. Java programs are taken and converted into machine code on the target platform. Saying that it's "the same as 'C' code" because the JVM is written in C is like saying that if I were to write a C++ compiler in Python, running the resulting binaries would be the same as running Python code.
In other words, idiotc.
autopr0n is like, down and stuff.
normal:gcc Benchmark.c -o Benchmark-c -lm
OmitFP:gcc Benchmark.c -o Benchmark-c -lm -fomit-frame-pointer
Optimized builds: (replace the X with the number(2,3) or symbol (s)) for example: O3norm:gcc Benchmark.c -o Benchmark-c -lm -O3
OXnorm:gcc Benchmark.c -o Benchmark-c -lm -OX
OXOmFP:gcc Benchmark.c -o Benchmark-c -lm -fomit-frame-pointer -OX
FHexpe:gcc Benchmark.c -o Benchmark-c -lm -march=pentium3 -O3 -pipe -fomit-frame-pointer -fforce-addr -falign-functions=4 -fprefetch-loop-arrays (http://www.freehackers.org/gentoo/gccflags/flag_
FHEOFP: (FHexpe with -fomit-frame-pointer) gcc Benchmark.c -o Benchmark-c -lm -march=pentium3 -O3 -pipe -fomit-frame-pointer -fforce-addr -falign-functions=4 -fprefetch-loop-arrays
Unsafe: UnsOFP with out (you guessed it: -fomit-frame-pointer) gcc Benchmark.c -o Benchmark-c -lm -march=pentium3 -fforce-addr -falign-functions=4 -ffast-math -O3
UnsOFP: gcc Benchmark.c -o Benchmark-c -lm -fomit-frame-pointer -march=pentium3 -fforce-addr -falign-functions=4 -ffast-math -O3
Bold indicates either leader or within 100ms of leader, Italic indicates a case where omit-frame-pointer is actually faster.
Integer test:
:) omit-frame-pointer just doesn't seem to be worth losing the debugging capabilities. Suprisingly, after the -ffast-math ones, #3 is simply -O3 without -fomit-frame-pointer (but only by .4 seconds) And another suprise was the IO: I highly suspect that that was mosly being cached in RAM & 7200 rpm drives
normal: 20990 ms OmitFP: 21410 ms
O3norm: 15230 ms O3OmFP: 15190 ms
O2norm: 15200 ms O2OmFP: 15490 ms
Osnorm: 15190 ms OsOmFP: 15250 ms
FHexpe: 15200 ms FHEOFP: 15500 ms
Unsafe: 15430 ms UnsOFP: 15250 ms
Double test:
normal: 22000 ms OmitFP: 23010 ms
O3norm: 17540 ms O3OmFP: 17080 ms
O2norm: 17640 ms O2OmFP: 17370 ms
Osnorm: 21300 ms OsOmFP: 18770 ms
FHexpe: 17180 ms FHEOFP: 17300 ms
Unsafe: 11660 ms UnsOFP: 11710 ms
Long test:
normal: 48510 ms OmitFP: 48690 ms
O3norm: 37020 ms O3OmFP: 37770 ms
O2norm: 38040 ms O2OmFP: 38490 ms
Osnorm: 37580 ms OsOmFP: 38290 ms
FHexpe: 38480 ms FHEOFP: 37910 ms
Unsafe: 37560 ms UnsOFP: 38430 ms
Trig test:
normal: 8360 ms OmitFP: 8400 ms
O3norm: 7810 ms O3OmFP: 7890 ms
O2norm: 7970 ms O2OmFP: 8040 ms
Osnorm: 9110 ms OsOmFP: 9000 ms
FHexpe: 7850 ms FHEOFP: 7990 ms
Unsafe: 5600 ms UnsOFP: 5590 ms
IO test:
normal: 2110 ms OmitFP: 2200 ms
O3norm: 1950 ms O3OmFP: 2050 ms
O2norm: 2120 ms O2OmFP: 2020 ms
Osnorm: 2120 ms OsOmFP: 2200 ms
FHexpe: 2190 ms FHEOFP: 2120 ms
Unsafe: 2330 ms UnsOFP: 2040 ms
Total
normal: 101970 ms OmitFP: 103710 ms
O3norm: 79550 ms O3OmFP: 79980 ms
O2norm: 80970 ms O2OmFP: 81410 ms
Osnorm: 85300 ms OsOmFP: 83510 ms
FHexpe: 80900 ms FHEOFP: 80820 ms
Unsafe: 72580 ms UnsOFP: 73020 ms
"Unsafe" is certainly the fastest, and according to all the printed values, it is accurate, but there is likely a reason why the man page describes -ffast-math as unsafe: I certainly wouldn't use it in anything I would use more than once
Benchmark may be very tricky and variant, you have to understand very well what are you doing. Could someone with experience in bechmarking verify is this bechmark has any sense? The author doesn't seem to known very well the languages he is benchmark. For example he doesn't seem to know that the JVM actually compiles on the fly and the VB bechmark is writen in VB6 old style.
"I think this line is mostly filler"
Compiler flags based on Apple TN2086: http://developer.apple.com/technotes/tn/tn2086.ht
Removed machine & username from command prompt only.
That's why the real test should be: Python/Linux vs Perl/Linux vs Tcl/Linux vs GCC/Linux vs OCAML/Linux vs Erlang/Linux vs Haskell/Linux vs whatever-is-available-in-source-for-linux.
In that case you can run the test on PIII, Athlon, P4, G3, G4, G5, USII, UIII etc - wherever linux is ported. Then you can see how different hardware helps to performance. And you could see how different compilers utilize thos hardware resources.
Less is more !
Hi,
this is completely off topic, but du you know a vt220 emulation with user definable fonts (softfonts?) implemented?
I tried some teminal emulators, e.g. jta, to replay some old ascii art movies, but none had this feature.
t.i.a
It's just Sun's implementation which is b0rked. IBM's JVM for Linux computed that trigonometic test eight times faster on my box...
The whole test ran in just 38.2 seconds, down from Sun's 131.5. (yes, I used the -server flag)
Real men use LUTs!
Or they use hardware which converts the number into a time index, then reads a value off of an analog signal generator. Less then one clock cycle!
autopr0n is like, down and stuff.
And doing runtime type analysis because your language lacks the necessary functionality to perform static analysis is retarded. Instead of going one-on-one with a straw man, maybe you could try staying on topic.
Scalability and code reuse.
Often times when someone writes something useful, other programmers want to use it as a building block in larger applications. The problem is that the interpreted languages make this kind of system building almost impossible. While your WonderWidget might work well for parsing a single file or two, the performance differences really begin to show up when it gets called for every file in a directory, or is scaled up to handle large files. An order of magnitude difference is trivial when used in an end-user application and the elapsed time is only a second or two. But that difference becomes really significant when working with large amounts of data.
I know people downplay the performance disadvantages of interpreted languages, but this is still a very real problem for businesses. A small corporate system has a thousand modules; the larger ones have upwards of 30,000. A single millisecond difference in execution speed makes a full second difference in user response time. While one second isn't likely to be noticed by a consumer-grade web user, online help desks can't wait 30 seconds for a page to load.
I guess I should qualify this further. I know of projects using interpreted languages that have failed - the application did work - albeit, it took 30 seconds (best time) to 5 minutes to respond to a single keystroke. It was later abandoned because despite buying faster hardware, the programmers couldn't get better than a 20 second response time. An average improvement of just 15 milliseconds per module would have saved the project!
The society for a thought-free internet welcomes you.
If speed isn't an issue, why is Java's design rooted in C++? There are far better designed languages than Java around. If I wasn't looking for a fast application, I'd probably consider using Python.
Java offers little advantage over C++. It has a garbage collector, has some useful libraries built in, and cross-platform compatability, and to some degree, makes pointers safer (at the loss of functionality). But that's about it. The Java libraries don't do anything so special that there isn't a standard C++ library to match it. And Java lacks pointers, polymorphism, and yet, for some rather obscure reason, keeps basic types.
I'm not saying C++ is a good language. It isn't, really. But nor is Java, by any stretch of the imagination. C++ has certain features, such as basic types, that are the legacy of C, and in some programs, are needed for speed purposes. Java has them because... Well, actually, there isn't a compelling reason to have them at all. And that's just one feature of the language. Java isn't very well thought out at all.
I wouldn't say I was an anti-Java zealot, but I can't see many advantages to using Java.
Comparing Perl to Java is foolish
Statements like these are given without justification and only reflect the ignorance of the author.
Since when does Awk provide packages, hashes, symbolic and hard references, OO constructs, closure, etc.
Please be specific on Perl's missing attributes that backs up your statement not meant for large projects at all
Just curious, what is the largest Python project (in terms of active programmers)?
I'm loathe to use Perl as production, considering most Perl code cannot be understood 2 weeks after it's written.
What specifically do you loathe about Perl? What features or attributes does, lets say C have, that Perl does not that contributes to readability issues?
I think you have limited experience in Perl and people tend to loathe that which is unfamiliar. Try to be more honest.
I think enough other people have already pointed out that the kind of comparision done in this article is rather useless. Different languages are designed for different uses, and while some languages might favour faster code, other might favour ease of development or portability.
Anyway, even when remaining within a same language or language family, the benchmarks are still quite meaningless. For instance when you want to compare the Performance of MSVC++ and GCC. The benchmark has several flaws:
- the code is too trivial. It doesn't show how good the compilers really are at optimizing
- the code is too library dependent. For instance, in the trig benchmark, only the runtime library is really benchmarked and not the code generated by the compiler itself
- for the floating point benchmarks, the options chosen for both compilers do not match. For MSVC++, the options chosen favour speed over accuracy, while the GCC options favour accuracy over speed.
The last point can very easily be illustrated with the trig benchmark.
On my computer (P4, 2.8GHz), I get the following results:
1) Options from the article: 10.9s
2) additional option -ffast-math : 6.9s
(this option is also a significant win for the double benchmark)
3) options above plus linking with CRT_fp8.o : 2.8s
The last option may need some explanation:
Programs compiled by MSVC++ by default set the math coprocessor to 64bit, while GCC programs set it to 80bit. Linking with CRT_fp8.o on Windows platforms makes GCC programs behave like MSVC++ programs and only use 64bit precision. For arithmetic operations, this makes no difference, but the built in transcedental functions become much faster if you reduce the precision of the coprocessor. So all in all, be were able to reduce the speed of the trig benchmark by a factor 3.9 just by changing the compilation options. This is almost exactly the difference seen in the article between the MSVC++ and the GCC results for the trig benchmark.
All in all, for trivial benchmarks like this, if you chose matching compilation options, different compilers give you almost the same results.
The only real weakness that GCC is showing is 64bit integer arithmetic. These are badly implemented in GCC and could be vastly improved.
Marcel
from the article:
i f.html
BEA has an interesting JVM called JRockit which promises performance improvements in certain situations, but unfortunately only works on Windows.
Not true:
http://edocs.bea.com/wljrockit/docs81/cert
Python is also great as a language to port from. The python.org stories from companies and groups using Python often say something like "We started with Python, and were wowed by it's ease etc., and then as our code matured we started converting parts of it to (C/C++/Java/etc)."
Now, you might want to take that as a bad thing, that Python isn't good enough to keep using indefinitely, but that's not really the reason they stop using it. It's more often a question of speed or skills needed for maintainence - not as many people know Python as they do C(++) or Java, and with a clear reference implementation to work off of, rewrites(partial or complete) for other languages are usually fairly straightforward. If only certain bits need optimizing, the C integration features become quite handy. The standard libraries make good use of C to optimize the heck out of the most frequently used features, which would account for the I/O being comparable with all other languages. A similar result, or at least one much closer than with this benchmark's tests, would probably occur if you benchmarked some of its handy built-in data types with their equivalents in other languages.
So there are, fortunately, plenty of ways to get around Python being slow in benchmarking, if you aren't doing something that requires a heavy amount of raw number-crunching like these benchmarks.
For new desktop apps Visual Basic 6 is no longer an option. I've been a full time VB developer for years. With the end of the VB line comes a switch to .NET or something else. VB.NET is not the next version of VB, but a complete rewrite. The best alternative I've found for rapid desktop app development is Python with the wxPython library. It's cross-platform, executes very fast, and requires little code for basic stuff. Microsoft would have us drop everything and move to .NET, but my experience with C# on .NET has been far less than pleasant. I know C++ so the learning curve is minimal. The problem is Microsoft's promises of automatic memory management and rapid development of complete apps is a joke. It's riddled with bugs and memory leaks. It requires far more coding than VB did or Python does. This is an important time for VB developers and they should move along to the freedom and speed of non-Microsoft tools.
Developers: We can use your help.
This whole "language benchmark" is brainless on a bunch of different levels. As others have previously mentioned, the benchmarks don't even vaguely resemble what people actually do with these languages. Plus, a lot of programmers can afford to take the performance hit to get the particular features of the language. And last but not least, the benchmarker hasn't got the vaguest notion what of what he's actually measuring!
why doesn't someone extend this test to my favorite interpreted language (REBOL) and my least favorite and most despised compiled language (see sig)? you know. just for giggles.
grey wolf
LET FORTRAN DIE!
The commentator (not OSNews) ran his benchmarks with the -server flag in Java. That's a terrible flag to use for short-term client applications, such as these benchmarks. Running the test without the -server flag, and the times will probably improve a little (just a guess).
That said, I have found Java on Windows to run noticeably faster that Linux or even Solaris. Not terribly so, but such that you can tell the difference. Wish they'd optimize it for Linux a tad more.
"Even if C did still enjoy its traditional performance advantage, there are very few cases (I'm hard pressed to come up with a single example from my work) where performance should be the sole criterion when picking a programming language."
I'm guess that in programming 3D game engines or any part of a video game for that matter, this would be one of the largest, if not the main, consideration.
SIGFAULT
The test was on the Linux box, why in the world wouldn't you compare Blackdown's Java. It was proven that it is the fastest implementation of Java !For Linux!. It was also proven that Sun's JVM is not very efficient on Linux.
http://www.volano.com/report/index.html
Java just-in-time compilation will fail to perform well in certain cases. In fact, even if it were to completely compile itself into native code right from the outside, it will still fail to perform well in these certain cases.
The cases I speak of are when multiple processes are using the same executable code, which for Java will be the class file, and for compiled C source will be an ELF executable image. When Linux and other Unix systems run these programs, the original executable image is memory mapped by the kernel or the run time loader. At this point the pages are all shared between multiple processes. The run time loader will link in necessary libraries, which will modify a few pages and result in them not being shared due to copy on write. But these are generally the jump tables and few other things that happen to be in the same page area. Most of the executable machine code is not modified.
When a Java class file is loaded, the executable as far as the system sees it to be is the Java run time engine itself. When it compiles the Java byte code at run time, it can produce good performing code, but it can't cause that code to be shared in memory with other processes that were running the same Java class file.
OTOH, Java applications tend to be run as threads, not multiple processes. As long as the compiling of the code is done so that there are no thread collisions in that (you'd know pretty quickly if there were), then you have one copy of the code shared between all threads.
The use of threads and Java is a tradeoff. You get the better program protection of Java (and not programming with pointers, which the average programmer is not capable of dealing with very well), but you also get the risk of threads (it's still possible to program badly for threads even in Java, and get data crossover where it should not be).
But when you see benchmarks that say one is better than the other, you better have a close look and see exactly what kind of thing is being tested, including whether multiple instances in processes, vs. multiple instances in threads, is being done. In some cases (like when it has to be processes, not threads) you might want to consider using a Java to native machine code compiler, or if you can handle it, program in C or C++.
now we need to go OSS in diesel cars
The author's benchmark doesn't account for this switchover:
http://java.sun.com/docs/hotspot/VMOptions.html
java -XX:CompileThreshold=10000 Test
other interesting flags:
-XX:+PrintCompilation
-XX:+PrintInlining
I cannot figure out for the life of me why the heck Cygwin's GCC was used over MinGW.
Every single binary created by Cygwin's GCC must be linked to cygwin.dll, which is an absolutely enormous performance overhead. GCC would have performed much more strongly had MinGW's GCC been used.
// file: mice.h
#include "frickin_lasers.h"
While we are at it, you may wish to check Java application benchmark I wrote some time ago. JatMark
"Man in the Moon and other weird things" - wfmh.org.pl/thorgal/Moon/
To be more accurate, it's proof that Java is not slow when performing trivial tasks.
These benchmarks test trivial operations, most of which can be done in just an instruction or two of machine code inside a loop on the test platform. Any language in a serious performance league should execute these at near identical speeds, modulo up-front time for JIT compilation. It would be laughable if a serious player didn't, and indeed it is laughable that two of the .Net languages were so much slower for I/O, that Java's new floating point performance is so abysmal, and that GCC has a performance so much slower than VC++.Net.
As the report acknowledges, benchmarking usefully is hard. Indeed, many very credible figures in the programming community refuse to compare languages directly in this fashion, precisely because it's so difficult to get any results that are realistically meaningful. To get a fair comparison of the raw performance of (to take everyone's pet holy war) C++ and Java, you'd have to write a much larger-scale test application in the field you were interested in, which would use not only trivial mathematical operations, but also wider optimisations for complex calculations, many function calls, various data structures and algorithms, and so on. I'm aware of no serious benchmarking exercise that has ever done this.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
Comparing Perl to Java is foolish, Perl is more like Awk than a general purpose programming language
.NET finally die. Guess who will attend the funeral? Yep, C and PERL (Father and son) will still be going strong and C (at least) will still be working hard.
.NET ...
If 'C' were a person, he'd be hardworking, fast, organized, efficient, smart, and flexible enough to solve nearly any type of problem. He'd also live a quiet simple live, eat right, excercise, and go to Church on Sunday.
If Perl were a person, he'd be C's fat, lazy, but brilliant son. Able to complete most any task with minimal effort, so he could go back to being fat and lazy. He would also borrow largely from the accomplishments of his father, but not work as hard to achieve the same result. Perl would party on the weekend, cosume large amounts of high fat food, drive a Harley Davidson motorcycle, and only attend Church on Christmas eve, just to make dad happy.
When Java and
Perl like Python is a swiss army chain-saw. You're right in saying Perl shouldn't be compared to Java. PERL is much more high level and can reduce development time from weeks to days. Perl can be used for large projects as long as you establish a dialect among the coders. Using Java for a large project can turn into a disaster quickly, especially if all the developers start writing all their own classes and doing platform specific things like OS specific system calls. I often use Perl to do mockups for Java projects since I can go from concept to working prototype in mere hours.
Do not underestimate the power of Perl, especially for web development and back-end development work. Perl is my absolute favorite programming language and I write in many including C/C++, VB, Java, PHP, but never
I think the author underestimates the impact of having Athlon specific optimizatins turned on for his c compiler(where applicable) while the Java HotSpot JIT compiler likely only optimizes very well for Pentium 4.
.NET languages with Java and used gcc as reference as well. Interesting for me is that Java can compete with .NET.
Conclusion would be: the JIT compiled Java on an Athlon is poorly or not at all optimzed.
Sidenote: the original author of the original benchmark wanted to compare
Further: obviously trig functions (which could be compiled to a single math processor opcode) are not optimzed at all in Java. From the language level calling a trig fuction is a call to a static method in the class Math. If that is "mapped" one to one to machine code it results in a JSR to the C function which contains only a few opcodes, but what a c compiler will compile to one opcode is in trivial 'mapped' Java about 10 opcodes.
angel'o'sphere
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
It is indeed, but the usual question is whether it's better to die on your feet or to live on your knees. And that's a much tougher question, if your loved ones are the ones with guns to their heads. I am pretty liberal, but there are limits in the real world.
And I suspect you have a deep misunderstanding. :-)
Did I mentioned that I am one of those guys who does write performance-critical maths code, and it does ship on more than a dozen platforms, and it pretty much doesn't use any of the specifics you mentioned?
I'm sorry, but I've played this game for a while. I have looked at the assembly output of numerous C++ compilers on different architectures to identify performance problems or compiler errors in low-level libraries, resolved tricky numerical analysis issues when computing the roots of quartics to double-precision accuracy, investigated writing what are effectively mini-interpreters for some repetitive coding tasks we do, including JIT and dynamic optimisation issues, made decisions about squeezing a precious 4 extra bytes into a data structure (precious when your structures might be used millions of times) or whether such-and-such a function can be non-virtual to save calling time, all against a background of automated numerical tests that squeek when your results on two platforms differ in the 14th sig fig or so.
The conclusions are crystal clear, and a language like Java will never even come close to producing the speed we need. Other radically different languages might; I have an interest in high-level, functional languages, and the interesting possibilities for compiler-provided optimisations there could make for some worthwhile experimentation at some stage. But not Java. It has too many "hard" down-sides, and precious little to make up for them in this context.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
Actually, C and C++ do suck as languages for writing operating systems. It's just that the more practical potential alternatives still suck even more.
It's kinda sad really. We've had near enough three decades of advances in programming language technology since C was invented, and yet we're still writing our most critical software in weak, cumbersome, unsafe languages that are just a step or two removed from assembly. Worse, much (thankfully not all) of the serious R&D in industry still seems to be based around these near-prehistoric monstrosities and the pseudoclones built on them. No wonder so few people are really innovating in the software world.
None of which negates your point that judging a language by a single project written in it is a daft thing to do, of course. I just happened to disagree with your counterexample. :-)
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
We have lots of "coders" but in truth there is an agregious lack of "designers".
In Aphorisim: Failing to Plan is Planning to Fail.
The real cost of software is generally in the noise floor compared to the *realized* cost of the software caused by people tyring to bean-count their way to a solution using over-simple metrics.
In the last decade that overly simple metric was money, before that it was kLOCS (thousands of lines of code).
"Good code" is just that, "good". The "right code" is outstandingly good for what you are doing and has all the right intangibles for your need. But like the difference between "good" and "bad" is like art, "good code" really doesn't *have* any single empirical metric. Heck, even "works" vs "doesn't work" well, doesn't work as a metric since you can only honestly go as far as "seems to work".
Efficacy, Correctness, Near Term Cost, "TCO", Competeion vs Penalty Cost (over the domains of time and money); all of these are important factors.
Whoever you be, save your self grief and cash: Find a good designer, who you can *talk* *to* *EFFECTIVELY* or just live in your mess.
Just now my company is being held hostage by one of our founders. She has the code effectively in escrow and won't talk to anybody about anything. She insists you "just tell her what to do and she'll do it" and GOD SAVE YOU if you have to change your mind about the slightest detail or explain it to a customer.
Really, hell hath no furry like a (wo)man engaged in self-protectionist isolation over a bad design and peacmeal implementation.
It was "cheap and fast" and that will be costing us for an *EXCEEDINGLY* long time.
You have been warned... 8-)
Innocent people shouldn't be forced to pay for inferior software development.
--"Code Complete" Microsoft Press
Problem is, when given one of those constraints, i.e. "if you just ", well, that never happens and you get platform specific code. Certainly C and C++ were more portable than their predecessors, but Java is an even further improvement. Thus some rather cool stuff, like I can build on NT and deploy to Solaris, OS/390 (and now zOS, or AIX. Same exact jar. These are not trivial apps either, thousands of custom classes with a 100 or so supporting jars, all running on an application server (WebSphere).
Pretty cool, and it works so well, you sometimes forget what a tremendous feat it is.
If you take average programmers, Java creams C++. If you take good programmers, C++ creams Java. If you take top programmers (tine fraction of a percent), Lisp/Scheme creams C++ and Java.
The advantage of Java is that it forces a specific programming style (object oriented), forces people not to use potentially dangerous constructs, etc. This is great for mediocre programmers who cannot discipline themselves, or for maintainability of a project over multiple sets of programmers, since it guarantees a certain standard for the code.
However, Java falls apart for complex projects, where you very rarely, but still on occasion, need something like templates or multiple inheritence, and almost every large Java project I've seen has some nasty cludges to work around lack of expressibility in Java.
This benchmark is completely pointless for a few reasons. No one is writing programs like this, and will never write programs like this. Programs are always more complicated. The operations dont consider the precision, which affects many of the languages trig functions. Which I feel should be thrown out. Most programs are not bottelneck by performance either. Almost any programmer can write a better program then a for loop, hopefully this benchmarker can live up to this. BTW... Wow Java is fast, but could someone write some real benchmarks to prove it.
mnewberg.com
If safety is critical, don't use any C variant. Really, 99% of security problems can be attributed to stuff the java virtual machine prevents from ever happening (e.g. buffer overflows). This type of error is routinely discovered even in C programs that have been subjected to multiple code reviews (e.g. some of the server software that is shipped with BSD). In a C/C++ program, discovering an error at run-time is usually fatal whereas in Java you simply handle the exception (and of course you build in some stuff to recover nicely from such unfortunate events).
As others have pointed out, templates are planned for the next Java release (in addition to a number of other nice language features). Compared to C++, the Java version of templates is particularly strong on typesafety (which in Java was a hard requirement whereas in C++ it is just convenient to have). If you go to javalobby.org, you can sign up for an alpha release of the jdk1.5. Alternatively, you can download some extensions that will work with existing versions of java.
Anyway, there are real-time variants of Java if you need hard real time and for soft real time stuff you can configure the garbage collector to be less of a pain in the ass (with a few percents performance penalty). Garbage collection used to be a problem when SUN still used a very naive garbage collecting algorithm, that was fixed long ago.
Jilles
use lightweight c++ . Otherwise Python is the choice.
my 0.2 cents
But if .NET really catches on (yeah, yeah Longhorn, but to echo a familiar refrain, where are the shrink-wrap apps? When is MS going to eat their own dogfood on this one?), someone should run a copy of J++ in a museum exhibit somewhere (it is perhaps like the Lisa to the Mac that finally caught on).
Slashdot isn't funny anymore...
Actually, numerical analysis in Python can be much nicer than in other languages.
Check out http://numpy.sourceforge.net
Using some basic features of the language, it does some things with numbers that just normally aren't very easy (let alone generate code as clean as what you see in numpy).
Python is one of those languages where style is important. There are a lot of things in Python that need to be written in certain ways that utilize the features of the language.
To give an example, let's say you need to build an arbitrarily large list of floating point numbers, then call some operation on them. Python can do a case like this extremely fast when written right. Most of the slowdowns in Python occur during lookup of names or loops. Both can be eliminated easily. See this code snippet that, for giggles, takes the sine of a list of numbers (although imagine a really long list).
l = [1,2,3,4,5]
from math import sin
output = map(sin,l)
These three lines will take a list of arbitrary size and munge it nicely. Furthermore, in Python 2.3 the map function will work on an iterator. Iterators allow you to repeatedly suck in data in a way that doesn't require having all of it in RAM at once. One nice application of this is to read from, say, an INET socket, as data comes in instead of buffering it all in RAM. So this could be extended as follows:
from urllib import urlopen
from math import sin
u = urlopen("http://www.somewhere.com/data")
def process(data):
return sin(float(data))
print map(process,u)
As an example to test the above, I decided (for no apparent reason) to load slashdot's main page in and measure the length of each line. This was no more than.
from urllib import urlopen
u = urlopen("http://slashdot.org")
print map(len,u)
Need the average line length of Slashdot?
from urllib import urlopen
u = urlopen("http://slashdot.org")
def avg(avgdata,newdata):
(sum,count) = avgdata
return (sum+len(newdata),count+1)
(sum,count) = reduce(avg,u,(0,0))
print float(sum) / count
Doing the above in other languages is a lot of work and usually much less expressive. In addition, the basic language features (iterator protocol, map, reduce, list types) make this remarkably efficient. Map and reduce are both tight C loops. They don't spend time doing symbol lookup except during initial invocation. Iterators allow all of this to occur without having to buffer everything in memory. It's actually iterators the whole way. Urllib uses a socket as an iterator, urllib's urlopener is an iterator, reduce then iterates that. Disturbingly, this can also make things easier to maintain. Less lines of code is really the big boon of python (although strong introspection comes second).
Also, there's more to it than just this. There are benefits you just don't get anywhere else. Try subclassing an integer anywhere else. Not subclassing a wrapper class for in integer--SUBCLASSING THE INTEGER! As of 2.3 all of python's primitive types are first-class objects.
Combine all of that with judicious use of classes and it can manipulate data in general (and numbers in particular) like nobody's business. If you want to see how powerful the interface can be, just load up Python 2.3 and type:
dir(5)
When you understand what you're looking at, you'll be amazed.
Thanks.. i never knew
Java 2 v1.5
It's not that hard, especially for a programming language where the users are assumed to be a little bit technical...
"There is more worth loving than we have strength to love." - Brian Jay Stanley
... because of the code execution model: local non-scalar variables have to be allocated dynamically (as with malloc, which is very expensive) this being the semantics of the language. Code which rarely creates new non-scalars (there is rarely such code when writing Java programs in the normal way), if compiled natively, can in principle perform as C code. See for example: dan.corlan.net/bench.html
There's no reason that Sun's JIT couldn't recognize that it's running on a P4 and generate SSE2 instructions to gain performance. If it doesn't, then it doesn't and that's a reflection of the quality of the JIT. If it does and it's still slower, then it's just slower.
of its JNI implementation.
When you call a trig function, it goes through the java.lang.Math class. However, Math delegates to StrictMath that is a native implementation. Now, I do not believe that Sun has implemented the trig libraries because if you dumpbin on java.dll where StrictMath resides, you will notice most the offsets between those functions is small (about 20 bytes).
So when you call any trig function, you have 2 levels of object packaging/unpackaging - likely before a call that sets fp numeric state before the operation.
http://math.nist.gov/javanumerics is a fun place to visit for more information on various math libraries and issues people have with java numerics.
/\/\icro/\/\uncher
//TODO: Think of witty sig statement
Eh? Static analysis can be done in Python. See pychecker.
On this older reviewy namo/dyn amo-1.html
http://www.arstechnica.com/reviews/1q00/d
HP labs showed that interpreting code which could even have been executed natively can actually improve performance. The idea is that many optimizations can not be done at compile time because of non-static variables; on the other hand the run-time engine can "discover" that said boolean is always false an drop a jump that a static compiler could not have ignored.
Note in particular that the Caml language is often found to be as fast, if not faster than C (even though it is intrinsiquely interpreted) for this reason.
To be completely strict, languages don't have any performance. The only way you can judge how 'fast' a language is, is to test how long it takes to write a program in that language.
They were comparing compilers and runtimes, I thought that was a given.
Karma: It's all a bunch of tree-huggin' hippy crap!
Actually strictly speaking, it's wrong to imply STL's typed containers are generics. They are templates. The difference is supposedly that when templates are expanded, List<String> and List<Goat> become different classes, by generating a new class which has the template expanded.
Java's generics (coming in Java 1.5), are 'true' generics in that there is always only one instance of the class, but the class has parameters.
For the most part generics are in fact a compile time feature anyway. This is why compilers for Java have existed which did generics, as far back as Java 1.2, and possibly earlier. The difference in 1.5 is it will be possible to dynamically see at runtime which parameters are being used for a generic type, via reflection.
Karma: It's all a bunch of tree-huggin' hippy crap!
It's actually a very simple problem made more complex for no reason. Look, languages can be broken down into three basic families.
Application development - C, C++, VB, Delphi, Java
All purpose scripting - Bash, Python, Perl, Awk
Web development - PHP, Javascript, ASP
Each family has it's function. No matter which language you choose from the above careful design and good use of algorithms go a long way as far as stability and efficiency are concerned. I've seen C++ programs run slower then Awk scripts. Why? Too much code. The C++ programmer probably spent lots of time programming things like I/O and buffer overrruns to the point where using Awk would have been a better choice.
Bottom line, experience is unfortunately the only way to gain an understanding of algorithms and optimization. A compiler can only aid the knowledgable and I consider myself just a beginner. However, as a programmer, knowing that Visual C++ or gcc produces the fastest code is a moot since there are many more factors that go into deciding which product to use. How about ease of use of the development environment and technical support? Case in point. I'm a big fan of Linux, but ever wonder why those apps never really look or act polished? That's the direct result of poor development environments for that system since programmers end up spending more time on the compiler and customizing the development environment than they do on the actual problem they wish to solve.
The man has an associates degree in computer science, and a Ph.D. in psychology. .
Not true at all and weird that you'd say so. If you actually read the brief bio blurb, you'd see he has a Bachelor's degree in computer science (Harvard writes their bachelor's degrees "backwards" as in A.B.) and a PhD in philosophy (not psychology), where he concentrated in artificial intelligence.