Slashdot Mirror


Performance Benchmarks of Nine Languages

ikewillis writes "OSnews compares the relative performance of nine languages and variants on Windows: Java 1.3.1, Java 1.4.2, C compiled with gcc 3.3.1, Python 2.3.2, Python compiled with Psyco 1.1.1, Visual Basic, Visual C#, Visual C++, and Visual J#. His conclusion was that Visual C++ was the winner, but in most of the benchmarks Java 1.4 performed on par with native code, even surpassing gcc 3.3.1's performance. I conducted my own tests pitting Java 1.4 against gcc 3.3 and icc 8.0 using his benchmark code, and found Java to perform significantly worse than C on Linux/Athlon."

63 of 954 comments (clear)

  1. Java Performing worse then C by ViolentGreen · · Score: 4, Insightful

    I conducted my own tests pitting Java 1.4 against gcc 3.3 and icc 8.0 using his benchmark code, and found Java to perform significantly worse than C on Linux/Athlon.

    Why is this a suprise? C has been most commonly used for so long because of it's speed and efficiency. I think anyone who has done much work with either developing or running large scale java programs knows that speed can definitely be an issue.

    --
    Not everything is analogous to cars. Car analogies rarely work.
    1. Re:Java Performing worse then C by Kingpin · · Score: 4, Insightful


      All that matters to anti-Java zealots is speed. The list of benefits coming from using Java is too long to take the speed-only view seriously.

      --
      Unable to read configuration file '/bigassraid/htdig//conf/14229.conf'
      Geocrawler error message.
    2. Re:Java Performing worse then C by finkployd · · Score: 3, Insightful

      Not always though, I think the thing people neglect to consider is that there are times when performance and scale are important enough that the benefits of Java do NOT outweigh C, and vice versa.

      I feel sad for someone who only has enough room in their world for one computer language.

      Finkployd

  2. Under Windows... by Ianoo · · Score: 3, Insightful

    I see once again that Eugenia (a supposed pro-Linux pro-BeOS person who doesn't use Windows) has done all her benchmarks [i]under[/i] Windows. I have a feeling that Python would perform a lot better if it was running in a proper POSIX environment (linked against Linux's libraries instead of the Cygwin libs). Probably the C code compiled with GCC would perform a fair bit better too.

    1. Re:Under Windows... by MBCook · · Score: 4, Insightful

      Because as we all know VC++ and the other Microsoft languages are so widly available for Linux/BeOS. I'm sorry but your comment is pure troll. It would be interesting to have things like GCC under Linux on the same computer there too, but you can't compare Microsoft's .NET to anything under Linux, because .NET doesn't run under Linux (I know about Mono, but that isn't MS's runtime).

      --
      Comment forecast: Bits of genius surrounded by a sea of mediocrity.
    2. Re:Under Windows... by Waffle+Iron · · Score: 3, Insightful
      But these where low-level numeric benchmarks. Except for the I/O one, they wouldn't have changed due to linking against different libraries.

      The review article is /.ed now, but from the test names on the summary table it looks like the tests are indeed mostly numeric. Unfortunately, only a small minority of people make their living writing number crunching code.

      For the vast majority of business and web-based apps, the bulk of operations involves string manipulation. If an app is compute intensive and not I/O or GUI bound, then the bottleneck is usually creating, modifying and destroying strings. Benchmarks on string handling would be more useful to most developers.

      However, doing string manipulation benchmarks isn't so simple. There are at least four approaches to strings, and some languages let you pick any of these:

      -- dangerous and very fast: using static buffers and in-place modifications like old-school C
      -- somewhat safer and may be fast: using semiautomatic memory management with mutable strings, like C++/STL or C with glib's g_string
      -- safer still: using totally automatic memory management with mutable strings, like Ruby or (IIRC) Perl
      -- safest: using totally automatic memory management with immutable strings, like Java or Python

      Of course, for each problem the algorithms would need to be structured differently to get the maximum possible speed in each of the above four methodologies.

      Basically, for string-intensive code, claiming that Java is just as fast as C will always be a false statement if you compare C code written in the first dangerous style vs. Java, which is always written in the fourth and safest style. No matter what technical tricks the VM writers come up with, there is just no way that they'll be able to match C's ability essentially zero-overhead in-place buffer operations over and over in the same spot that stays loaded in the L1 cache. (Actually, you probably could write Java code that operates on raw character arrays, and it might approach the speed of C. But that would probably look even uglier than the C code.)

      In the few cases that I've ported a string-intensive high-level-language algorithm to raw low-level C code with few or no mallocs (not a trivial task), I've gotten at least a 10X speedup on the CPU-bound tasks, and at least 10X less memory usage. (Note that I did those tests largely out of curiosity. For most applications, even a 10X speedup is rarely worth the increased development time, bug vulnerabilities or maintenence issues. My opinion is that if you have to write code like this, you should confine it to a C extension library to a high-level language like Python.)

      I've found that STL can be faster or slower than Java, depending on how smart you are. It's very easy to inadvertently get C++ to thrash around with needless automatic data copying.

      Languages like Perl and Python can be very competetive on string operations if you know how to use their libraries. By using the most powerful operations that work on the largest chunks of data at one time (Python's re.findall(), for example), you take advantage of the fact that the library call is mostly written in C. Bit-banging in a dynamic interpreted language is usually dog slow, as the Python numbers seem to show on the summary chart.

      To sum it up, most people write apps whose performance can't be predicted by a few simple language benchmarks, because the way the app is written can affect the performance more than the language it's written in.

  3. Re:Trig functions... by bigjocker · · Score: 3, Insightful

    What is interesting in these functions is that, as pointed in the article, there seems to be something wrong with Sun's implementation for Java. Removing this test JDK 1.4.2 executes almost on par with Visual C++ (the winner).

    This is (once again) proof that Java is not slow, in fact it's really fast. It's slow starting, and yes, consumes more memory than native code, but the gained security, stability and ease of programming (reduced development times) are worth the memory use increase.

    Also, the memory use should be addressed by project Barcelona (I believe these will be available in the forthcoming J2SDK 1.5, along with generics, enums, etc).

    --
    Life isn't like a box of chocolates. It's more like a jar of jalapenos. What you do today, might burn your ass tomorrow.
  4. Re:Trig functions... by Dr.+Evil · · Score: 4, Insightful

    Don't forget that it is also percieved as slow since just about any application anyone has seen for a desktop environment written in Java has a sluggish GUI.

    Yeah, I know Java's strengths aren't in the Desktop arena, they're in development and the back-end.

  5. this is just so bogus by rpeppe · · Score: 4, Insightful

    Benchmark code like this does not represent how these languages are used in practice. Idiomatic Java code tends to be full of dynamic classes and indirection galore. Just testing "arithmetic and trigonometric functions [...] and [...] simple file I/O" is not going to tell you anything about how fast these languages are in the real world.

  6. Where is Fortran? by Aardpig · · Score: 2, Insightful

    It's a pity that the present-day language of choice for high-performance computing, Fortran 90/95/HPF, was not covered in this study. There has been anecdotal evidence that C++ has approached Fortran, performance-wise, in recent years, but I've yet to see a proper comparison of the two languages.

    --
    Tubal-Cain smokes the white owl.
  7. About the Python performance by ultrabot · · Score: 5, Insightful

    Note that Python is pretty easy to extend in C/C++, so that speed critical parts can be rewritten in C if the performance becomes an issue. Writing the whole program in C or C++ is a premature optimization.

    --
    Save your wrists today - switch to Dvorak
  8. What about development ease... by Slick_Snake · · Score: 2, Insightful
    There is more to programming languages than pure speed. I have written programs in a number of languages and find it irritating when people start dissing languages because they are slower at run time. I have on numbers occations written applications for people in "slow" languages because it was easy to write, easy to maintain, and was not a time critical application. In most cases the speed limiting factor is the human sitting in front of the screen.

    IMO a program should use whatever tools are available and appropreate for the job, and not just worry about what is faster.

  9. Re:What about coder's performance? by nuggz · · Score: 2, Insightful

    I agree, time to market is key.

    I like python, it is easy to write, and keep it somewhat clean.

  10. Read the OSNews thread by be-fan · · Score: 5, Insightful

    There were a number of problems with this benchmark, which are addressed in the OSNews thread about the article.

    Namely:

    - They only test a highly specific case of small numeric loops that is pretty much the best-case scenario for a JIT compiler.

    - They don't test anything higher level, like method calls, object allocation, etc.

    Concluding "oh, Java is as fast as C++" from these benchmarks would be unwise. You could conclude that Java is as fast as C++ for short numeric loops, of course, but that would be a different bag of cats entirely.

    --
    A deep unwavering belief is a sure sign you're missing something...
  11. Re:Wow by thoolihan · · Score: 3, Insightful

    Keep in mind too that these benchmarks were all run on windows. I think gcc plays a lot nicer with glibc compared to the windows native libraries. Also, as pointed out, it's about being portable, not the most optimized compiler.

    -t

    --
    http://unmoldable.com W:"No one of consequence" I:"I must know" W:"Get used to disappointment"
  12. Not testing languages by xtheunknown · · Score: 3, Insightful

    You are not testing the languages, you are testing the compilers. If you test a language with a crummy compiler (gcc sucks compared to commercial optimized C++ compilers) you will think the language is slow, when in fact, the compiler just sucks. The only valid comparisons that can be made are same language, different compilers.

    --

    They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.
  13. wrong questions by ajagci · · Score: 4, Insightful
    The Java JIT has been comparable to C in performance for many years on certain microbenchmarks. But Java remains a "slow language". Why?
    • The design of the Java language and the Java libraries means that enormous numbers of calls are made to the memory allocator in idiomatic Java.
    • The Java language has several serious limitations, such as the lack of true multidimensional arrays and the lack of value classes.

    So, yes, you can construct programs, even some useful compute intensive programs, that perform as well or better on Java than they do in C. But that still doesn't make Java suitable for high-performance computing or building efficient software.

    Benchmarks like the one published by OSnews don't test for these limitations. Microbenchmarks like those are still useful: if a language doesn't do well on them, that tells you that it is unsuitable for certain work; for example, based on those microbenchmarks alone, Python is unlikely to be a good language for Fortran-style numerical computing. But those kinds of microbenchmarks are so limited that they give you no guarantees that an implementation is going to be suitable for any real-world programming even if the implementation performs well on all the microbenchmarks.

    I suggest you go through the following exercise: write a complex number class, then write an FFT using that complex number class, "void fft(Complex array[])", and then benchmark the resulting code. C, C++, and C# all will perform reasonably well. In Java, on the other hand, you will have to perform memory allocations for every complex number you generate during the computation.
  14. Re:Trig functions... by Doomdark · · Score: 5, Insightful
    Don't forget that it is also percieved as slow since just about any application anyone has seen for a desktop environment written in Java has a sluggish GUI.

    It's in many ways unfortunate that with JDK 1.2 (Swing) and onwards, Sun pretty much dumped fast native support for GUI rendering. It has its benefits -- full control, easier portability -- but the fact is that simple GUI apps felt faster with 1.1 than they have done ever since (or even more). This is, alas, especially noticeable on X-windows, perhaps since often the whole window is rendered as one big component as opposed to normal x app components (in latter case, x-windows can optimize clipping better).

    Years ago (in late 90s, 97 or 98), I wrote a full VT-52/100/102/220 terminal emulator with telnet handling (plus for fun plugged in a 3rd party then-open SSH implementation). After optimizing display buffer handling, it was pretty much on par with regular xterm, on P100 (Red hat whatever, 5.2?), as in felt about as fast, and had as extensive vt-emulation (checked with vttest). Back then I wrote the thing mostly to show it can be done, as all telnet clients written in Java back then were horribly naive, doing full screen redraw and other flicker-inducing stupidities... and contributed to the perception that Java is and will be slow. I thought it had more to do with programmers not optimizing things that need to be optimized.

    It's been a while since then; last I tried it on JDK 1.4.2... and it still doesn't FEEL as fast, even though technically speaking all java code parts ARE much faster (1.1 didn't have any JIT compiler; HotSpot, as tests show, is rather impressive in optimizing). It's getting closer, but then again, mu machine has almost an order of magnitude more computing power now, as probably does gfx card.

    To top off problems, in general Linux implementation has been left with much less attention than windows version (or Solaris, but Solaris is at least done by same company). :-/

    --
    I like paying taxes. With them I buy civilization -- Oliver Wendell Holmes
  15. Less simple benchmarks by DuSTman31 · · Score: 4, Insightful

    The optimisers in sun's Java VM work on run-time profiling - they identify the most run sections of code and use the more elaborate optimisation steps on these segments alone.

    Benchmarks that consist of one small loop will do very well under this scheme, as the critical loop will get all of the optimisation effort, but I suspect that in programs where the CPU time is more distributed over many code sections, this scheme will perform less well.

    C doesn't have the benefit of this run-time profiling to aid in optimising critical sections, but it can more afford to apply its optimisations across the entire codebase.

    I'd be interested to see results of a benchmark of code where CPU time is more distributed..

  16. Poor benchmarks by Anonymous Coward · · Score: 2, Insightful

    Come on guys, carefully choosing that simple benchmarks I can easily prove any language's faster than GCC.

  17. It all depends on what you are doing by finkployd · · Score: 2, Insightful

    I like java for some things, and the performance has even improved a bit lately. However if I am doing ANYTHING that has to scale and perform well under heavy load that uses cryptographic functions (especially public key encipherment), there is no way I can even seriously consider Java.

    Someone (meaning anyone other than me) should do a benchmark of THAT, I'm sure it would be quite telling.

    Finkployd

  18. performance depends on the application by lambent · · Score: 2, Insightful

    It's well known that benchmarks aren't the be all and end all. They're often just statistics geared towards a ultra-specific application (remember all those /. stories about benchmark cheating vendors?)

    I've seen examples of gcc in a cygwin shell kicking visual-c++ ass at load up times of huge image data on a wintel box. I've also seen java (jdk 1.3) annihilate native c code on console apps calculating complex mathematical formulas on a linux box. This goes for both AMD and Intel chips.

    Moral of the story? These languages are all suited to specific uses. Analyze your tasks, your platforms, and your compilers. Learn how to use optimizations properly. Evaluate your need for portability. Do a few tests for performance in different languages and compilers to see which one actually is fastest for your current application.

    There is no single "fastest" language.

  19. Re:Speed? No. Stability. Yes by Isochrome · · Score: 4, Insightful

    OK, Speed does matter a lot.

    But what about type safety? Java has no generic typed containers, like the STL. This means you tend to find errors at runtime instead of at compile time.

    I need to know that my code is as safe as possible. I don't want a user to find a bug because my hand tests didn't get 100% code coverage every time.

    And how about predictable performance. I would much rather know that this function will tak 200ms all of the time instead of 100ms most of the time a 10 s due to garbage collection occasionally.

  20. Re:java vs C by jrexilius · · Score: 2, Insightful

    Yeah, with the increase in hardware price/performance the performance consideration is becoming less and less of a consideration in _most_ applications. There are still environments where efficiency is of paramount concern (the combination of great speed and low resource drag). Examples I work with are real-time financial trading applications, network back-bone servers (routers firewalls, intelligent switches, etc.), mobile and embedded devices, server daemons, and network applications (packet sniffers, etc.).

    For general business processing applications and most web applications, efficiency is less of a concern and cost/time-to-market/maintainability/security are bigger.

    I like these benchmarks but would like to see ones that also benchmark the other characteristics of languages (such as lines of code to do a common task, number of tests that need to be performed to validate common functions, memory space, etc. etc.)

  21. Language performance arguments miss the point by tizzyD · · Score: 5, Insightful
    Consider what was done years ago with assembly. The performance was incredible, and the amount of superfluous garbage in the code was minimal. Hey, if you wrote the assembly, why would you spend time putting it in?

    Then, with more and more languages, especially ones with VMs, you get further and further away from the hardware. The end result: you lose performance. It does more and more for you, but at the expense of real optimizations, the kind that only you can do.

    Now the zealots will come out and say, "Language X is better than language Y, see!" To me this argument is boring. I tend to use the appropriate tool for the job. So:

    • Python for scripts, prototypes, proofs of concept, or components where performance generally is not an issue.
    • For desktop apps, Visual Basic (yep, most IT apps are in VB). There is no justifiable reason for an IT department group to write a sales force reporting system in C++! If you want C++, go get a job at a software company. Stop wasting money and time making yourself feel like a hotshot. [I'd consider Kylix here if it was based on Basic. Why? Because honestly, Pascal is just about dead, and Basic is the king of the simple app. Let's just live with it and move on. I do want a cross-platform VB . . . ]
    • For web apps, well, I stick around PHP/ASP.NET. Why? Portability! And moreover, the sticking point in a web-based app is not the UI layer; it's usually the underlying data extraction and formatting. Don't waste your time with lower level languages there. IMHO it's just not worth it. JSP and Java stuff, yuck! Too much time for too little bang.
    • Java/C# (also consider mono/LISP for most production apps. Why? Portability! I want no vendor holding me by the balls. I want platform independence on the back end, and these are the few ways to achieve it. I'd include Haskell/OCAML here when appropriate. Perl? I'm loathe to use Perl as production, considering most Perl code cannot be understood 2 weeks after it's written. I'd rather take the hit in performance and be able to pass the code to someone else later.
    • C++/C for components--just components--where performance is at an absolute premium or there exists some critical library that only has this kind of interface. But this step has to be justified by the team, with considerable explanation why a different architecture could not suffice. Otherwise, the team could waste time checking for dangling pointers when instead it could be doing other things, like finishing up other projects.
    • Assembly? Only when there is not a C complier around. Embedded stuff. Nowadays, you just do not have the time to play.

    Yes, my teams use many languages, but they also put their effort to where they get the biggest bang for the buck. And in any business approach, that's the key goal. You don't see carpenters use saws to hammer in nails or drive screws. Wise up!

    --
    ...tizzyd
    1. Re:Language performance arguments miss the point by BrownTiger · · Score: 2, Insightful

      Use whatever forks for you. There aren't any golden rules. If someone have large C++ libraries he/she can easily beat java/vb in development/deployment. We use VC++, and outperform VB coders by 20 - 30%. ( Plus code a LOT more stable ) Brian

    2. Re:Language performance arguments miss the point by Fr33z0r · · Score: 2, Insightful
      Then, with more and more languages, especially ones with VMs, you get further and further away from the hardware. The end result: you lose performance. It does more and more for you, but at the expense of real optimizations, the kind that only you can do.

      I don't really agree with this, look at it from a high-level API stance for starters, I'd much rather write some DirectX or OpenGL than write the assembly code necessary to cover my bases with all the 3d hardware out there - with no guarantee my 3d code would work on future hardware. The good old days of calling a BIOS interrupt to put your display into 13h and writing direct to the video memory at (320*y)+x are dead and buried, unfortunately (I'll admit those were fun times :D)

      The above is a somewhat extreme example of how low-level code can be not only inefficient (unless you're *seriously* hard-core) but utterly pointless due to the inordinate amount of time it would take to write said code. It's an extreme example, but it translates almost directly to today's processors - being the complicated beasts they are. You look at things a good optimising compiler will just do for you completely transparently, like branch prediction, mmx, 3dnow, and a host of others (recently, and notably, hyperthreading for instance)...

      If you think you can do a better job of writing low-level code than these compilers can do of optimising your high-level, you're either still living in the early 90s, or you're one hell of a programmer.
  22. Re:They should benchmark development time by ultrabot · · Score: 4, Insightful

    Productive for you now ... but what about 6 months down the road? What if you want to realize your product to the world, how hard is it to extend it?

    The advantages over Java are even increased 6 months down the road. Python code is much more readable and maintainable, hence easier to extend. Dynamically typed object model scales incredibly well.

    I used to think the same about Perl vs Java, until I started looking at frameworks like Cocoon and they're all written in Java.

    Comparing Perl to Java is foolish, Perl is more like Awk than a general purpose programming language, and not meant for large projects at all.

    --
    Save your wrists today - switch to Dvorak
  23. Re:They should benchmark development time by slezakdj · · Score: 2, Insightful
    Point well made, though I could come up with some examples where coding in C++ would be faster than in Java (same for Java over C++). In addition, you should decide on a language given the type of application. All languages have advantages and disadvantages, including execution speed. Some languages are great with debugging, string manip, arithmetic, readability, and maintenance, as we all know. Don't just code in the language you know best. Try to code trig functions in Prolog (please feel free to show an example).

    If the big concern is speed, why not go to the basics and code in assembly (or machine code if you're crazy). Implement some algorithms that will give you a desired precision and that will use the memory that is within the resources. Heck, just use up all your memory resources and create a huge lookup table...

    -I do not move. The world moves around me.-

  24. Re:Trig functions... by bigjocker · · Score: 4, Insightful
    This is a perfect example of why I don't like java. If I use java then I have to stick to the decisions made by someone else even if they are completely wrong for my situation. But you are free to use yor C/C++ optimized functions in Java. Just make a wrapper class and access them natively, just like Java does access a lot of Math functions:
    public static native double acos(double a);
    public static native double asin(double a);
    Those are from the
    StrictMath
    class, used by the
    Math
    class. You did know that you have access to Java libraries source code, didn't you? For real math/science stuff java is horrible For real math stick with Fortran, SciLab or Matlab. For Real Time applications use C. No language will suit all needs. Personally I develop enterprise applications using Java and Games using C. Also, what I find humerous is the whole NIO (new IO) stuff. basically Java started out using threads to deal with multiple IOs but due to scaling issues they developed 'new' IO which is basically the equivilent of select! Yeah thats real 'new'!!! The tradittional IO subsystem in Java was a traditional one: sockets, streams, buffers, etc. It Is very scalable, just look at the Tomcat, Jetty and JBoss project before NIO appeared. NIO is just an optimization for very specific tasks, for some stuff you still need a separate Thread for each connection.
    --
    Life isn't like a box of chocolates. It's more like a jar of jalapenos. What you do today, might burn your ass tomorrow.
  25. Totaly missing parts by musicmaker · · Score: 2, Insightful

    I don't know about anyone else, but most programs in the real application world do alot of string manipulation, and I have seen some pretty shocking results of string manipulation benchmarks showing Java the worst with the C++ class second, and Python actualy leading the pack. It would be usefull to also see the overhead calcs for object management too. Java is so memory heavy we have problems with machines that have 4 Gig of RAM configured.

    It's pretty stupid to run benchmarks for a language in a non native environment for the python marks.

    Yet again OS News publishes a completely meaningless story.

    --
    Everyone is living in a personal delusion, just some are more delusional than others.
  26. Windows a good choice for this test by nberardi · · Score: 3, Insightful

    Windows was a good choice for this test, because many of the development languages that were used in this test aren't really mature enough in *nix. (i.e. .Net languages and arguably Java) A better test would be doing both tests on both OS's, because GCC is really more optimized twords Linux, while VC++ is more optimized twords Windows. I would have rather seen VC++ vs. Borderland C++, because that is a more real world business example.

  27. These kind of benchmarks are so 1970s by Junks+Jerzey · · Score: 3, Insightful

    It is amusing that the obsession with raw speed never goes away, even though computers have gotten thousands of times faster since the the days of the original wisdom about how one shouldn't be obsessed with speed. Programmers put down Visual Basic as slow when it was an interpreted language running on a 66MHz 486. It was still put down as slow when it shared the same machine code generating back-end as Visual C++ running on a 3GHz Pentium 4. And still some people--usually people with little commercial experience--continue to insist that speed is everything.

    Here's a bombshell: if you have a nice language, and that language doesn't have any hugely glaring drawbacks (such as simple benchmarks filling up hundreds of megabytes of memory), then don't worry about speed. From past experience, I've found it's usually easy to start with what someone considers to be a fast C or C++ program. Then I write a naive version in Python or another language I like. And guess what? My version will be 100x slower. Sometimes this is irrelevant. 100x slower than a couple of microseconds doesn't matter. Other times it does matter. But it usually isn't important to be anywhere near as fast as C, just to speed up the simpler, cleaner Python version by 2-20x. This can usually be done by fiddling around a bit, using a little finesse, trying different approaches. It's all very easy to do, and one of the great secrets is that high-level optimization is a lot of fun and more rewarding than assembly level optimization, because the rewards are so much greater.

    This is mostly undiscovered territory, but I found one interesting link.

    Note that I'm not talking about diddly high-level tasks in language like Python, but even things like image processing. It doesn't matter. Sticking to C and C++ for performance reasons, even though you know there are better languages out there, is a backward way of thinking.

    1. Re:These kind of benchmarks are so 1970s by EvilTwinSkippy · · Score: 2, Insightful
      I'm a TCL nazi. Back when I started (hint, the Pentium Bug was news) we would have to compile critical sections of our programs in C to get any kind of acceptable performance.

      Today, everything is in script because it's not worth the bother anymore. In 1998 I had to write my own affine transformation code in C to get a GUI to work at anywhere near real-time. Today I can run a planetarium simulator (read LOTS of calculations) at an acceptable speed in just script.

      --
      "Learning is not compulsory... neither is survival."
      --Dr.W.Edwards Deming
  28. Re:They should benchmark development time by SlashdotLemming · · Score: 2, Insightful

    Dynamically typed? Bleh.

    I don't know much about Python and I'll give it a go when I get a chance, but it's really hard to take your comments seriously when you call Python a "Silver Bullet" in your sig ;)

  29. Re:What about coder's performance? by Dalroth · · Score: 4, Insightful

    Raw performance will ALWAYS be an issue. If you can handle 100,000 hits per day on the same hardware that I can handle 1,000,000 (and these are not made up numbers, we see this kind of discrepency in web applications all the time), then I clearly will be able to do MORE business than you and do it cheaper. That gives me a competitive advantage from now till the end of time. If you throw more hardware at the problem, well, so can I and I'll still be ahead of you.

    Performance realities do not go away, no matter how much we may wish they would. Now, does that mean you're going to go write major portions of your web application in assembly to speed it up? No, probably not. But your database vendor may very well use some tricks like that to speed up the key parts of their database. You sink or swim by your database, so don't say it doesn't matter because it absolutely does.

    Anyway, in my day-to-day operations, I can think of quite a few things that get compiled directly to executable code even though they don't have to be. Why would you do this if performance wasn't an issue and we could just throw more hardware at it?

    1. Regular expressions in the .NET environment are compiled down to executable code, then executed.

    2. XSL transformations in the .NET environment are compiled to a form of executable code (I don't think it's actual .NET byte code, but it may be) and then executed.

    3. The XmlSerializer classes creates a special compiled executable specifically created to serialize objects into XML (byte code!!).

    And the list just goes on and all of this eventually ends up getting JITed as well. My pages are 100% XML based, go through many transformation steps to get to where they need to be, and on average render in about 70-100ms (depending upon the amount of database calls I need to make and the size of the data). This all happens without spiking our CPU utilization to extreme levels. There is *NO WAY* I could've done this on our hardware if nobody cared about performance.

    As always, a good design is the most important factor. But a good design that performs well will always be superior to one that doesn't.

    Bryan

  30. Re:Sitting on a Benchmark by happyfrogcow · · Score: 2, Insightful

    is like comparing a bird and a dolphins ability to swim in water

    Our beloved Penguins can swim quite well under Linux^H^H^H^H^Hwater, thankyou!

  31. Re:Trig functions... by dnoyeb · · Score: 2, Insightful

    There's that, and the fact that MS does not have any "languages."

    On a realer note, the JVMs are written in "C" a fact that some people just don't seem to understand. A Java program when running is a form of a C program. Thus, their is no reason to have slower math functions except that the JVM was poorly written.

    The whole comparison of non-graphic Java to C or C++ is moot as C or C++ is the basis of all JVMs I know of.

  32. G++? by phorm · · Score: 2, Insightful

    I know it ties into the GCC libs, but does G++ behave any better/worse than GCC. Comparing VC++ (a C++ compiler) and GCC (a C compiler) is a bit skewed. Also, how about a comparison of GCC in windows VS linux (comparable machines), just to see if the OS has any bearing on things?

  33. Re:Trig functions... by Anonymous Coward · · Score: 2, Insightful
    On a realer note, the JVMs are written in "C" a fact that some people just don't seem to understand. A Java program when running is a form of a C program. Thus, their is no reason to have slower math functions except that the JVM was poorly written.


    What's your point? One could argue that any program when running is a form of machine code thus they should be running as fast as they possibily can. (Which is true of course!)
  34. Re:Java as fast as c++????? by radish · · Score: 2, Insightful

    So it couldn't possibly be a problem with the app? It has to be the language it's written in? By that logic, C/C++ must suck really badly, because we all know how unreliable Windows 95 was. Puuuurlease.

    --

    ---- Den ene knappen er powerknapp, den andre er Bender voice knapp "Bite My Shiny Metal Ass"

  35. Re:They should benchmark development time by rveety · · Score: 2, Insightful

    Comparing Perl to Java is foolish, Perl is more like Awk than a general purpose programming language, and not meant for large projects at all.

    Ummm.. Slashdot is written in Perl, as are many other large projects. I've yet to see anything like Slashdot written in Awk.

  36. Re:Trig functions... by jorleif · · Score: 2, Insightful

    The JVM might be written in C and be a native program, but it's still running the Java bytecode in a virtual machine. If the virtual machine does not have the math functions you mentioned as primitives their performance depends on the JVM/JIT-engine.

    So in a benchmark comparing compiler performance I can't see how that is "moot".

  37. Java benchmarks are flawed. by _LORAX_ · · Score: 2, Insightful

    1) JIT optimizations don't always kick in until a function has been run several times. Since the bechmarks only run once, they are crippled on java.

    2) Java's IO function work on UTF-8 or other system dependant character set. So in essence java is doing twice the ammount of work during the IO benchmark.

    I'm sure other people will comment as well, but overall these numbers are not that suprising for code that was just copy and pasted from c code. Why do people expect that ANY language will perform well using another languages code.

  38. Cost of Hardware vs. Cost of wetware by G4from128k · · Score: 4, Insightful

    Raw performance will ALWAYS be an issue. If you can handle 100,000 hits per day on the same hardware that I can handle 1,000,000 (and these are not made up numbers, we see this kind of discrepency in web applications all the time), then I clearly will be able to do MORE business than you and do it cheaper.

    You raise excellent points. For many enterprise and server applications, performance is an issue. But I never said one should care nothing abut performance, only that in many applications the cost of the coder also impacts financial results.

    For the price of one software engineer for a year (call it 50k to 100k burdened labor rate), I can buy between 20 to 100 new PCs (at $1000 to $3000 each). If the programmer is more expensive or the machines are less expensive, then the issue is even more in favor of worring about coder performance.

    The trade-off between the hardware cost of the code and the wetware cost is not obvious in every case. A small firm that can double its server capacity for less than the price of a coder. or the creators of an infrequently-used application may not need high performance. On the other hand, a large software seller that sells core performance apps might worry more about speed. My only point is that ignoring the cost of the coder is wrong.

    These different languages create a choice of whether to throw more hardware at a problem or throw more coders at the problem.

    --
    Two wrongs don't make a right, but three lefts do.
  39. Re:They should benchmark development time by ultrabot · · Score: 2, Insightful

    Rubbish. A skilled Java programmer could write a typical piece of code just as quickly as a skilled python programmer - the difference is that becomming a skilled python programmer is easier.

    Do you base this assertion to actual experience, or just a hunch that "it surely must be so"? If both languages are used to solve the same problem, the Python program is much more concise. It's not physically possible to create the Java program as quickly, given the same typing speed. Not to mention the difference in semantic complexity, which determines how fast you can churn out that code (assuming nonzero brain latency).

    I guess people who have never tried dynamic typing can't comprehend how much faster development can be using it.

    --
    Save your wrists today - switch to Dvorak
  40. Not a fair test - Frame Pointers by Compenguin · · Score: 3, Insightful

    His benchmark isn't fair, he's omitting the fame pointer on VC++ but not gcc. How is that fair?

  41. Re:trig calls in gcc by greg_barton · · Score: 3, Insightful

    On top of this, the Java/Linux VM from Sun (and most other Java/Linux VMs) is compiled using gcc. I don't know for sure, but I wouldn't be suprised if Sun erred on the side of correctness and standards compliance, so I'll bet they compiled the VM without inlining any trig function calls.

  42. Consider the logic... by tizzyD · · Score: 3, Insightful
    But that's what you know so well. I bet if I took a team of VB programmers who knew VB as well as you know VC++, and then challenged them to produce a series of business oriented apps, in most cases you'd lose. How many VC++ programmers does it take to create a business app? 2 * VB Programmers usually -OR- how long does it take to create and debug a business app in VC++? 2 * VB time. ;-) JK!

    Guido van Rossum noted in an interview the following statistic, and I think it bears considerably on appropriateness:

    This [ed: these stats] is all very informal, but I heard someone say a good programmer can reasonably maintain about 20,000 lines of code. Whether that is 20,000 lines of assembler, C, or some high-level language doesn't matter. It's still 20,000 lines. If your language requires fewer lines to express the same ideas, you can spend more time on stuff that otherwise would go beyond those 20,000 lines.

    A 20,000-line Python program would probably be a 100,000-line Java or C++ program. It might be a 200,000-line C program, because C offers you even less structure. Looking for a bug or making a systematic change is much more work in a 100,000-line program than in a 20,000-line program. For smaller scales, it works in the same way. A 500-line program feels much different than a 10,000-line program

    So then, unless you quantify the types of apps you build, the team you use, and the results that are expected, my experience has shown me that most of the time, for business apps, it's overkill. Now, if you're in a dev team at a software company, well then, I could consider the other side.

    --
    ...tizzyd
  43. Coming in 1.5, but you can do this now by SuperKendall · · Score: 4, Insightful

    That's a feature built into Java 1.5, but you can get a test reference implementation which is about 96% of the features now to try it out. It has a really clean syntax and provides the benefit you seek.

    --
    "There is more worth loving than we have strength to love." - Brian Jay Stanley
  44. Reminds me of... by vasqzr · · Score: 2, Insightful


    Reminds me of my 6th grade 'science fair' project.

    I took a couple different compilers, languages, did some loops and math and such, timed them all.

    "Which computer language is the fastest"

    About half way through the project I realized how big of a waste of time it was.

    What kinds of things should you be testing?

    Speeds of function calls???

    Implement various sorting algorithims?

    Audio/Video compression/decompression?

    When it comes down to it, it's all the same math, and any good compiler is going to come close to making the same darn code.

    By now, we all know that you use one language for one thing, and another language for another. For various reasons.

    A hammer is your only tool if all your problems are nails, isn't the cliche?

  45. Re:What about coder's performance? by FiskeBoller · · Score: 2, Insightful

    ... and Smalltalk zealots not too far behind.

    I did my own productivity benchmarks between C++ and Smalltalk in 1996. I consider myself very adept at both languages. At the time I was coding C++ CORBA internals that had to function across 10 platforms (gawd, what a pain). I was also involved in a Smalltalk ORB project.

    My productivy benchmark was completing foundation frameworks for a financial trading package. The time required to complete equal functionality in C++ was 10x the amount it took in Smalltalk. I found it agreed with similar claims in language productivity.

    It's important to consider the context in which the program will operate. This drives the requirements a solution will need to fulfill, and in turn, may influence the choice of environment, frameworks, libraries, and language.

    There are cases where the speed of delivering accurate, new functionality is paramount. In these cases, I wouldn't want to be using C++.

  46. Re:Trig functions... by shemnon · · Score: 3, Insightful

    First off, SWT only performes well on windows, and stack on top of that that the principal native abstractions are taylored to a win32 environment. Based off of that it is easy to see how SWT performes quite nicely on Windows.

    Elsewhere it sucks. MacOS, GTK, photon, Motif. Even porrly writeen swing programs outperform on those platforms.

    But back to your FUD. Yes, bad programmers make ugly and poor performing GUI code. Swing is no different in that regard. But have you looked at recent swing programs in the 1.4.2 version of the JDK? Tried stuff like CleverCactus (a mail client)? Synced your MP3s on a new RIO? Used Yahoo's Site Builder to make a web site? There are excelent swing progams out there. Many you probobly don't realize are java swing apps!

    But since SWT is only in early adopter land we haven't seen the real dogs of GUIs it can make yet, especially since you have to do such arcane and ancient tasks in SWT as managing your own event queue! :( Give the same bad programmer SWT and you won't get a bad GUI instead you will get a non-fucntioning GUI.

    --
    --Shemnon
  47. .NET benchmark flawed. It is faster. by Anonymous Coward · · Score: 3, Insightful

    I see just one small issue with the benchmarks. Microsoft claims, that all .NET languages are compilled at the runtime. This means, that the first pass of the execution through the function has a compile time added on top of the execution, which falsifies somewhat the .NET execution time benchmark. I did some simple tests that confirm this. To my surprise, .NET languages are actually faster than Visual C++, Borland C++ or GNU C+ for a simple 1/n series calculation without visible loss of accuracy. Don't ask me how it is possible. I don't know, but it is a fact that my benchmark shows. My best guess would be that the just in time compiler is better in getting code optimized for the CPU in the particular machine it runs or maybe it is better in filling the cache. The key of the benchmark is to write software in such a way that it runs through the function at least two times. The first time it runs just to allow just in time compiler to compile the code and then it runs subsequent times to measure performance. Below is the schematics of my benchmark:

    double benchmark(int number_of_iterations);

    void main (void)
    {
    Time start,end;
    double outcome;

    benchmark(1); // This is to allow .NET "just in time" compiler to compile the benchmark function

    for(int i = 1; i < 11; ++i)
    {
    start = CurrentTime(); // CurrentTime is a placeholder here for a system time function in ticks
    outcome = benchmark(i*1000000);
    end = CurrentTime();
    lprt (i,outcome,end-start); // lprt is a placeholder for a nice formatting print here
    }
    }

    double benchmark (int number_of_iterations)
    {
    double s,t;

    s = 0.0;
    t = 1.0;

    for(int i = 1; i < number_of_iterations; ++i)
    {
    s += 1.0/t; // This is the body of the benchmark
    t += 1.0;
    }
    return (s);
    }

    As you can see above, I run the benchmark function once with counter of 1 and ignore its outcome before starting to measure time. The key is to allow compiler to compile the benchmarking function before running actual benchmark. Once it is done, I run then the benchmark 10 times for succesively larger counter from 1 billion to 10 billion and print number of iterations (in billions), the accuarcy and the time it takes to run. The idea here is that under the assumption that the benchmark time is related to number of iterations as a linear function I can easily find linear best fit function between number of cycles and run time in the form of

    time = a * number_of_cycles + b

    and then use value of a as a measurement of the benchmark. The value of b is good check, how the benchmark behaves. If it is large, then something went wrong. In my case it was always close to zero. I'm now away from my home computer and I don't have all the compilers, that were tested in this article, so I can't repeat those benchmarks modified to this method at the moment, but you guys might try to do it yourself.

    Some people might challenge this by stating that the compile time for .NET is part of the execution time but I disagree. My position on this is, that in most real life cases the software runs into the particular functions many times thus creating long exectution times. It is rare, that a signle function call creates long exectuion time that is annoying to the user.

    Best regards.

  48. Delphi & Kilyx by the_mushroom_king · · Score: 3, Insightful

    Why are the two best Borland languages never included in benchmarks? Maybe in just the odd-ball that doesn't use C++, Java, or Micro$oft. - TMK

  49. Reader/Writer classes in java benchmark affect res by mlamb · · Score: 2, Insightful

    By replacing the Reader/Writer classes in the java benchmark with their InputStream/OutputStream counterparts I realized a 24% IO performance boost when running the 1.4 JVM with the -server option. The Stream classes don't bother with the unicode conversions that the Reader/Writer classes do. Since the other benchmarks didn't perform unicode conversions (at least the C/C++ ones - can't speak for the other langs), this seemed like a reasonable modification.

    Curiously, without the -server option, this resulting in a 78% performance HIT.

    - Marty

  50. Re:Trig functions... by Nurgled · · Score: 4, Insightful

    Faster processors should enable us to achieve more, not achieve the same old stuff much less efficiently.

  51. wow, talk about a lame benchmark by autopr0n · · Score: 2, Insightful

    I mean, the functions are so simple, the code generated doesn't stress anything. Not any of the 'advanced' compiler, or even architecture features. None of the good features of a JIT.

    I mean seriously, they do math on all the ints from one to one billion. Why even bother? Adding large 32 bit ints takes exactly the same amount of time as adding small ones (but I guess you save one variable by doing math with the counter. Or one extra line of code saved)

    I'm sorry, but this is the most pointless compiler benchmark ever.

    A good language comparison would be to have a bunch of groups of people try to code up the best implementation they could in whatever language, of some complex problem, and use that as the baseline.

    --
    autopr0n is like, down and stuff.
  52. Re:Trig functions... by tjansen · · Score: 2, Insightful

    >>Sorry, dude, but SWT is nowhere *near* as complete as Swing, in terms of functionality.

    Maybe, but most *users* will chose a SWT app over a Swing one anytime. Actually most users will refuse to use a Swing app, they feel strange and look ugly.
    The main reason for Eclipse's success (and the demise of the other free IDEs) is that only Eclipse offers a pleasant GUI, which no Swing-based IDE can.

  53. Re:Trig functions... by matfud · · Score: 2, Insightful

    Suns implementaion is only a reference implementaion. Other providers can provide faster/different implementaions.

    the reason for the slowdown between 1.3 and 1.4 is that 1.3 introduced a new class called StrictMath that provided better cross patform consistency the standard Math class. It was slower though. In 1.4 the standard Math class was rewritten so that internally it uses the StrictMath class.

    Other implementaions do not have to use this approach.

    matfud

  54. Re:Trig functions... by Milo77 · · Score: 2, Insightful

    "select" is actually very useful for the case where you might want more than *one* thread for a connection. in the case where a client is properly multithreaded, you can continue to receive requests even though previous requests haven't yet completed. the ideal solution is to have a thread pool, where a thread is assigned to incoming requests, not just an incoming connection. in the latter case you can get the situation where you may have 10 threads (one per connection), but only a single connection is getting hammered(and all client threads are being forced to wait on a single server thread - which is probably io bound). in the former case, you'll have 10 threads in a pool with no ties to any particular thread. when a single connection starts getting hammered, more threads can be assigned to serivce that single connection. Java's nio (and select) are perfect for implementing such a system. so, while select can be used as you suggest in order to decrease throughput, it can also be used as i suggest to increase it dramatically.

  55. Re:If speed isn't an issue by arevos · · Score: 2, Insightful

    I assume you don't acutally mean "standard" but "common".

    Unofficially standard ;)

    Does it have pointers or not? (Yes it does - and yes they are restricted - and the issues that that cuases is not overly severe).

    It has references, which aren't the same thing. C++ has references (e.g. char&) and pointers (e.g. char*). And yes, when I say it makes "pointers safer" I mean referencing. Whatever :)

    Java is a FAR, FAR, FAR, beter ___LANGUAGE___ than C++ - ignore the Vm, the libraries, etc... and look at the language - Java is way better.

    I have. I've used Java and C++ in two rather large projects. And I really can't see why Java is that much better. Or, indeed, that much different.

    I don't have to worry about pointers going astray (but then again, I don't have pointers full stop), or garbage collection, but apart from that, what's the difference? Ignoring the libraries, that the only thing I can think of that's different. Well, except that C++ has multiple inheritence, and so forth.

    Perhaps I've missed something. Could you explain why it is "way better"? Remember to ignore the Vm, the libraries and just to focus on the language.

    C++ would have been stillborn if it was not C-like - Java would have been if it was not C++ like.

    Agreed. That doesn't make Java a good language, though; that's just a reason why it's bad.

    That said, I've never really understood that argument. I mean, how long does it take to learn a new language? A few hours for an experienced programmer, really. Making Java C++ may have saved a day or so of programming time on one programming project, but that's nothing compared to how long software takes to develop. On the other hand, companies rarely act in a logical fashion when it comes to software.

  56. Re:They should benchmark development time by Empyrean9 · · Score: 2, Insightful

    "Dynamically typed object model scales incredibly well."

    Sorry, no it doesn't. You can't overload functions. You can't use true polymorphism. You can't enforce inheritance. You can't code to an interface, or abstract class. You can't just look at a function definition and know what type of parameters it contains. You can't detect type-based errors until runtime. These are all the things that will slow you down in six months, and beyond. I've been there, and thats why I will never go back to a dynamically typed language for anything over 2,000 lines.