Slashdot Mirror


Comparing the Size, Speed, and Dependability of Programming Languages

In this blog post, the author plots the results of 19 different benchmark tests across 72 programming languages to create a quantitative comparison between them. The resulting visualizations give insight into how the languages perform across a variety of tasks, and also how some some languages perform in relation to others. "If you drew the benchmark results on an XY chart you could name the four corners. The fast but verbose languages would cluster at the top left. Let's call them system languages. The elegantly concise but sluggish languages would cluster at the bottom right. Let's call them script languages. On the top right you would find the obsolete languages. That is, languages which have since been outclassed by newer languages, unless they offer some quirky attraction that is not captured by the data here. And finally, in the bottom left corner you would find probably nothing, since this is the space of the ideal language, the one which is at the same time fast and short and a joy to use."

26 of 491 comments (clear)

  1. Why is Verbosity Bad? by eldavojohn · · Score: 5, Insightful
    This plots verbosity against slowness making:

    And finally, in the bottom left corner you would find probably nothing, since this is the space of the ideal language, the one which is at the same time fast and short and a joy to use.

    I must ask why the author assumes that verbosity is bad and why lack thereof makes it a "joy to use."

    I think verbosity in moderation is necessary. I have read many an article with developers arguing that they don't need to document their code when their code is self-documenting. Do you make all of your variables and class/function/methods a single character for the sake of verbosity? I hope not. And I would think that reading and maintaining that code would be far less than a joy.

    I don't even need to argue this, according to his graphs we should all be using Regina, Mlton or Stalin (a scheme implementation). But instead languages like Java and Perl and C++ prevail. And I would guess that support and a mediocre range of verbosity are what causes that.

    Great work in these graphs! But in my opinion, verbosity when used in moderation--like a lot things--is far better than either extreme.

    --
    My work here is dung.
    1. Re:Why is Verbosity Bad? by xZgf6xHx2uhoAj9D · · Score: 5, Insightful

      I agree totally about the need for verbosity. I'm a fan of Donald Knuth in this regard.

    2. Re:Why is Verbosity Bad? by jbolden · · Score: 4, Insightful

      Look where those languages are on the chart.

      Java mostly against the left wall (i.e. mostly as fast as C). Perl right against the bottom (i.e. very small code).

      It appears to me what this showed was that people like the walls.

    3. Re:Why is Verbosity Bad? by ucblockhead · · Score: 5, Insightful

      The author is not talking about verbosity in bytes. He's talking about verbosity in code points. Talked about in this way, a thirty character variable name is no more verbose than a single character variable name.

      --
      The cake is a pie
    4. Re:Why is Verbosity Bad? by tknd · · Score: 3, Insightful

      There's good verbosity and bad verbosity. Let's take for example java's System.out.println. This is bad because it is such a common function that you are bound to use it over and over again but good of course because you know it is not a keyword (it comes from the System.out library). The result is something twice as long as it needs to be and chews up screen space. As a human reader of the source, this becomes cumbersome and harder to read the full program.

      Languages that can allow you to pull in identifiers from other scopes can solve this issue while reducing the bad kind of verbosity. For example in perl there is something called Exporter which allows defined symbols by the source package to be imported into the current scope through syntax like:

      use Some::Package qw( someFunction );

      Now instead of Some::Package::someFunction, for the rest of the file I can just do someFunction since I've already declared I'm importing it at the top. If the reader is interested in knowing where someFunction comes from, searching from the top of the file will reveal the use Some::Package qw( someFunction ); line.

    5. Re:Why is Verbosity Bad? by timeOday · · Score: 5, Insightful

      I'm not sure we're sharing a definition of "verbosity. Perhaps the useful definition here is "the inverse of expressiveness." Of course expressiveness is inherently relative to what is important in a given application. For most purposes, assembler is hopelessly verbose, or lacking in expressiveness, since most of what you're writing out is meaningless detail. But for certain other purposes (performance) those details become crucial, so for that application only assembler has the necessary expressiveness. Again, I am not contradicting myself by saying assembler is both lacking in expressiveness and high in expressiveness, it depends on the application. I've heard German is a fantastic language for writing technical legal documents, whereas the Inuit language is supposedly very good for describing cold weather and snow.

    6. Re:Why is Verbosity Bad? by ChrisMaple · · Score: 3, Insightful

      One thing in C that many people complain about is that it's too easy to get in trouble with pointers. Part of this problem is due to the way that pointers are declared often looks different from the way pointers are used, particularly if pointers and array indexes are freely interchanged. The confusion is due to a poorly chosen system of expression.

      --
      Contribute to civilization: ari.aynrand.org/donate
    7. Re:Why is Verbosity Bad? by ArcadeNut · · Score: 4, Insightful

      verbosity != self-documented

      Ruby example:
      do_something() if some_time 7.days.ago

      To me, this line of Ruby code is perfectly clear and self-documented. It is also about as short as you could make it. If I had to write the same code in Java, it would be long enough that I would feel it required a comment.

      This is where so many programers go wrong. The code is self documenting in the respect that you know it's checking the age of something to see if it was 7 days ago, but you have no idea WHY it's checking that. Why not 6 days ago? What if it's 8 days ago?

      Comments should say WHAT and WHY you are doing something not HOW you are doing it. The HOW is the code itself. If someone was to look at that code, do they know what your intent was? Were you looking for something that was exactly 7 days old, or something 7 days old or older.

      --
      Visit the Arcade Restoration Workshop @ http://www.arcaderestoration.com
    8. Re:Why is Verbosity Bad? by dpilot · · Score: 4, Insightful

      Vermont ski areas have more words for "snow" than the Inuit.

      But they all mean "ice".

      --
      The living have better things to do than to continue hating the dead.
  2. Pet peeve by QuoteMstr · · Score: 5, Insightful

    Programming languages don't have attributes like size and speed: implementations of these languages do. Take Common Lisp for example: SBCL is blazing fast, while CLISP is rather pudgy (albeit smaller). Any conforming Common Lisp program will run on both. Or consider Python --- IronPython and CPython have different performance characteristics. (I'm too lazy to link these now.)

    Point being, describing a programming language as "fast" makes about as much senese as describing a natural, human language as "smart".

    1. Re:Pet peeve by Anonymous Coward · · Score: 4, Insightful

      A badly-written Assembler program will never achieve the speed of a reasonably-written C++ program. And most people cannot write good Assembler code.

    2. Re:Pet peeve by jilles · · Score: 3, Insightful

      Yes of course you are right. And more recently we actually have languages with coherent and consistent behavior across implementations (somewhat of a novelty, just look at C implementations). There's several Ruby interpreters that run the Rails framework now. The fastest one (or so it is claimed often) is JRuby, which runs on top of the Java virtual machine, which has has quite many implementations on a wide variety of hardware optimized for running on e.g. embedded hardware or on CPUs with thousands of cores and impressive levels of compatibility given these differences. So saying language X is faster than language Y is a quite meaningless statement these days. Faster on what, and under what conditions and at what required level of stability, and with what kind of benchmark? Most C programs are fast, except they have all but 1 core on the CPU idling because threading and concurrency are very hard to bolt on to a C program. Which is why some performance critical messaging servers are done in languages like Erlang.

      Most C programmers believe it is the closest thing to native aside from assembler. Probably correct if you ignore 40 years of progress in the hardware world but downright naive in light of modern day X86 processors. Technically x86 is not a native instruction set anymore but a virtual machine language that happens to have an in hardware translation to the 'real' instruction set. Like all such translations, it comes at a cost of lost flexibility, and indeed performance. But it is worth avoiding having to rewrite all those pesky compilers. So rather than shatter the assumptions C programmers make, they actually support them by sacrificing transistors. The only real difference between Java and C execution model (aside from better defined semantics in Java) is that Java does the translation in software, at run-time, taking into account performance characteristics of both the running program and the underlying hardware. That in a nutshell is why the LLVM project exists to do the same for C programs (and indeed many other languages, including Java).

      Of course you have to take into account the levels of abstraction and indirection provided in application frameworks as well. Those come at a cost and that cost is typically high in scripting languages (but you get so much in return). Java is often unfairly compared to C. I say unfairly here because it is usually a comparison of application frameworks rather than the language implementation. Java is in 'laboratory' conditions quite fast, even comparable to C and applies all the same performance optimizations (and then some). Except, nobody programs Java that way (except some dude at work who managed to produce the suckiest Java class ever, different story) . Similarly, C mostly lacks the rich frameworks that are common in the Java world. Transcode a C program to Java and you end up with a code base that is still pretty fast (e.g. quake 2 has a Java port). Stupid people in both camps believe that it is an either-or type decision between the two and that it is somehow inherent to the language what you end up with. Clever engineers know that 95% of their code is not performance critical at all (i.e. cpu idling most of the time), and that it makes a hell of a lot of sense to do whatever is necessary to get that 5% performing as best as possible if the cost in terms of stability, productivity, security, portability, etc is low enough. That's why server-side c is not a commonly required job skill.

      That's why Apple used the llvm platform to port Mac OS X to mobile phones and why Google chose to implement a heavily customized Java vm on top of familiar C/C++ components to emulate their success. Good engineers know when to choose what. Both iphone and android are excellent achievements that move beyond the stupid status quo of wondering which language is better.

      --

      Jilles
  3. Verbosity is bad because by Limerent+Oil · · Score: 3, Insightful

    Verbosity = ( 1 / Expressiveness )

    1. Re:Verbosity is bad because by Goaway · · Score: 4, Insightful

      Nowhere near true. Objective-C is more verbose than C, yet still more expressive, and largely self-documenting to boot.

  4. Re:Scala by jbolden · · Score: 4, Insightful

    Functional languages in practice often implement nlog n algorithms in quadratic time or memory. One of those in a bench mark is devastating. We really understand how to optimize imperative languages well, we don't have the same level of knowledge / experience regarding functional.

    I agree this is a pity, but there doesn't seem to be any easy solution. Hopefully this gets fixed over the next generation.

  5. Re:Related site... by alvinrod · · Score: 4, Insightful

    What's so special about it? It looks just like a regular perl program to me. /ducks

  6. Re:Related site... by viyh · · Score: 4, Insightful

    Perl is all of the above; it is what you want it to be. That's the beauty of it. It can be a completely obfuscated ASCII art drawing or it can be a very concise, to-the-point one-liner or anything in between. It's the Swiss army chainsaw.

    --
    "I have never let my schooling interfere with my education." --Mark Twain
  7. Ruby by RiotXIX · · Score: 4, Insightful

    On the plus side, both versions of Python can claim many of the smallest programs in the collection. Ruby (8, 1) might also compete for titles, but unfortunately its performance is so bad its star falls off the performance chart.

    Then why the fuck is the Ruby community hyping it so much, and drawing nieve young developers in to a trap?

    Not flamebait.

    Why can't they make a language, or extend a language like Ruby, such that one can program it as a scripting language, but then add verbosity optionally (i.e. declaring the data types and their sizes, private / static etc. & whatever the hell makes a program light weight and fast) optionally? It's my hope that if I stick with Ruby one day it I won't be forced to learn Python because performance won't be "Ruby's big issue" in every discussion, but really, that is *just* a hope. I hope this isn't a mistake.

    --
    "You know you don't act like a scientist, you're more like a game show host." Dana Barret
  8. Re:Ocaml by 16384 · · Score: 3, Insightful

    I don't about the rest of you, but functional languages don't fit my brain, and to me it's not worth the struggle. On the other hand, procedural/OO languages are no trouble at all, and I can learn new ones quickly. I don't understand why anyone would choose lisp, scheme, etc. Maybe they were born with bigger stacks, because when working with functional languages I'll go into stack overflow mode in a jiffy :-)

  9. Re:Scala by DetpackJump · · Score: 3, Insightful

    I was curious about that too, so I'm looking at the code. The Java implementation seems to be importing a non-java library, jgmplib. Anyone know what is going on here?

  10. Re:Related site... by CastrTroy · · Score: 3, Insightful

    Run Javascript through an obfuscator, and see how readable it is. "readable in all circumstances" can't possibly exist. Anybody could make unreadable code in any language.

    --

    Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
  11. Re:Forth, the RPN notational programming language by rbrander · · Score: 5, Insightful

    I fell hard for FORTH in the mid-80's - it was the programming language of some instrumentation I had to work with. I never did program the instrumentation, but got all the books on the language and interpreters/compilers (the distinction blurs in FORTH...) for it for a number of computers. I did some useful utilities for the PC using "HS/FORTH". I benchmarked it against some other solutions for a course in comparitive computer languages about 1985, and for heavily stack-based (ie very recursive) programs, nothing could touch it. I'm talking QuickSort faster than C.

    But I have to admit: though I busted butt, re-writing and re-re-writing programs to make them as clear and readable as possible, though I re-read the remarkable "Thinking FORTH" by Leo Brodie many times (and would recommend it to anybody who wants to understand the craft of programming, whether they want to learn FORTH or not)....despite it all, most of my FORTH programs were hard to read a few months later.

    There were things it was really good at - that is, the programs WERE readable later, the problem and language were a match - and others where I could just not seem to decompose the problem into FORTH words that fitted well together without a lot of "glue" to manipulate the parameters on the stack.

    That was the most frequent problem with my FORTH programming - one ends up trying to manage several parameters on the stack and doing a lot of stack "DUP SWAP ROTATE" - actions to line up the parameters to hand to the next word. I would re-compose the words and the parameters they wanted to clean up some code, and find I'd made some other code all hard to read.

    FORTH was also profoundly command-line oriented, and when the word went all GUI-centric on me with no good "Visual FORTH" or "Tk/FORTH" on the horizon, I slipped from grace. I can't see getting back to it now, either; lets face it, a huge bonus for any programming language choice is its popularity, so that others will maintain your code, so that you can get help and code fragments with a quick google.

    But I still think that FORTH should be a completely MANDATORY learning experience in all University and Tech CompSci degrees. You can jump from C to Perl to Python far more easily than to FORTH - it really comes at problems from another angle and working with it for years has been an enormous asset to my non-linear thinking when it comes to problem-solving.

    And perhaps if more students learned it, FORTH would rise in popularity for some problems, out of its decades-long sinecure in embedded systems (it started off programming radio telescopes, and undoubtedly still does...) Since it is inherently object-oriented (yes, an assembler-sized OO language from the 1970's, you heard that correctly) it would be an excellent interpretive, experimentation-friendly scripting language for applications. I'm currently needing to do a lot of VBA in Excel at work, and I have a strong suspicion I'd be twice as productive in "FORTH for Applications". It's a tragedy Bill Gates fell for BASIC (of all languages) instead.

  12. island paradise by epine · · Score: 5, Insightful

    When the first 4K TRS-80 showed up at my high school, I had the option to learn BASIC, Z80 machine code, or APL up the street at the local university.

    One of my early exercises with BASIC was writing a set of nested for loops which called a subroutine (gosub). I put the next statement that controlled the for loop iteration inside the subroutine, and the return statement for the subroutine inside the nested for loops. It still worked! At that point I understood that there were mechanistic languages and languages with a solid conceptual basis.

    APL's reputation for inscrutability was only halfway deserved. Often the problems arose when you were trying to shoe-horn a data structure that didn't want to be an array into an array, because that was your only hammer. Later APL supported nested arrays, which increased the data structuring options, but I think by then the PR battle was lost.

    In the original APL, it was kind of painful to pass more than two arguments to an APL function. This lead to programmers passing in flags to the function encoded in the array's rank, which were extracted to the tune of rho rho rho, while imaging the knapsack folding problem in Colossal Cave Adventure. As brutal as any language I've used. But you have to give APL a bit of a pass in some respects. Like vi, it was designed in 1963 to work well within the constraints of a paper teletype.

    The next level of inscrutability arose because the APL primitives could often be combined in novel ways to yield surprisingly powerful algorithms. IIRC, the IBM 370 APL included a JIT compiler for certain common APL idioms. A one line program I wrote in APL to find primes (sieve of Eratosthenes) ran ten times faster than the compiled PASCAL program by the CS student sitting next to me.

    Understanding APL was a lot easier if you were familiar with functional programming languages, but these hadn't been invented yet. Hey, I didn't know this: the Wikipedia page credits APL as a direct influence on FP, which I first heard of in 1982. Father knows best.

    So you encounter this unfamiliar pattern of 15 familiar symbols for the first time, and you brain is polluted with horrible iterative solutions from BASIC or PASCAL, and the beauty of the expression is denied to your limited frame of consciousness.

    Like solving a Suduko? Hardly. It takes me twenty minutes to solve a typical five star Sudoku. It used to take me about the same amount of time to puzzle out an unfamiliar APL one liner, which might be anywhere from 10 to 40 characters. There is one small difference: after decoding the APL algorithm, I usually slapped myself across the head and moaned to myself, "I am unworthy to drool on the shoe laces of the grand designer, but I will learn!" Never got that feeling from Sudoku.

    Wrestling with the higher art of APL was like giving your ignorance a root canal. Sometimes the root canal made me barf up my milk: when the highest art of APL was applied to shoe horn a data structure unsuitable to array representation into an array representation anyway, like the Beethoven scene in Clockwork Orange.

    The third case is where the one liner isn't all that difficult, but it's doing it in more dimensions than the brain wishes to visualize. This is a case where a picture is worth a thousand words. Your 20 character APL function would have been better presented as a caption on a one page UML diagram. Never figured out how to embed a UML diagram in an APL lamp statement on my VT100 terminal.

    Another problem APL suffered was too much kinship with Forth. To thrive in APL, you needed to create hundreds of tiny APL functions, which the implementations of the day mashed together into a single unmaintainable workspace.

    And the system interface tended to suck.

    But other than that, what's not to like?

    I could have composed instead a tedious, but germane post on Shannon's first law: concision is a function of preconception. It's a rare breed of programmer who thrives in a language which provides su

    1. Re:island paradise by Anonymous Coward · · Score: 3, Insightful

      Lisp, arguably the grandfather of a lot of programming languages, was specified around 1958. While the first efficient implementations were a long time coming from that, I don't think you can claim APL was a forerunner to FP languages.

      Nice post otherwise :-)

  13. A language does not have performance by brunes69 · · Score: 3, Insightful

    Performance is created by the compiler, not the language. A C program compiled with a shitty compiler is going to run slower than a Ruby one in a good VM, even though C is running native on the CPU. For that matter, what if I take the C code and compile it with the CLR as a VM target?

    I wish people would stop trying to compare languages by performance, it does not make any sense. The only language it makes any sense for at all is assembler.

  14. Re:What kind of verbosity? by rrohbeck · · Score: 3, Insightful

    I prefer
    print foreach @list;
    though. Perl FTW as usual :)