C++ the Clear Winner In Google's Language Performance Tests

Paul Dubuc writes "Google has released a research paper (PDF) that suggests C++ is the best-performing language on the market. It's not for everyone, though. They write, '...it also required the most extensive tuning efforts, many of which were done at a level of sophistication that would not be available to the average programmer.'"

Comments

Common knowledge

[PitaBred]

I thought this was common knowledge. Did anyone ever doubt that?

Re:Common knowledge

[Anonymous Coward]

It's been common knowledge for at least a decade that Java is 6 months away from being quicker than c++.

Re:Common knowledge

[AuMatar]

There's a lot of Java-ites who claim that Java is just as fast. They're idiots, but they're vocal.

Re:Common knowledge

[EvanED]

A GC is a net loss, but don't think it doesn't have good effects mixed in there. Allocation of memory is a few instructions with a GC; malloc() can't hope to be in that same league. On a whole, GCs improve the memory locality of your program as it runs, with some substantial hiccups at the collections; manual memory management hinders it as your heap becomes more fragmented.

On a whole I think most programs nowadays should be written in a managed language; the performance gap just doesn't matter for most things.

Re:Common knowledge

[Unoriginal_Nickname]

Boost/0x shared_ptr can't handle circular references.

That's why people focus on memory management when they talk about C++: it's filled with 'gotchas' that will completely fuck you up if you don't completely understand everything you are doing. That's why so many C++ 'fans' hate automatic memory management. It's not because they're attached to the idea of manual memory management, it's because C++ forces you into a culture of paranoia and it's difficult to adjust to a world where the standard library isn't out to kill you.

Re:Common knowledge

[TheRaven64]

Compiler writers in particular know this, which is why even GCC uses GC. Yes, it's home-built application-specific carefully-applied-and-tuned GC, but it's GC nonetheless.

No it isn't, it's just the Boehm GC with half a dozen small patches applied.

Oh, and in a multi-threaded application, free() can be more expensive (in a latency sense) than malloc()

Not really. Recent malloc() implementations use per-thread pools, so free() just returns the memory to the pool (so do most GCs). Multithreading is where you really start to see a big win for GC though. If you're careful with your data structure design, then you can just about use static analysis to work out the lifetime of objects in a single-threaded environment. When you start aliasing them among multiple threads, then it becomes a lot harder. You end up with objects staying around a lot longer than they are actually referenced, just to make sure that they are not freed while one thread still has access to them.

It's also worth noting that most examples of manual memory management outperforming GC are, like most examples of hand-crafted asm outperforming a compiler, are quite small. If you can do whole-program analysis and write special cases for your algorithm, then you can almost certainly outperform general code, whether it's a compiler optimisation or a garbage collector[1]. If you try to do this over an entire program, then you're very lucky - most of us need to deal with libraries, and a lot of us are writing libraries so can't control what the people calling our code will be doing.

As an anecdote, I recently implemented GC support in the GNUstep Objective-C runtime. I tested a few programs with it, and found that memory usage in normal operation dropped by 5-10%, with no noticeable performance change. The code inside the runtime was simplified in a few cases because it now uses the general GC rather than a specialised ad-hoc GC for a few data structures that need to have multiple critical-path readers and occasional writers.

Some high-performance applications (e.g. database servers) have been known to avoid the latency of free() by handing off critical memory blocks to a dedicated thread which just sits in a loop calling free()

This will cause a serious performance hit on modern malloc() implementations. The memory will be allocated from one thread's pool and then returned to another, meaning that almost every allocation in your 'performance critical' thread is going to require acquiring the global allocation lock. I'd be surprised if code using this pattern scaled to more than 4-8 cores.

[1] Although, interestingly, Hans Boehm's team found that per-object allocator pools that most of the C++ advocates in this thread are suggesting have very poor performance compared to even a relatively simple GC. They tend to result in far more dead memory being kept by the program (reducing overall performance, by making swapping more likely) and, if implemented correctly, required much more synchronisation than a per-thread general allocator in multithreaded code.

Sensationalism at its finest

[Toksyuryel]

RTFA and take a good hard look at what they compared it to: Java, Scala, and Go. This post is a complete non-story.

basic

[theheadlessrabbit]

They didn't test BASIC? Lame...

Re:C/C++ faster but produces more bugs

[Erbo]

And, while C++ will always necessarily be faster to execute, there's no question that the other three languages will be faster and more straightforward to develop in. (Which, in general, makes them a net win, as programmer time is almost always more expensive than computer time, except in certain corner cases which should be obvious.)

Why? Three words: Automatic memory management.

No more worrying about whether you've allocated the right buffer size for your data...and maybe allocated too little resulting in an overrun screw, or allocated too much and wasted the memory. And no more forgetting to free that memory afterwards, resulting in a memory leak. You can write expressions like "f(g(x))" without having to worry about how the return value from "g" is going to be freed, allowing a more "natural" coding style for compound expressions.

I maintain that automatic memory management, while not great for code-execution performance, is probably the single biggest boon to developer productivity since the full screen-mode text editor. (Not saying "emacs" or "vi" here. Take your pick.)

Granted: You can retrofit a garbage-collecting memory manager onto C++...but that code will rob your C++ code of some of that enhanced execution performance which is probably the reason why you chose to develop in C++ in the first place.

Re:Environment, conditions and parameters

[Daniel Dvorkin]

I remember that Borland Pascal (in 19991) executed almost 10 times faster than Borland C++ on a consistent basis on the same systems.

Apparently, the reason it executed so fast was because it was reaching 18000 years into the future to run on the computers of the galaxy-spanning civilization built by the hyperspatial god-beings who will be our distant descendants. Man, Borland had some great tech in its day.

Java is fast

[wurp]

In some situations Java will be faster than unoptimized C++ - JIT compilation will do enough of a better job than vanilla C++ to make the difference. In general, C++ will clearly be faster. However, I think what most of the people you're qualifying as idiots get up in arms about (rightly) is the assumption that so many programmers seem to make that Java will be many times slower than C++. That's (usually) just wrong.

In particular, here's what Google's analysis had to say about it on page 9:

Jeremy Manson brought the performance of Java on par with the original C++ implementation

They go further to say that they deliberately chose not to optimize the Java further, but several of the other C++ optimizations would have applied to Java.

For most programming tasks, use the language that produces testable, maintainable code, and which is a good fit for the kind of problem you're solving. If it performs badly (unlikely on modern machines), profile it and optimize the critical sections. If you have to, write the most critical sections in C or assembly.

If you're choosing the language to write your app based on how it performs, you are likely the one making bad technical decisions.

Re:Java is fast

[Splab]

For me it doesn't matter which language is faster, I'm using the one that solves my problem the fastest (e.g. shippable product fastest) and right now, Java is the main player for me.

Also, since our CPUs aren't getting any faster, we need to use languages that makes threading easier the safest way and on that topic, Java is miles ahead of C++. (Java used to have an utterly broken threading model, but these days it works [tm]).

Re:I don't know about that

[mario_grgic]

That's because Java floats are not the same as C floats so compiler can't use the CPU's FPU to do the work for it directly. Java float types make more wider guarantees than C floats (that are usually just restricted to underlying hardware float). This is exactly the reason why Fortran beats C or C++ when it comes to numerical computation. The Fortran compiler can output faster code because language makes some guarantees and it's easier/possible to optimize it better.

Re:I don't know about that

[rjstanford]

Talk to Java heads they'll tell you Java is already faster than C++. They can show you some contrived tests to demonstrate this too!

Take a look at the comments on http://jeremymanson.blogspot.com/2011/06/scala-java-shootout.html about the paper:

Here's one from the top:

The benchmark did not use Java collections effectively. It was generating millions of unnecessary objects per second, and the runtime was swamped by GC overhead. In addition to the collections-related fixes I made (as described in the paper), the benchmark also used a HashMap when it should have just stored a primitive int in Type; it also used a LinkedList where it should have used an ArrayDeque. In fact, after being pressed on the matter by other Googlers (and seeing the response the C++ version got), I did make these changes to the benchmark, which brought the numbers down by another factor of 2-3x. The mail I sent Robert about it got lost in the shuffle (he's a busy guy), but he later said he would update the web site.

Changes which, I might add, are still far easier for the average Java peon-on-the-street to understand than the C++ equivalents. The fact that the paper was comparing one program in C++ that had been optimized to within an inch of its life with another program, in Java, that had had someone spend about an hour "cleaning it up a little," makes for a grossly unfair comparison.

The fact that the "naive" (far more common) programs were all relatively the same speed was insightful.