Slashdot Mirror
Speed Test 2: Comparing C++ Compilers On WIndows
Nerval's Lobster writes "In a previous posting, developer and programmer Jeff Cogswell compared a few C++ compilers on Linux. Now he's going to perform a similar set of tests for Windows. "Like all things Windows, it can get costly doing C++ development in this environment," he writes. "However, there are a couple notable exceptions" such as free and open-source cygwin, mingW, Express Versions of Visual Studio, and Embacadero. He also matched up the Intel C++ Compiler, Microsoft C++ Compiler, and the Embarcadero C++ 6.70 Compiler. He found some interesting things — for example, Intel's compiler is pretty fast, but its annoying habit of occasionally "calling home" to check licensing information kept throwing off the rests. Read on to see how the compilers matched up in his testing."
>> its annoying habit of occasionally "calling home" to check licensing information
Calling home for the latest NSA exploits to inject in to your application? /tinfoil-hat-no-so-paranoid-these-days-dept
I do believe that no one on slashdot cares about this.
Did calling home really throw off the results? Since that is something that ordinary users would have to put up with, I would think it should be part of the test. It might be difficult to get an average, but testing Intel's compiler only when it is at its fastest doesn't seem fair.
does the Intel one still slow down on AMD systems and or trun out code with AMD slow down blocks?
Based on his description, he is using a very synthetic benchmark:
The code I’m testing contains no #include directives, and makes use of only standard C++ code. It starts with one class, and then is followed by 6084 small classes derived from various instantiations of the template classes. (So these 6084 classes are technically not templates themselves.) Then I create 6084 instantiations of the original template class, using each of the 6084 classes. The end result is 6084 different template instantiations. Now, obviously in real life we wouldn’t write like that (at least I hope you don’t).
So in his own words, the code does not reflect realistic compiles. There is no reason to assume that the result generalise to any programs that anyone actually cares about.
Also, there are no error bars of any kind listed.
In other words, I have no reason to assign any meaning to these numbers.
I took a quick took at their website. It looks quite scammy, they only talk about how much you will save, not about how much it will cost.
After clicking through the buy-now buttons twice, I found the C++ version was $4000.
I should use this sig to advertise my book ISBN-13 : 978-1501515132.
My understanding is that they never explicitly 'slowed down' AMD systems; but that the binaries produced by their compiler refused to honor the capabilities flags of non-intel processors (eg. even if an AMD CPU lists 'SSE2, SSE3' among supported instructions, it would get the fallback to non-SSE instructions, while Intel CPUs would get whatever their supported instructions lists specified). No actual 'here be lots of NOPs for no reason'; but x87 on a machine that can do recent SSE is probably enough to achieve the same effect...)
LLVM has got to be dynamically linking and stripped by default. There are switches on the other compilers that will let you do that, and it looks like they're being ignored.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
This doesn't test the speed of generated code. I like to know which compiler produces faster code when looking at benchmarks.
According to the fine article, "The Intel compiler occasionally âoecalls homeâ to an Intel-owned Website to check licensing information. When it does so, it prints out a message about when the current license expires. I didnâ(TM)t use the results when that happens, since it would add time and skew the timing results. " WRONG. The tester should not have excluded these results where time was wasted with this nonsense: If WE the users have to put up with it, it SHOULD be included in the benchmarks.
Sometimes the "writing on the wall" is blood spatter...
My understanding is that they never explicitly 'slowed down' AMD systems; but that the binaries produced by their compiler refused to honor the capabilities flags of non-intel processors
Oh, my. Just how many major non-Intel x86-64 CPU vendors are there? AMD, and...? It's suspiciously similar to the ACPI and SecureBoot affairs, don't you think?
Ezekiel 23:20
Well, our wiki overlords list 15 known CPU IDs; but one of them is intel, one is AMD, one is a VM, and most of the rest are the forlorn epitaphs of the fallen.
I'd just like to see a C++11 compiler for windows.
File under 'M' for 'Manic ranting'
The reason there are these feature flags is that, they can be used to identify the capabilities of the processor. It's not supporting processors that aren't theirs, it's supporting the same features in all processors that support the features.
Benchmarking compilers on how long it takes to compile would be like benchmarking cars based on how long it takes to fill the gas tank.
There are so many things that can affect compile time more than the compiler - and the end customer really doesn't care anyway. Frankly, if you want a 3-5x speedup, just put the whole thing on an SSD and let it fly.
I have mod points and I am not afraid to use them
They seem to be the ones, that bought Borland... ('s division developing development tools, such as C/C++ compilers, and Delphi..) http://en.wikipedia.org/wiki/Embarcadero_Technologies
Also Microsoft's Jim Radigan held a cool presentation in GoingNative 2013 where he reveals some optimization tricks done by the MSVC++ compiler. It also shows some screenshots where Windows is being compiled on a monster multi-core machine.
VIA was also one that was affected by Intels compiler behaviour.
I've worked at places where everything gets built every night to ensure nobody has broken anything.
So, in some cases, you may well compile many many times.
I'm not sure it's how I'd primarily choose a compiler, but in some places it could certainly be a factor.
Lost at C:>. Found at C.
Just use libsimdpp ( https://github.com/p12tic/libsimdpp ) or any of the myriad similar wrappers. With modest time investment you get almost optimal implementation for multiple instruction sets on any compiler you use.
If memory serves, their argument varies (depending on whether the FTC appears interested or not) between 'fuck you, it's the Intel compiler collection, and it'll do what's best for Intel. Go suck an Opteron if you like AMD so much.' and 'Gosh, we sure know about the capabilities flags; but we can't be sure of the details of other vendors'(*cough*shoddy, probably reverse engineered illegally*cough*) implementations of certain complex features, and our customers expect our compiler suite to provide stable, correct output, so reverting to the x87 codepath is our only real option..."
I would argue that your executable returns with expected result nearly instantaneously.
Support the EFF and Creative Commons. The war is coming, and they're supporting you...
Oh, VIA... I'm honestly always a bit surprised to see them still trying.
Back before Intel got (slightly) serious about cheap, with 'atom' and AMD got slightly serious about low-power, with some of their APUs, they made more sense, (in particular, a number of rather interesting x86 embedded specialty boards were VIA based, for situations too low-power or cost constrained for a p3/p4); but lately they've been a much tougher sell. Still some interesting specialty stuff; but 'Unichrome' graphics are such a clusterfuck to deal with that they make AMD look like GPU driver gods, and Intel look (while slow) nearly infallible, and both Intel and AMD have put some rather more aggressive parts into what used to be VIA's playground.
With modest time investment you get almost optimal implementation for multiple instruction sets on any compiler you use.
I'm using ClozureCL and SBCL. I don't think that this is going to work. :-)
Ezekiel 23:20
Dude, lay off the meth.
Sorry, my post was directed to the parent of your post. Somehow I misclicked somewhere and didn't notice.
Doesn't matter, it's still an interesting thing to study. Maybe if I ported and macroified the whole thing for ClozureCL, some good use for me could come from it, too! :-)
Ezekiel 23:20
The Intel compilers do NOT "phone home" for licensing. What they do "phone home" for is to send anonymous usage data. When you install, you're asked if you want to opt in to this - it is not enabled by default. Licensing is done entirely locally for single-user licenses. See http://software.intel.com/en-us/articles/software-improvement-program for more information.
Why would they need to "reverse engineer" features (instruction set extensions) that are already publicly documented for the benefit of compiler writers and assembly language programmers?
Ezekiel 23:20
Long, long ago some review site ran a Via CPU based system while spoofing the CPU ID to appear as an Intel CPU of similar capabilities.
They expected a few percent gain in the FP and INT benches, but oddly got an 8-fold increase in reported memory bandwidth. The other benchmarks appeared to reflect a real increase in memory performance.
Don't wipe your arse with Intel, they're so dirty you'll end up shittier.
A lot.
Typing this in a AND Phemom II black edition which is very fast and not that far off from an i7 back in 2010 when I purchased this. True the newer ones are slower per ghz sadly.
But what if AMD's next chip kicked ass! Remember the Athlon and the later AthlonXPs were the fastest x86 chip you could buy a decade ago?
Tomshardware would include Skyrim and other Intel compiled apps and whine how slow their inferior AMD chips are and intel fan boys would gleam ... but regardless I have a problem with Intel.I do not want to pay more money for a CPU that provides less value.
Even if you do want to argue on this price/performance I get unlocked bios that can support virtualization. Not $900 computer to run VMware natively. Any AMD chipset can run it if you turn it on in the bios!
To answer the grandparent YES INTEL compilers DO CRIPPLE AMD if they do not include SSE3, registers, and other items and use non IEEE standard FPU x87 to make their cpus look better. It is like Nvidia crippling OpenCL to force developers to make CUDA apps who then go on how slow AMD's radeons are etc.
http://saveie6.com/
My understanding is that they never explicitly 'slowed down' AMD systems
You are wrong:
"Overview of CPU dispatching in Intel software"
http://www.agner.org/optimize/blog/read.php?i=49#121
Posting to cancel moderation.
But what if AMD's next chip kicked ass! Remember the Athlon and the later AthlonXPs were the fastest x86 chip you could buy a decade ago?
It could theoretically happen, but the Athlon's success was as much about AMD coming up with a decent architecture as it was Intel simultaneously dropping the ball with the Netburst architecture.
How good would the Intel compiler have to be at optimizing on AMD processors to avoid accusations that they were deliberately slowing things down?
Visual C++ has this handy /MP option which tells the compiler to do multi-threaded compiles. On some of our build machines (with 16 cores) this gives an almost linear increase in build speeds. It's obvious from the author's discussion of multi-core that he is not aware of this option and did not use it.
A performance benchmark which doesn't turn on the go-fast option is not going to produce meaningful results.
The author also doesn't discuss debug symbols. VC++ generates debug symbols by default, whereas the other compilers do not. Generating builds without symbols is not a reasonable scenario for most builds, so this makes the file size comparisons rather meaningless.
That is a form of explicitly slowing down and a rather blatant one. Like if someone decides to 'run' the 100 meter by hopping on one foot.
Naturally, they wouldn't. It's just liars lying to people not technical enough to catch it.
It's not really, but it is their job not to sandbag them. The ICC isn't a freebie you get with the CPU.