Slashdot Mirror
Apple Hardware VP Defends Benchmarks
He said Veritest used gcc for both platforms, instead of Intel's compiler, simply because the benchmarks measure two things at the same time: compiler, and hardware. To test the hardware alone, you must normalize the compiler out of the equation -- using the same version and similar settings -- and, if anything, Joswiak said, gcc has been available on the Intel platform for a lot longer and is more optimized for Intel than for PowerPC.
He conceded readily that the Dell numbers would be higher with the Intel compiler, but that the Apple numbers could be higher with a different compiler too.
Joswiak added that in the Intel modifications for the tests, they chose the option that provided higher scores for the Intel machine, not lower. The scores were higher under Linux than under Windows, and in the rate test, the scores were higher with hyperthreading disabled than enabled. He also said they would be happy to do the tests on Windows and with hyperthreading enabled, if people wanted it, as it would only make the G5 look better.
In the G5 modifications, they were made because shipping systems will have those options available. For example, memory read bypass was turned on, for even though it is not on by default in the tested prototypes, it will be on by default for the shipping systems. Software-based prefetching was turned off and a high-performance malloc was used because those options will be available on the shipping systems (Joswiak did not know whether this malloc, which is faster but less memory efficient, will be the default in the shipping systems).
As to not using SSE2, Joswiak said they enabled the correct flags for it, as documented on the gcc web site, so that SSE2 was enabled (the Veritest report lists the options used for each test, which appears to include the appropriate flags).
http://www.osnews.com/story.php?news_id=3877
the article is analyzing if the recent announcements from Apple were innovation or simple catch up.
Eh, we do this sometimes, when it is appropriate. In this case, I have a PR contact at Apple who asked me last week if I wanted to talk to someone about WWDC, and we set up a call last weekend, for this afternoon. It just happened to coincide with the benchmark discussion, which Greg was eager to set straight (he had read the arguments and already compiled his responses :-). We also talked a bit about some other topics, but nothing of interest that you haven't read elsewhere.
I'm a science guy, and for the calculations and simulations done here at the physics dept. where I work, the IBM power4 kills just about everything else. And when I saw the powermac calculate fractals with mathematica faster than the xeon box by more than a factor of 2, I was very excited (although a little cautiously) to see we will soon get power4 performance for well under $20,000
this is the funniest claim i've seen in a while. not only does apple do this, so does dell, and so does virtually every consumer-oriented company on the planet. gas companies shave a TENTH of a PENNY off gas prices to make them seem cheaper.
a department store (was it macy's?) started this practice. the funny part is that the aim wasn't really to fool consumers into thinking it was less expensive. alas, the real purpose was to force cashiers to open the register, since the customer was almost always going to be due some change.
woof!
SPECfp: The Power4+ at 1.7 Ghz has the highest SPECfp score (1699 @ 1.7Ghz); higher than Itanium (1431 @ 1Ghz), the most recent Alpha (1482 @ 1.15Ghz), and the Pentium 4 (1229 @ 3.0Ghz).
SPECint: As far as SPECint, the Power4 is not in the lead (1113 @ 1.7Ghz), but is still respectable when compared to Pentium4's (1200 @ 3.0Ghz).
The G5/970 should do similarly or better than the G5/970 (since the G5/970 is running at 2.0Ghz vs Power4+ 1.7Ghz). One caveat is that the G5/970 has a smaller on-chip second-level cache (512kB vs 1.5MB), which will hurt its performance on some codes.
Certainly Apple's test uses a drastically different compiler than the reported SPEC results. This results in absolute numbers that are lower, but Apple's relative comparison is still reasonable, IMHO. I think it is safe to claim that Apple has really closed the gap in processor speed and now has processors with comparable performance to the fastest chips money can buy. About damn time. :)
you know this and I know this but many trolls don't know this. I think Apple just got tired of hearing how PCs are faster and what not. Personally I was blown away by the keynote. Also, for anyone wondering I'm using the developer preview now and if the release of Panther is anything like the preview, holy crap. It is nice. There are a ton of tiny improvements here and there that really make it nice, even nicer than Jaguar. These are little things that weren't mentioned in the keynote.
-
Actually, I just watched the video again. He actually said:
--
The internet is the greatest source of biased information in the history of mankind.
Overpriced is not the right word. More like Underpriced.
I urge anyone to compare the featureset of Final Cut Pro 4 ($899) vs. similar solutions in the PC world. Avid Xpress DV doesn't even stack up, and with all the plugins and tools, you'll end up spending far more to equal twice the price of the Apple G5 hardware.
It really amuses me when people talk about 10.x updates as if they are service packs. Someone yesterday mentioned this saying "Microsoft doesn't charge us for SP.x upgrades", which was really comedic. Windows ServicePacks just fix broken stuff, and sometimes even break more. With OSX 10.x updates you get brand new features all the time.
I wish people really understood how this shit worked.
Man are you way behind the times. I can do that even with my dual 1Ghz G4.
The Apple Quake 3 benchmarks disabled hardware acceleration. They were solely testing the CPU, or trying to at least. The guy from... oh, damn, what's that guy's name, the guy who did the OpenGL demo yesterday. He did the same thing. They did all the rendering they could in the CPU.
That's why the numbers were low compared to other tests that used accelerated graphics.
The G5 would have the same problem if it was working on a dataset that was 1.5x the size of its physical memory.
MJC
The 30th International Symposium on Computer Architecture had an interesting panel discussion on benchmarking in industry and academia, with people like John Hennessy, Dave Patterson and Gurinder Sohi on stage. The conclusions: most benchmarking in industry, especially SPEC, is a pack of lies. And benchmark results published by academic researchers aren't much better. So, not really much point in losing a lot of sleep at least over their SPEC numbers.
I just got a 450MHz G4 Cube (pre-owned, obviously).
I have used high-end workstation-class machines, both RISC and CISC, multi-GHz Intel machines, and Macs back to System 6. This Cube is without a doubt the best computer I have ever owned or used.
That having been said, I have seen Apple make some prety serious hardware and customer service mistakes. I would buy another Mac in a heartbeat, but I would wait for these systems to ship for at least six months before buying one of them. Wait until you can check Mac help forums. Find out what the problems are, if any. You don't want to spend $3000 on a computer, and have the paint chip off.
If you fall off a building, go real limp, because maybe you'll look like a dummy and people will be like hey, free dummy
I hate it when people ask silly questions without reading the first thing about story. Here is the quote to save you from scrolling back to the beginning: "Joswiak added that in the Intel modifications for the tests, they chose the option that provided higher scores for the Intel machine, not lower. The scores were higher under Linux than under Windows, and in the rate test, the scores were higher with hyperthreading disabled than enabled. He also said they would be happy to do the tests on Windows and with hyperthreading enabled, if people wanted it, as it would only make the G5 look better."
Interesting, when I took a course in Optimizing Compilers last year the concensus was that GCC is pretty awful when it comes to optimizations. Even general non-architecture dependent optimizations. The lecturers reason behind it was twofold.
First most research on compilers are being done at big corprorations. IBM being the single largest as I understood it. Naturally they put their optimizations in their own compilers first, the rest of the world have to implement them from their papers. (If they are lucky and the algorithms are not patented.)
Second if you were to put a good optimization in GCC it wouldn't take long before all other compilers had that optimization as well. GCC is OSS afterall.
We did comparisons between GCC and SunCC on UltraSPARC. SunCC minimal optimizations (O1) beat GCC with maximum optimizations (O4).
I'm just finished a course on vectorizing/parallelising compilers. There the situation is that even the best commercial compilers are pretty much equivalent to junk. Implementing the vector algorithms is a lot harder though. Even compared to complex SSA-form optimizations.
On the research level gcc is not as bleeding edge as other compilers. So if you run example code that shows the merits of a particular optimization, gcc may look not so good. But in practice, it's quite good.
My experiences with UltraSPARC are also a few years old, but gcc was faster and produced better code than Sun CC back then. You have to make sure to set -march=ultrasparc, of course. And I'm not sure about UltraSPARC but normally gcc -O4 does not do more than -O3, which basically is -O2 with function inlining. You can also get some boost with profile based optimization with gcc.
In summary, gcc produces very good code, but you might have to use some little known options for it. For example, gcc on Athlon XP and Pentium >= 3 may gain significant floating point performance with -mfpmath=sse,387 (I got >10% speed-up on lame, gcc's code was even faster than icc's with vectorizer). Another option worth knowing is -malign-double and the regparm attribute.
Another thing you have to keep in mind is: recent optimization advances normally are not big breakthroughs but small incremental advances. Many of them only help in a handful of special cases. gcc 3 has many more optimizations than gcc 2.95.3 and they were so proud of it that they said "much faster code on x86", and then there was whining and gnashing of teeth when most software was unaffected or even slower.
The only platform where I really would prefer the vendor cc is HP-UX on PA-RISC. The HP CC consistently produced 10-30% faster code than gcc (although that may have changed, I haven't used gcc > 2.7 on HP-UX).
Admittedly, this was when the PowerPC was pretty new, and the choices were the IBM/AIX compiler which was robust and produced fast code but required an AIX box in addition to a Power Mac, or the nacent Metrowerks CodeWarrior compiler which run natively on the Power Mac, but generated poorly optimized code.
If I recall my history timeline correctly, after CodeWarrior came
- the Apple MPW "MrC" compiler (better code than CodeWarrior 1.0, but with a wacky command-line "IDE"), then
- gcc for PowerPC (cruddy code back then), then
- the Motorola PowerPC compiler (better code than Apple's compiler, with NO IDE - it plugged into the CodeWarrior or MPW IDE).
- Then Motorola inexplicably stopped selling their compiler.
- Later Motorola bought Metrowerks.
- Somewhere along the line, gcc learned to generate better PowerPC code.
- Eventually, Apple pretty much shelved their "MrC" compiler, and settled on using gcc for Mac OS X
- Monday, Apple released their "Xcode" environment -- still using gcc, I believe.
Apple's MPW tools are still available (free) here for Mac OS 7/8/9. The new Mac OS X tools including Xcode are available here.As a side note, it's really nice to see Apple giving away a full development suite for free, and continuing to put development time and effort into improving it.
-Mark
DELL's own comments on SPEC benchmarks and turning off hyperthreading for best results:
0 2- khalid.htm
http://www.dell.com/us/en/biz/topics/power_ps3q
I'm at WWDC right now and posting this comment from Safari running on a G5. I don't care what any of the benchmarks say -- this machine screams from a user's point-of-view.
No matter what I throw at it, I can't get either one of the CPUs above 50%.