Slashdot Mirror
Apple's G5 Speeds Challenged
An anonymous reader was the first of a seemingly infinite stream of people to submit a URL to an argument that makes the case that the G5 isn't quite what Apple wants you to think of it. The evidence? Apple's own press material. Worth a read.
--- Often in error; never in doubt!
I can vouch for those unfortunate enough to have worked in the grocery industry (and have an idea of what that kind of mechandising entail) but this is hardly insightful. It happens on nearly everything that you buy.
When asking the pricing managers (which work for the chain, not an individual store) they replied that there was a study once done, indicating that there is a psychological tendancy to shy away from certain "maker" numbers as being too big. For example, the masses statistically believed that twenty dollars was too much to pay for item x, but for some reason, nineteen ninety-nine was not too much to pay for the same item. Funny thing is that with the same item, eighteen dollars would again be too much, but seventeen ninety-five wouldn't.
Even if the study is flawed or bogus, it is still being taught in the "front-line" marketing schools, (ie. grocery, drug-store, clothing, etc.) , and so I expect we will see nineteen ninety-five for many many years to come.
Lazy-sorry about the AC.
I remember a couple of years back when folks were amazed to find out that Intel had cooked their Spec benchmarks by putting Spec specific code in their compilers! I.e., the compiler recognized Spec code and had special super optimized routines for it. Running Spec against gcc is probably the more honest test. possibly the most honest test of both CPU's in the present environment in whiich gcc has been pretty well hand nursed to give optimum results for both CPUs if you use the correct -xxx options. Result: the Apple benchmarks are probably closer to reality than the Dell benchmarks (although turning off hyperthreading is a bit outrageous).
The author of the article makes the point that most programs use a single processor unless specifically written for using two, so we should downplay the dual processor results. A good point on the surface but examine it more deeply and it has two flaws:
1) This is Apple's Pro machine and many of the users are in the Graphic Arts, Audio and Film industry. The most siginificant programs in these fields do get optimized for the Mac platform.
2) I don't know about you, but it is normal for me to be doing several things at once on my computer. Listening to music, downloading email, munging video, plus about a hundred background tasks. The OS itself balances these separate tasks between the processors, so there is a very real and significant advantage to the dual processor even if the individual programs don't take advantage.
-I have no Sig yet I must scream...
32 bit memory addressing is 4GB not 2GB.
Actually, they *don't* make sense. He compares, for example, numbers generated by Apple to numbers generated by Dell in each of his tests. He uses those numbers against each other; he makes the claim that Apple is boosting their numbers, yet never once takes a look at Dell's numbers (or any of the rest) and the possibility that they may be boosted as well.
In short, I agree with the grandparent. This is a very well-crafted troll.
"Times have not become more violent. They have just become more televised."
-Marilyn Manson
If I'm not mistaken, the limit for physical memory without paging is 4GB. *Windows* has had an arbitrary limit of 2GB for some time.
;-)
(Of course...I could be mistaken
Given a choice between free speech and free beer, most people will take the beer.
hmm.. that image seems to suggest they've tested a dual g5 against a single proc p4...?
Go read Jon "Hannibal" Stokes article about the world of benchmarking., over on his site, Ars Technica.
h tm l
http://arstechnica.com/cpu/2q99/benchmarking-1.
This will give you at least a basis for understanding why benchmarking is used, and what makes or breaks any given set of results. Also, feel free to argue about anything and everything that is said about these benchmarks, since, apparently, everyone of you is in the benchmarking labs day in and day out, testing systems and looking at the results on a scientific level.
I also think benchmark scores are, quite frankly, marketing bullshit. A processor designer can tweak a program and a compiler any number of ways to increase thier scores. The true test would be to use the SPEC benchmark suite with no flags set on the compiles for either platform. That way you are testing just the base processor, with no SIMD instructions, no disabling of the software prefetch algorhythms, no "cheats" as it were. Then test those same systems with every trick in the book thrown in. Then look at the difference. This will probably give you a better picture of the performance you will see in real world activities.
If you have a machine that absulotely sucks donkey when using no "cheats" and then you see this amazing boost in performance when the "cheats" are enabled, you probably are dealing with a highly optimized and specialized instruction set, which can be either very good for specific applications, but absolutely horrible for programmers who don't have access to, or don't bother to research, the abilities of that processor.
These are the benchmarks I'm interested in most. And it'll be at least late September before we see any of that.
Also, while all this is interesting, in an intelllectual sort of way, what about the actual perfomance gains over the current crop of G4's? Why not take a look at the difference between the SPEC scores of the dual 1.42GHz G4 towers, vs. the dual 2GHz G5's? That alone will tell you more about the increase in speed and power that has been delivered. If Apple had been smart, instead of trying to impress and piss off the x86 sparkheads they should have posted those scores as well, to give a real side by side comparrison between the speed and power of the G5 vs the bottlenecked, processor starving, gimp that is the G4. But that would make too much sense, wouldn't it? And you know marketing is all about confusing your consumer into beleiving that the latest and greatest is really what they want, not some old machine from 3 months ago...
Don't Ask Questions. I don't know the answers and even if I did I wouldn't tell you.
With regard to price, if you're after a high-end system, he represents that the high end of the Dell line comes in at $3680, yet rapidly returns to promoting the idea that a $2000 Dell is equivalent. In an effort to configure up an Intel system comparable to the new high-end Apple PowerMac G5, I ran the Dell configurator. It clocks in at $3939:and that's with a lesser video card and a smaller, slower IDE hard drive (add $840 for SCSI, a better comparison with Serial-ATA). I don't think I was being unfair in my selection of components. (OK, add $30 for a USB floppy on the Mac if necessary)
This guy certainly has a point about the non-optimized Intel benchmarks, but he reveals his prejudice by not offering a fair price comparison.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
One of the biggest CAD packages, PRO/Engineer , runs on HP-UX, IRIX, Solaris and Linux.
I passed the Turing test.
that's like saying because they used GIMP instead of Photoshop on the Xeon, the Xeon is at a disadvantage.
SO? The image may turn out the same, but Apple were doing benchmarks using GCC compiler. Until Intel want to provide a compiler for the PPC 970, it's the only way to standardise the test.
The other thing that really shits me about this is that all the same people crying "foul" were the same ones pointing at that Adobe Premiere article not so long back, where the P4 beat the G4. Well, other than the fact the stupid reviewer had enabled the server renderer trick to take advantage of the 2nd CPU on the G4, all it shows is Adobe Premiere performance.
Just like, all this shows is SPECs compiled with GCC.
-- james
Apple used G5 specific optimizations in GCC. They also used a specialized malloc(), which they didn't use for the PC. Also, they disabled SSE2 on the PC. And hyperthreading on the Xeon. And they used specific hardware tweaks on the G5.
(besides, even if GCC isn't poorly optimized for the x86, one could argue that the NAGWare Fortran compiler, used for most of the floating point tests, is.)
The following sentence is true. The preceding sentence was false.
4 GB is for the OS and application TOGETHER. The stock behaviour on Linux and Windows is to give 2 GB to the OS and 2 GB to the application. You can go as high as 3GB to the application Linux, but there are some serious warnings against going even that high.
-sirket
Common name for the bus in 'big-iron' business.
I.E. Nortel's Passport 15k backplane can do something like 60Gb/s throughput (been a while since I looked up spec. Might be more, and I have no idea about the frequency of it)
The main reason for calling it backplane vs a bus, is typically the differrence in the connection types, etc.
Obviously you have never run CAE/CAM programs. (That being Computer Aided Engineering / Computer Aided Manufacturing.. I-DEAS, Pro Engineer, SolidWorks, Catia, etc.) This is the hard core stuff that Boeing, Ford and Toyota use that cost hundreds of thousands of dollars for one license. At my university, the undergrads get to use it on P4 machines and the grad students get to use it on 64 bit HP-UX workstations.
Even if you use a fairly simple FEA (finite element analysis) on something, for example finding the levels of stress in some objects you have modelled when it is bent in different ways, or modelling the flow of water or air through some pipe bends, this amount of RAM is very desireable. Basically the program builds and solves a bunch of 2000x2000 matrices for you. Even a simple one like the pipe bend took something like 3 hours on a P4/512MB and there was a multi-GB swap file needed. I was in the lab very late that night. That is where super-large amounts of RAM are necessary.
I really don't give a shit if Apple fudged their benchmarks on SPEC. I expect as much from them, AMD and Intel. I take those numbers with a grain of salt. However, it's difficult for me to believe that they could have fudged the systems to such an extent that the G5 is twice as fast in Photoshop and Mathematica. I don't care what the benchmark is, you can't have results like that without the chip actually being faster than the competition. At this point, it's just a matter of degree. Maybe the G5 is only 1.8x faster in Photoshop under ideal conditions in both tests; who knows? Do I care? No, because it'll be faster.
... but come on Apple; where's double-precision AltiVec?!). The audio tests were also very telling. While I'm a bit skeptical about the applications not being the same, I think it does say a lot about the audio capabilities of the G5 and what it can do with a scant 25% of its CPU power.
The guy from Wolfram Research made it clear that the G5 outclasses the Pentium 4 in the scientific computing arena to such an extent that it doesn't even compete with it anymore; it competes with high-end UNIX workstations (and beats them, too, apparently
Bottom line, people are starting to try and eek out the edge on Mac vs. PC performance, and that's a good thing. With the G4, that was impossible because the G4 boxes were outclassed by such a huge margin by the x86 ones. Any way you look at it, these machines are competitive. And they run Mac OS X; the Pentium 4 does not. Therefore, I'll be buying the G5 next because I'll get competitive performance with the best OS on the planet.
This guys site is jumping all over Apple about being slower...when using a single processor!
This guys site even says:
"SPECint_base2000 is a single-processor test, so in the following results, where the computer has a second processor, it is either disabled or not used." then goes on to say after the benchmarks using only single processors: "As you can see, the PowerMac G5 is NOT the world's fastest personal computer. In fact, the Dell Dimension 8300 beats the PowerMac G5"
Well a big DUH is in order. Steve Jobs even SAID it was slower. He had a graphic up that showed how the single processor G5 was slower on INT based benchmarks etc etc. It was when they used benchmarks using DUAL processors that it really shined.
Yes, after all this in the article, THEN he goes on to rate the dual processors, but not before he trashes the Mac on something that the Mac had already admited to. I mean, that's pure trolling.
Bottom line, Apple used certain results in all the tests to market the new computer...just like this guy used the same tests to filter out what HE wanted everyone to see.
Also, Apple should never use benchmarks to market anything. No one should. It's too easy for others...no matter what the system to say "well, if you configure blah blah blah with this and compile with blah blah blah you'll see the Commodore 64 is really blah blah blah.
Enough already.
"Music is everybody's possession. It's only publishers who think that people own it." - John Lennon.
For $600 you might be able to score an old iMac.
For $800 you "might" be able to score a brand-new eMac, which will run OS X like a dream, and be able to do anything you can do on your economy PC.
Mod down posts with a "Free Mac Mini/iPod" sig, they're spam!
There is no saying which is right, and I don't think this guy was really trying to. If you read his writeup, he says that Apple claims a certain Dell model benchmarks at value $X, while Dell claims that the same model can do $Y.
He doesn't actually say that one or the other is correct -- he says that the most charitable thing you can do is split the difference and go with the average -- and the kicker is that even that midway point is higher than what Apple claims for the G5.
You've got a good point, but I think this guy is aware of it as much as you are. He's not saying that each vendor's analysis is authoritative, but that the truth probably lies somewhere in the middle, and that middle ground might or might not look to be in Apple's favor (in fact, it doesn't seem to be in Apple's favor).
DO NOT LEAVE IT IS NOT REAL
oops, not HAD enabled it, but HADN'T enabled it. Stoopid me
At work, we just bechmarked the Dell systems a month ago and got very similar results to Apple for the "base" rate. The article seems to be quoting the "peak" rate for the Dells. It's not valid to compare peak rates yet because gcc 3.3 and os 10.3 aren't really fully optimized yet.
The article also complains that using the NAGWare compilers is not a valid test since they're too slow. But I think the NAGWare compiler is a more vallid comparison than intel's compiler because most real-world computing is done with NAGWare because it fully implements the F95 spec and is more portable. In addition NAGWare is well tested for accuracy and it also very much cheaper.
The Dell benchmark numbers are pure fantasy. They never occur in real-world use.
There are 10 types of people in this world, those who can count in binary and those who can't.
I just read the veritest document myself. This guy did not show the complete picture at all. First, in favour of Apple, these tests were run on Mac OS X 10.2.7. I don't think this is the 64-bit kernel that can really take advantage of the G5. I'm not a mac head, but I'm guessing this is Smeagol?
So right from the start the G5 is seriously crippled in these tests. Especially if they don't even take advantage of 64-bit as seems to be the case. Now, on the the other points that the spl dude makes.
Sure the special malloc library seems a bit unfair. But then again, do these tests really focus on memory allocation? I'd think they are limited by CPU power more than memory allocation. In any case, it'd be nice if we saw results without this library.
Now for the comfusing part. The 8300 only has a single cpu. For the base tests, they use hyperthreading and an SMP kernel. They do the exact same test for the 650's base test, hyperthreading with a single processor and an SMP kernel. The G5 system is run with a single processor as well.
So it seems this test is well balanced and fair. The confusing part is the rate tests.
For the 8300 they have no hyperthreading and a uniprocessor kernel. For the 650 they have no hyperthreading and an SMP kernel with two processors. The G5 system is run with two processors. It's unclear why they chose not to use hyperthreading on the rate test. It could be that hyperthreading actually reduced the scores of these tests. I'm no expert on the SPEC tests and hyperthreading, but what I do know is that hyperthreading is an intelligent technology. It can't always increase speed, it depends on what kind of code it's running. In the rate test it's possible that hyperthreading is unable to yeild any improvements, in which case the overhead of enabling hyperthreading may make the scores worse than without hyperthreading.
At anyrate, the tests were a LOT more fair than the dpl guy makes them out. And considering that the G5 could be seriously crippled by not running 64-bit and who knows what other optimisatoins, I'd say that the numbers are still impressive.
My guess is the did use Altivec optimizations. See this section
For the Mac
â Installed theTachyon development environment version 6K452. This provides the appropriate development tools for generating the SPEC binaries and installs Appleâ(TM)s version of the GCC compiler ( version 3.3 build 1379 ) on the test system
For the Dell
â Downloaded GCC version 3.3 ( gcc-3.3.tar.gz ) from http://gcc.gnu.org.
â Followed the documented steps to build and installed GCC v 3.3 on the system.
And here from the appendix
-fast
This flag is used with C and C++ and specifically targeted to the G5 and enables G5 specific instruction usage, tuning and 64 bit arithmetic. In addition to enabling the -O3 optimization level, it also enables the use of C99 aliasing rules and relaxed IEEE math operations.
G5 Specific instruction usage sounds suspicious. I really like the relaxed IEEE math operations.
I also like this part
â Installed a high performance, single threaded malloc library. This library implementation is geared for speed rather than memory efficiency and is single-threaded which makes it unsuitable for many uses. Special provisions are made for very small allocations (less than 4 bytes). This library is accessed through use of the â"lstmalloc flag during program linking.
Doesn't say anywhere that they did the same for the Dell.
I don't think Apple was looking for to even of a field for this test.
Here are some SPEC results I googled for, commisioned from SUN on their Xeon based Fire V65x, running a single 3.06 GHz Xeon. You'll notice that they, too, disabled Hyperthreading. Obviously, Sun would have wanted these benchies to be as fast as possible. So, probably, the single thread used for SPEC scores is best suited by TURNING HYPERTHREADING OFF.
s 20 03q2/cpu2000-20030520-02193.pdf
Meaning, if Apple's results are reliable (which I think they are...levelling both machines by optimizing them for neutral operations and having them run neutral code), they tuned the Dell FOR SPEC. They didn't decrease its performance -- they probably increased it a bit.
http://www.specbench.org/osg/cpu2000/results/re
Just because you put the words "Fast" or "Hyper" in front of a chip's feature doesn't automatically make it faster, as any BIOS hacker knows.
Hey freaks: now you're ju
It's interesting that many people are critical of the benchmark results Apple trumpets. Certainly there are many that can address that topic. Depreciating the numbers, however, doesn't address the more interesting topic, which is:
How do you like the rest of the motherboard?
1 Gig frontside bus X2, PCI-X X3, Serial ATA X2 with separate channels for each drive, USB 2.0/FW 800 & 400, AGP 8X, DMA for every I/O function without bandwidth contention, etc.
Seems to me this may be the most advanced motherboard ever put in any desktop, affordable computer.
And even if you don't believe the SPEC stuff, how about the software demos? Just lies, I suppose.
If y'all had taken the system diagram Apple is showing and substituted 2 Pentium 4's or 2 XEONs for the 2 970's, it would be touted as an Alienware-buster and proof that Wintel is King.
Better check your bubbles for bursting.
As the author of a "hate mail" (starting "Flamebait- My question is: Why is MacNN giving any attention to this?"), I can verify that my post was taken from MacNN and not mailed to the author. It was meant as a critique of the article, not an expression of hate to the author as it is presented.
/. (and that's saying a lot) is picking apart these numbers, and that a detailed analysis is poking through the posts. But I'm saddened that they are buried amongst a landfill of posts from passionate, persuasive people spending their neural energy on "mac fanatics" and "deceptive marketing."
So let me take a moment to reiterate the original point.
I am, like any Mac user, deeply concerned when somebody claims that Apple is using false numbers. The author is welcome to his or her opinion, but I found the claim - that the Veritest numbers are false - was never substantiated: they're as true as any other benchmark. So, no biggie.
But it it the tone of the article that got to me. Claims like, "Apple is attempting to deliberately mislead," and "Apple cheated" and "a significant percentage of [Mac users] are crazy fanatics" have no place in a technical discussion of benchmarks, and undermine the author's believability. All authors have a point of view, but bias is another animal altogether. Authors need to be open-minded to be believable, and this author's use of hyperbole and emotional phrases betrays a certain zeal. Despite what may have been the author's best efforts, the article is not a level-headed, rational discussion about benchmarks. It is a fanatical rant.
And, hey, I'm all for fanatical rants. Not only do I enjoy them, but I am the source of many. My objection, though, was to the editors of the Mac News Network (MacNN) for posting this article, unqualified, as news. It is not news. News informs, and a fanatical rant actually does the opposite: it polarizes. People take a side and stick with it, regardless of facts. The speed of the new G5s is a very very important issue, and this article is a step backwards in understanding these highly complex comparisons.
So, needless to say, I find it *highly* telling that my editorial objection was co-opted by the author as "hate mail." And the response to it just further underscores my point that this is not a rational investigation, but a crusade.
I'd also like to note that, for whatever reason, MacNN has since removed the news article from their site.
I'm glad that the comparative level-headedness of
In reality, icc produces really good code for Athlon/Opteron. So good in fact that the SpecInt scores of Opteron with icc in 32-bit-mode are better than the scores with gcc in 64-bit-mode. -- Despite the extra 8 registers in 64-bit-mode.
Finally! A year of moderation! Ready for 2019?