Benchmark Program Rewritten to Favor Intel?

BrookHarty writes "Interesting article over at Van's Hardware, that BAPCo the maker of the SysMark benchmarking program, has re-written its SysMark 2002 benchmark program in favor of Intels P4. AMD joined BAPCo in order to "correct" these "broken" results. AMD reports that BAPCo's SysMark 2002 (written by Intel Engineers) is a collection of tasks to summarize "Real World" performance. Interestingly, these tasks are selected for Intel's favored performance, while removing certain tasks that favor AMD. Vans Hardware has additional information on BAPCo's Shady history."

Comments

Clang!

[MaxVlast]

That sucks. If you can't have the fastest processor, have the fastest benchmark program!

Re:Clang!

[Anonymous Coward]

Is anyone reminded of the way that Apple would get so obsessive about just the behaviour of certain tasks in certain applications for their performance tests.

"bullshitian" blur in iDrawPrettyPictures

Or something. Whatever it was Macs and PPCs are very very good at it, even if not at anything else.

Re:Clang!

[Anonymous Coward]

What did you think? Its zionists, its intel!

Re:Clang!

[Anonymous Coward]

point 1.

amd is, mhz for mhz, dollar for dollar, pound for pound (the currency & weight ;-)

is THE FASTEST x86 cpu out there.

If I give you $300 and tell you to go buy the fastest x86 cpu you can for the money....it's an AMD cpu... period.

point 2. i own 2 amd based systems, and about 7 intel based systems. i'm not biased, i have likes /dislikes for both companies...but CURRENTLY, i'm going to recommend amd 90% of the time. What most Intel FANS (read: biased) DON'T UNDERSTAND, is while they are bashing AMD, if it were not for AMD they would be paying a lot more for their precious little Intel processors. Very likely around $800-$900 for a dinky 1 Gigahertz P3 right about now. How can you read slashdot, and know how important competition is and still be an Intel cheerleader? They must be ignorant or stupid.

Point 3. it's simple. a competitor like AMD, simply means that if you prefer AMD, you are getting a damn good processor with a LOT of value. If you prefer Intel, while not quite the value of AMD,you are still getting a very good cpu, with good value. Because of AMD, we the consumers are enjoying greater value.

Without AMD, I can guarantee you that the current situation would not exist.

Intel would definitely be the equivalent of Microsoft....a monopoly. Trying to control all aspects of the industry, charging rediculous prices, trying to build their own 40 billion in liquidity.

Re:Clang!

[Anonymous Coward]

Did anyone bother to read in Tom's hardware about it?

Tom's hardware

Big deal

[ObviousGuy]

The P4 has that long pipeline that allows it to speed up quite a bit, so long as the branch prediction doesn't blow it. As compilers become tuned to exploit this, it's plausible that the Athlon's performance is going to lag quite a bit more than it already does. That there is some benchmark out there that is specifically designed to show off this strength of the P4 is no real surprise to anyone, is it?

Heck, this is what Sun was doing a few years back. It's almost an industry standard to have your own "benchmark lab" on the payroll.

Besides, AMD has always been the value chip company. You can't expect them to keep up with Intel forever.

Re:Big deal

[GigsVT]

I don't think there is much motivation on the part of compiler writers to optimize for this particular implementation of the x86-32 ISA. This isn't like previous chips, where new cache handling opcodes were added, which compilers could use if available. I've talked to people much better versed in compiler writing than myself, and they all seem to agree, when it comes to "optimizing for P4", their answer is going to be "don't hold your breath".

Re:Big deal

[chicoy]

If this is the most widely used benchmark, it sounds like a big deal to me.

If someone says that they are independent, and they are not, then this is a big deal.

Even if this is something that is expected because it is 'an industry standard', it doesn't make it right.

Let's make a big deal out of this so that people that don't have the same expectations do not fall for these bs results.

Re:Big deal

[njdj]

it's plausible that the Athlon's performance is going to lag quite a bit more than it already does.

It doesn't. The fastest X86 processor is currently made by AMD. See a non-BAPCo performance comparison.

AMD has always been the value chip company. You can't expect them to keep up with Intel forever

Are you an Intel employee? Intel isn't keeping up with AMD. The P4 is underperforming, as well as overpriced. Take a look at the web page referenced in the lead story.

Re:Big deal

[ergo98]

Intel has used the "once compilers catch up..." scam for years, and every time people find themselves with a long obsolete processor by the time the software the theoretically exploits it arrives.

My general practice is to ignore any synthetic benchmarks because they represent no real world value whatsoever: Instead I look to application benchmarks, like compressing divx movies or rendering 3D scenes, if that was the use that I had in plan for my PC.

Re:Big deal

[l33t-gu3lph1t3]

For a year and a half AMD managed to blow Intel out of the water in the performance arena, from the time they both reached 1.0GHz with the P3 and Athlon, to the time Intel started releasing Northwood core P4s with the extra 256KB L2 cache. The longer pipeline of the P4 has nothing to do with performance. All it does is enable the processor to ramp in clock frequencies easily. A general rule of thumb is the longer the pipeline, the lower the IPC (instructions per clock/cycle) and the LOWER the actual number-crunching performance.

Re:Big deal

[Anonymous Coward]

So you would trust a benchmark from a webpage that is in the midst of bashing another benchmark? A little biased, aren't we...

Re:Big deal

[Ninja Programmer]

You are just regurgitating the Intel corporate line, Mr. ObviousGuy.

Compilers *could* take advantage of P4's long pipeline to exclusion, or they could take advantage of AMD's *WIDE* pipeline. Microsoft (the leading compiler vendor for x86) has chosen to tagret *BOTH* platforms.

If you view AMD as solely a *value chip* company you obviously have had your head in the sand for the last 3 years. AMD has been in a neck and neck race, where it has been mostly ahead of Intel ever since the release of the Athlon.

Re:Big deal

[Ledskof]

I trust it more than I trust a benchmark company with a history of getting kickbacks.

Re:Big deal

[fmaxwell]

Nice try, but the link in question reveals a page which does not even have the name AMD anywhere on it. What it says is:

Van's Hardware Journal is a site dedicated to providing honest, accurate, hard-hitting, and thorough reviews and analysis of the latest in computer hardware and related high technology.

Re:Big deal

[Anonymous Coward]

The rest of the site has nothing but Intel bashing and AMD dick-licking. THG it ain't.

Re:Big deal

[fmaxwell]

So you would trust a benchmark from a webpage that is in the midst of bashing another benchmark? A little biased, aren't we...

"We" aren't, but apparently you are. When a web page takes a critical look at benchmarks and exposes those that are biased, then I would tend to trust that site to choose an appropriate, unbiased benchmark for CPU comparisons.

Re:Big deal

[Dr. Spork]

I'm sorry, but I feel like I have to point out that your statement "Athlon's performance is going to lag quite a bit more than it already does" seems to imply that you actually think the P4 has the performance lead. You must not be reading slashdot much (check out 2600+ benchmarks and weep).

Also, the story didn't imply this was a big deal. It only remids us of all the dirty tricks Intel is forced to resort to when they try to maintain a market lead with a grossly inferior product. As long as people know this, benchmark-cooking is really no big issue.

Re:Big deal

[Anonymous Coward]

Yes, that's why the press release from AMD is so important in establishing fairness...

Sheesh, open your eyes!

Re:Big deal

[Anonymous Coward]

Yes, Tom's Hardware Guide.

"Look how this Intel chip with a good motherboard that shuts off if it detects heat does not overheat! Now look at this AMD chip plugged directly into a wall socket. See the smoke? That is because AMD is inferior."

Good example of a non-biased site.

Re:Big deal

[C0LDFusion]

Even if this is something that is expected because it is 'an industry standard', it doesn't make it right.

That's exactly right. Phony numbers in accounting books have been standard in alot of businesses in the past decade. See where it's getting us.

But the answer isn't to bust a couple of guys who do it, but to eliminate the mindset that fosters this behavior. Companies that think they can put out fake benchmarks to make themselves look better are no better than the companies that say their earnings were bigger than they actually were.

Re:Big deal

[Anonymous Coward]

My general practice is to ignore any synthetic benchmarks because they represent no real world value whatsoever: Instead I look to application benchmarks, like compressing divx movies or rendering 3D scenes, if that was the use that I had in plan for my PC.

of course these are the first things optimized for specific processors....

Re:Big deal

[Sivar]

Besides, AMD has always been the value chip company. You can't expect them to keep up with Intel forever.

AMD has had a superior (in design) processor architecture to Intel since the K6 was released (though the K6 had mediocre FPU performance, the design was still more elegant--ask any x86 assembly programmer). The Athlon has given the P2, P3, AND P4 a run for its money, and early benchmarks of the hammer would seem to indicate that the expensive Itanium 2, which almost nobody actually uses, is going to be outrun as well.
The Pentium IV's really looong pipeline does allow the P4 to run at higher clockspeeds, but the branch prediction you mentioned is instant death. Branch mispredictions happen VERY frequently in any CPU (note the K6 had the most sophisticated branch prediction unit up until the "XP" series of Athlons) but with the Pentium IV, a single branch prediction requires up to 20 full clock cycles of work to be discarded.
The Pentium IV has other questionable design desisions that hurt performance as well. It has 8K of L1 cache, the same amount found in the ancient 486 processor, whereas the Athlon has that amount squared and doubled (128K). Current P4's have more L2 cache, but L2 cache is less important and slower. (Note though that the P4's L2 cache is particularly fast L2 cache)
The P4 has buffers to remember a series of decoded x86 instructions so that it does not have to decode them again--these are almost required because of the terribly long pipeline--but it doesn't have enough to speed things up in server environments. Most servers execute a wide variety of instructions such that the buffered instructions get very little use before being replaced by new instructions. This is even more a problem on systems that run many different applications at once, but this problem can be demonstrated just with DB servers (which use plenty of instructions) as the P4 tends to not scale as well as the Athlon MP when a second or third task is added (such as mail serving, web serving, etc.)

One dissapointment that I had with the Athlon is that AMD never used the excellent EV6 bus to its fullest. Athlons are superior in multiprocessor capabilities because different processors needn't share access to the memory bus. On Intel SMP setups, even on P4 Xeons (Which, IMO, are inferior to P3 Tualatin chips by the same company) when one CPU accesses main memory, it locks main memory for the other CPUs. All other CPUs have to sit and twiddle their transistors while the main memory is on use by only one CPU.
On AMD SMP setups, ALL processors can simultaneous access memory, merely sharing the bandwidth simultaneously. So, if one CPU is only using 100MB of memory bandwidth, the rest can be used by other CPUs at that time.
Unfortunately, this doesn't really matter much with only two CPUs, which is the largest AMD configuration you can get. You can, of course, see it in action with 8+ CPUs on EV6 Alpha setups (AMD licensed the bus from DEC's Alpha team) but Alpha setups are expensive as hell and are a dying breed.
If AMD had created a quad or 8-way setup, we would see the true power of a good design.

Fortunately, the Hammer has an even better design (one made by AMD no less) on an even better CPU. I fully expect the Hammer series to wipe the floor with all Xeons and possibly the Itanium 2 because of its design. An integrated memory controller that will tremendously drop memory latency, twice as many general-purpose registers of twice the size (Much less pushing and popping, for those that know some assembly) and, unlike the big vendor 64-bit processors, the ability to split half of the general purpose registers into chunks of 16 and 32 bits when huge numbers (2^64) are not needed. (On an Alpha/SPARC/R12000, if you want to store the number "42" you must use all of a register that can hold values up to 18,446,744,073,709,551,615. A bit wasteful)

Re:Big deal

[VAXman]

I don't think there is much motivation on the part of compiler writers to optimize for this particular implementation of the x86-32 ISA.

Over 80% of new computers being shipped are P4 based. Therefore, applications will be optimized for P4. Not optimizing your application for P4 would be look a hardware vendor not releasing a Windows driver for their device.

Re:Big deal

[VAXman]

The Pentium IV has other questionable design desisions that hurt performance as well. It has 8K of L1 cache, the same amount found in the ancient 486 processor, whereas the Athlon has that amount squared and doubled (128K).

Obviously you flunked your freshman-level computer architecture course. The P4 8K L1's 2-cycle load-use latency is 50% better than Athlon 128k L1's 3-cycle load-use latency (not even accounting for P4's clock speed advantage). The difference in hit rate between 8k and 128k is only about 5% meaning that it is substantially faster to go with the small/fast cache than the big/slow cache. Do the math - even an infinitely large 3-cycle load-use cache is slower than an 8k 2-cycle load-use cache.

Cache size comparisons are more meaningless than megahertz comparisons. Whenever somebody tries to justify a big cache size without looking at performance, just walk away. AMD is playing marketing games with their slow-as-molasses (but massive) L1 cache.

I won't bother to address the rest of the technical errors in your post...

Re:Big deal

[ergo98]

If they optimize the processor to do really well in Lightwave, for instance, then the application benchmarks will reveal the benefits in a very obvious, and easily duplicated, mannre. For people who need the power for their Lightwave processing needs, then that is absolutely completely suited for them. Synthetic benchmarks, on the other hand, often have a tenuous correlation with real world applications, and are numbers that are largely abstract to most purchasers.

Re:Big deal

[Sivar]

Obviously you flunked your freshman-level computer architecture course. The P4 8K L1's 2-cycle load-use latency is 50% better than Athlon 128k L1's 3-cycle load-use latency (not even accounting for P4's clock speed advantage).Obviously you are imagining things, as I never said that was not the case. Latency is important, but it doesn't matter if the cache size isn't large enough to fit enough code in to enjoy the low latency.
The difference in hit rate between 8k and 128k is only about 5% meaning that it is substantially faster to go with the small/fast cache than the big/slow cache.
Really? That's interesting, and here's me wondering why both AMD and, other than in the P4, Intel have wasted so much money adding more cache memory.

Because you seem to be such an expert, so why don't you go ahead and list a few common programs for me that have a working set of less than 8K--the size that will fit into the tiny L1 cache. Can't find any? Gee, I guess that makes the size of the cache pretty important then. When a program's working set has to be swapped in and out between L1 and L2 cache, suddenly that latency doesn't much matter. Of course, you may feel free to prove to me that the P4 can run addition loops faster. Those will fit into about 8k.

Do the math - even an infinitely large 3-cycle load-use cache is slower than an 8k 2-cycle load-use cache.
Who was it again flunked their freshman computer architecture course? You're saying that if the Athlon had 512MB of L1 cache that the system would be slower than the P4 and it's 8K of lower latency cache?
What math is it that I should do? Do you know what the working set of a program is?
Having a tiny amount of cache is analogous to having a tiny amount of RAM. Put 32MB of low-latency RAM in your system. Overclock some DDR SDRAM to 200MHz (AKA "400MHz" by people that don't understand clock speeds) and set it to CAS2. Tell me how your system performs. Just as your system will have to swap just about all running code to disk, the Pentium IV will not be able to contain the core loops of the various running programs in L1 cache. The vast majority will have to be dropped to L2, which is significantly slower and higher latency, kinda defeating the purpose of that 8k of fast memory, no?
Working sets that cannot be fit into the P4's 256k or 512k or L2 will then be relegated to main memory and moved to L2 then L1 when the data is executed, and anything that won't fit in main memory (very rarely which includes the working set of a program) will be swapped to disk if the platform supports virtualizing memory.

In closing, your comment was surprisingly brash and conceited, not to mention rude and totally innacurate. Thankyou.

Re:Big deal

[Anonymous Coward]

hahahaha, way to rub it back in

Re:Big deal

[jsse]

The longer pipeline of the P4 has nothing to do with performance. All it does is enable the processor to ramp in clock frequencies easily. A general rule of thumb is the longer the pipeline, the lower the IPC (instructions per clock/cycle) and the LOWER the actual number-crunching performance.

In fact longer pipeline also increases the loss of processing cycles at time of pipeline-flush (when branching, interrupt, etc.). That explain why PIII outperformed the first release of Pentium 4 in some benchmark test.

Re:Big deal

[Hoser McMoose]

Over 80% of new PCs sold next year might contain P4-based cores. At the moment it's only slightly over 50% of all new PCs sold (the Athlon sells up around 20%, the PIII still gets a fair number of sales (10%?), and Celeron chips based on the PIII core are still quite common as well (20%?).

Still, when compared to the total number of PCs out there, the ~200 million P4-based PCs that will be on the market by the end of next year isn't all that big of a number. There are still a LOT of people out there using PIII, PII and Pentium systems, as well as their Celeron derivatives of such chips, not to mention all the Athlons and K6s, as well as the Hammer chips that will be out next year as well.

Long story short, there are a LOT of x86 architectures out there, and they ALL have different optimizations requirements. Since virtually no one hand tunes their own applications for a specific chip (which would require getting down to the assembly level for the most part), it's up to the compilers to optimize for a specific chip, and generally speaking, it's a LOT more effective and a lot easier to just do general optimizations rather then core-specific optimizations. General optimizations will benefit ALL the x86 chips out there, while core-specific optimizations will only benefit a small proportion of the chips for any given time frame, after which point a new chip will come out to replace that core.

Re:Big deal

[VAXman]

According to Hennesy & Patterson, 2nd Edition, page 391, the total miss rate (for SPEC92) of a 8k 4-way set associative cache (like the P4's) is 2.9%. The miss rate of a 128k 4-way set associative cache (like Athlon's) is 0.6%.

The hit time for P4 is 2 cycles, and for Athlon it's 3 cycles. The L2 hit / L1 miss is ~10 cycles for both. Everything further out is approximately the same so we can ignore it for simplicity.

So, the average memory access time for P4 is (0.971 * 2) + (0.029 * 10) = 2.2 Cycles. The average memory access time for Athlon is (0.994 * 3) + (0.006 * 10) = a little over 3 cycles.

Suppose Athlon had an infinite size L1 cache (or 512 MB if you like to use numbers). The highest hit rate it could ever achieve is 100% (actually slightly less, since you cannot eliminate complulsory misses). The average memory access time would then be 3 cycles - which is higher than P4's 2.2 cycles!

BTW, Paul DeMone wrote a pretty good article about P4's L1 cache.

Re:Big deal

[Colin Bayer]

It has 8K of L1 cache, the same amount found in the ancient 486 processor, whereas the Athlon has that amount squared and doubled (128K).

8,192 ^ 2 = 67,108,864 * 2 = 134,217,728.

Close enough.

Re:Big deal

[Hoser McMoose]

FWIW, last I checked, Van actually works for VIA, in the division that designs the VIA/Cyrix processors (ie the people that VIA bought from IDT when they acquired the IDT Centaur Winchip design team).

He's no fan of Intel, though I don't know that he's exactly AMD's bitch either.

Re:Big deal

[Anonymous Coward]

You're talking about a 5% change in the hit-rate. (otherwise your wrong). Going from a 95% hit-rate to a 90% hit-rate means your miss-rate goes from 5% to 10%. That's a +100% change in the miss-rate.

This means you're accessing L2 twice as often (with a penalty of SEVERAL cycles).

If you want numbers to back this up...
Here's some numbers I recently recieved with the spec2000 integer benchmarks for a direct-mapped L1 data cache. I now wish I would've run these out to 128K... oh well. (Higher associative caches will give the same general trend)

Direct Mapped Cache:
4k 8k 16k 32k
bzip2 0.0773 0.0601 0.0461 0.0392
gcc 0.1021 0.0846 0.0725 0.0662
gzip 0.0739 0.0608 0.0481 0.0392
parser 0.1188 0.0787 0.0562 0.0446
vpr 0.1360 0.0894 0.0741 0.0584

Going from 8k to 32k in these cases gives you about 40% less L2 accesses. Thats a HUGE difference when your talking about hundreds of millions of memory references. (That many accesses are easy with today's hardware. These numbers alone are from 500M references near the middle of the programs)

Granted, the lower latency of the P4 L1 cache's are nice, but the balance between low latency and hit-rate is a ticky one. Too much in any one direction and your runtimes turn to crap.

(sigh, now if only I could fine my /. login =)
-lib

Re:Big deal

[cpeterso]

Yes, the hit rate for an Athlon is 3 cycles, but its cache hit rate is 110%! now that true high performance! ;-)

Re:Big deal

[kryps]

You are correct. But I think the original poster was just trying to make a statement about the instruction cache.The P4 has 12k micro-op trace cache (roughly equivalent to a 20KB L1 instruction cache) but the Athlon has a 64KB L1 instruction cache which provides space for much larger code fragments/algorithms. It is very well possible that the larger instruction cache combined with better branch prediction accounts for a good amount of the performance advantage of the Athlon on many working sets.

-- kryps

Re:Big deal

[Hoser McMoose]

Actaully AMD had the performance lead for longer then that. They took the performance crown away from Intel the day that the Athlon was first released, since it came out at 650MHz when the fastest PIII was only 600MHz, and the Athlon was, at that time, just slightly faster clock for clock then the PIII. AMD kept the clock speed lead and increased their clock-for-clock performance over the PIII up for the next few years. Intel only just started to catch up with the P4 2.0GHz (which was released just a little bit before the AthlonXP, ie when the fastest Athlon was only at 1.4GHz).

Basically for the last 3 years (since the release of the Athlon), AMD has had the fastest x86 chips for about 2 years. Intel has had the fastest x86 chips for about 6 months, and for the remaining 6 months it's been too close to tell which was faster.

As for the P4s long pipeline, I'd say that it WAS largely responsible for increasing the performance because it allowed Intel to clock the chips so damn high. They clocked the P4 up to 2.0GHz easily on a 180nm fab process. Compare this to the PIII which they struggled to get up to 1.13GHz on the exact same 180nm process (and that took them until 1 year after their first attempt failed miserably and had to be recalled completely). AMD did slightly better with their Athlon design, but it still was only able to clock up to 1.73GHz on a 180nm process, and they had a more advanced process then Intel did in some ways (ie they were using copper interconnects).

Long story short, performance is determined (in an overly simplified way) by IPC * clock speed. With the P4, Intel looked to sacrafice IPC slightly to dramatically increase the clock speed with the goal of overall faster performance. When compared to the PIII at least, they definitely succeeded. Compare the fastest 180nm process PIII (1.13GHz) to the fastest 180nm process P4 (2.0GHz) and which do you think is faster?

Re:Big deal

[Hoser McMoose]

Don't underestimate the potentially detrimental performance impact of the rather small L1 data cache of the P4. Something like 1 out of every 3 instructions in x86 code is a load or store, so even a relatively small difference in cache hit rates can make a BIG difference in performance. This is a large part of the reason why the P4 benefits so much from going from 256K to 512K of L2 cache, it needs it's L2 hit rate as high as possible to offset the relatively low L1 hit rate.

Just what is the exact difference in hit rates between the P4's 8K of L1 data and teh Athlons 64K L1 data cache (the other 64K is an instruction cache, more on that later)? Well that depends entirely on your application. Most numbers I've seen would tend to indicate that your 5% number isn't too far off for most applications though. However, that 5% turns into a VERY large performance difference. Even the fastest L2 cache (ie that which is used in the P4) has a 7-10 cycle latency, and main memory latency is up around 250-300 cycles these days.

The Athlon also has another slight cache advantage, in that it's cache is dual-ported, while to the best of my knowledge, the cache of the P4 is not. That means that the Athlon can be doing loads and stores at the same time, while the P4 can't. This helps the Athlon avoid some real-world latency that might not show up in the theoretical latency numbers.

Ohh, as for the total cache sizes, it's really only valid to compare 8K for teh P4 to 64K for the Athlon. The Athlon has a 64K data and 64K instruction cache. The P4 has a 8K data cache but a 12K micro-op trace cache instead of instruction cache. The trace cache, IMO, is the best/most innovative part of teh entire P4 design, as it stores already-decoded instructions in the cache rather then the x86 instructions. The end result is that this about equivalent to a 20-22K L1 instruction cache, except faster.

Anyway, long story short, I think that the P4's trace cache is great, but it's data cache is too small IMO. It allows Intel to clock the chip to very high speeds, while still keeping the memory latency low, but it makes the chip VERY dependent on the L2 cache and memory subsystem. Memory latency is HUGE these days (for comparison sake, the 386 had a memory latency of 1 clock cycle, today we have a memory latency of about 250-300 clock cycles), and with the P4's high clock speed but long pipeline, any sort of pipe stalls are a VERY bad thing for performance. Even stalling to get data from the L2 cache can be somewhat bad for the P4.

Re:Big deal

[Sivar]

Thankyou, that was a much more informative post than your previous one, and probably more informative than mine as well. :)
Regarding the comment about infinite cache, your message seemed to imply that latency was all that mattered, and that an Athlon with a huge (say, 512MB) cache would be slower than a P4 with it's 8K of faster cache. Seeing as how hust about everything would be run entirely from L1 cache on the Athlon, that seemed rather silly. Perhaps I misinterpreted what you were trying to say.
Indeed, the L1 cache of the Pentium IV is extremely fast. The benchmarks of handling data sizes of 8K are astonishing, but size does matter as has been seen by increasing the L2 cache of the exact same Pentium IV.
As I understand it, Intel chose an 8K cache both to reduce transistor count and to reduce L1 cache latency, which worked, but is of dubious real value. (it is difficult to tell for sure unless Intel makes a P4 with a larger, slower cache, which I doubt)

The purpose of my original post wasn't really to focus on cache size vs. speed, but to highlight questionable design decisions made with the P4. There are other problems (what I consider problems) with the core as well, which I didn't highlight. For example, why is bit shifting so incredibly slow on the Pentium IV? It's faster on just about any other Intel or AMD processor and has long been recommended as an optimization for doing multiplication/division of numbers whose values neatly fell on bit boundries (256, 65,536, etc.) and now this optimization can actually make code /slower/ on the P4? Eh?
Looking at the execution time of various instructions, the P4 has taken quite a few other steps backwards as well.

I remember when people were criticizing AMD for calling the Athlon a 7th generation processor. The rationale was that the P2 was about 30% faster per clock than the Pentium Classic, and the Pentium was about 30% percent faster (this all depends on the code of course) than the 486, yet the Athlon was not 30% faster per clock than the Pentium 2 or K6.
I wonder where all of these people are now that Intel's "7th generation part" is not only not 30% faster, but is actually 30% slower in most tasks, on average.
Strange world we live in.

Re:Big deal

[Anonymous Coward]

I think your sig neads a revision. it is contradictory to your entire post.

Re:Big deal

[MoneyT]

I could swear I've heard all this before, like maybe here: http://www.apple.com/powermac/processor.html

While I will admit that apple does skew their tests in their favor, they have been saying this for years, AMD realized it a while back when they switched to an alternative numbering system. Only intel continues with the idea that Mhz is the only measure of a processor.

Personaly speaking, I think processors, in theri current 32-bit forms have essentialy hit a brick wall. But Intel doesn't want to admit that yet becasue they've invested so much into making people believe in Mhz, that to try to convince people that 1ghz is as fast as the 2ghz they bought last year would be nearly impossible.

Re:Big deal

[Anonymous Coward]

IANAE, but I have two questions. First, is the cache miss rate you are talking about referring to running the spec92 software testing suite? Second, if I have a ten gig cache and I am running code that needs 1 meg of code space and 1 meg of data space why would I ever have a cache miss. Thanks

Re:Big deal

[cioxx]

But Intel doesn't want to admit that yet becasue they've invested so much into making people believe in Mhz, that to try to convince people that 1ghz is as fast as the 2ghz they bought last year would be nearly impossible.

You lost me right there. I would suppose Intel would push the public to believe that 2ghz is faster than 1ghz when comparing P4 2.xx with Athlon 1.n chips

Clarify ;)

Re:Big deal

[Anonymous Coward]

Ladies and Gentlemen-

Please read the parent post to see how moronic you look when you try to argue on a topic after only a two day section in your freshman Intro to Computer Architecture class.

Thank you.

Re:Big deal

[cheezedawg]

The P4 still has quite a performance lead. I just took your advice and checked the 2600+ benchmarks (on tomshardware). The Athlon 2600+ only beat the 2.8 GHz P4 in 3 of the 28 tests. The Athlon beat the P4 in just under 10% of the test- that doesn't sound like the P4 is a "grossly inferior product" to me. And BTW- those benchmarks don't include the SysMark tests that are in question here.

Intel's performance lead is even more impressive when you realize that the 2.8 GHz P4 is available to customers starting Monday, but the Athlon 2600+ won't be availble to the OEM's for another month (and even longer before customers have it).

Re:Big deal

[cheezedawg]

Whoops. I meant that 3/28 is just over 10%.

Re:Big deal

[Anonymous Coward]

I think he's talking about if intel made a faster chip that had less mhz. Like now that they have the 2.x ghz p4s, would they make p5's that used less power, better chips, faster, but ran at only 1.x ghz?

Re:Big deal

[MoneyT]

What I meant was in terms of the itanium. LIke I said, current processors have hit a brick wall. There isn't much farther they can go without some new technology. So we have new tech, AMD is working on 64 bit chips, IBM has the new Power4 chips (I think that's what they were, I'm tired right now) for the macs and pcs and then there's the itanium stuff from intel. One of the reasons intel hasn't and won't for a while pushed the itaniums is that it would be nearly impossible for them to convince their consumers that the chip with a lower Mhz is actualy better than the ones they have now. It's the same battle that Apple and AMD are fighting, except Intel has to fight it against themselves

Re:Big deal

[fmaxwell]

Yes, that's why the press release from AMD is so important in establishing fairness...

Facts are facts, no matter where they come from. If AMD had a factual press release that was independently verifiable, then what are you suggesting the site do? Publish the same information and not credit AMD? Then you'd be screaming that you found a conspiracy when it was discovered that the information was based on an AMD press release.

What's your idea of a "fair" site? One the uses the BAPCo SysMark utility and does not mention that it was written by Intel to favor their CPUs?

It's just a CPU manufacturer we're talking about here, not your wife. If you're happy with your CPU, that's great, but don't start making baseless attacks against a web site simply because they pointed out Intel's underhanded benchmark manipulation.

Re:Big deal

[Anonymous Coward]

Nice try Sivar, but your previous comment about 8k "squared and doubled" totally destroyed your credibility so far as I am concerned.

Not because of the arithmetic errors, but because it makes absolutely no sense to compare measurements of mixed dimensions (in this case, bytes and bytes^2). It's like trying to say that 10 cm is larger or smaller than 5 sq. cm -- it simply doesn't make any sense.

If you ever try to compare two quantities with different dimensions on them, it is a sure sign that you've made an error somewhere. If you work in science or engineering, you learn that lesson pretty damn fast.

ps. to anyone who is wondering -- dimensions are not the same as units. so you can of course compare centimeters to inches, and so on. Of course it is good practice to stick to a single units system whenever possible, just to avoid errors in arithmetic.

Re:Big deal

[Anonymous Coward]

How "fast" a processor is per clock cycle does not matter.

The number of clock cycles per second does not matter.

The only thing that matters is how long it takes the processor to correctly evaluate a set of useful instructions.

If Intel is currently defeating AMD by clocking the P4 to unheard-of speeds... well then, more power to them.

thank you

Re:Big deal

[Anonymous Coward]

Unfortunately for you, your pedantry does not make what Sivar said incorrect.

Re:Big deal

[Anonymous Coward]

-1, Uninformed

Re:Big deal

[Handpaper]

Optimising code for different processors is nothing new (though I agree that optimising benchmarks is misleading). DVDx2.0 (available from dvd.box.sk) installs to give two executables, one optimised for P4. Hovever, the other one works very well on an Athlon 1.2GHz (and even runs under WINE!)

Re:Big deal

[Codifex Maximus]

Hoser, the first Athlon was a 500 Mhz version of the Slot A. I know because mine is still running sweetly after all this time. I do have a new Athlon 1400 though and it is mighty nice too.

Heh, I see no reason to pay any premium to Intel for a processor with no immediate benefits.

Anyway, nice post.

Re:Big deal

[Anonymous Coward]

Idiot, he wasn't talking about dimensions, it was a simple arithmetic comparison.

Your math isn't much better than your reading skills, btw.

Re:Big deal

[Zalgon 26 McGee]

Correction: the Athlon doesn't have 8K squared and doubled in its cache. 8192^2*2 will give you 128 Megs of cache...

drool

Re:Big deal

[Anonymous Coward]

Athlon's L1 cache should be optimized for large workloads using a "Block Prefetching" technique.

It's based on loading RAM contents into the L1 caches via "shotgunned" move operations, and since Athlon does have a very large L1, a considerable amount of memory addresses could stay cached.

About the L1 cache latency issue, P4's D-L1 cranks out latency *every 2 cycles*, while Athlon's L1 cache is *pipelined*, and therefore it may return data to the execution units *every cycle*.

So now...

[Libor Vanek]

...is AMD going to rewrite Sysmark to favor AMD?

Re:So now...

[.milfox]

Well, how about just distributing the task load by what 'typical users' of the application commonly use?

According to the article, only the task loads which favor the P4 are emphasized by the benchmarks.

Of course, who determines the task load is another matter.

AMD with True performance Initiative

[Anonymous Coward]

Check out this link here. It appears AMD is now in a position to redefine performance metrics.
...Formalising the goal of AMD's True Performance Initiative announced last October, Robinson said the CAI aims to establish a new industry metric for processor performance. He said the metric would be established through a consortium of industry representatives...
It's not just Mhz now...

hmmm....

[Anonymous Coward]

does it means that sysmark`s benchmarks is to intel as arthur andersen`s audits were to enron?

Re:hmmm....

[screwballicus]

does it means that sysmark`s benchmarks is to intel as arthur andersen`s audits were to enron?


It otherwise might, but this possibility may be somewhat mitigated by the reality that being 'in bed with finance executives' registers as just slightly less repugnant than the prospect of being 'in bed with benchmark coders.'

Re:hmmm....

[Anonymous Coward]

or perhaps sysmark benchmarks are to Intel what Arthur Andersen's benchmarks are to AMD (see PR ratings benchmarks...).

But seriously, look at the AMD analysis again (as conveniently published at Van's). Pay particular attention to the summary slide and the one that shows weighting between applications.

What you'll notice is that in SysMark 2001, by far the dominantly weighted office application is MS Access - it completely swamps Outlook, Word and Excel. That doesn't sound realistic to me - and it is the biggest single change between SysMark 2001 and SysMark 2002 - these are massively reweighted to give the greatest weighting to Outlook, Word and Excel. Ask yourself whether that's reasonable based on average usage (I think it is, but you might disagree).\

Now, here it is - go and look at that summary slide. The original wieghting in favor of access accounted for 90% of the Athlon performance advantage. So all this stuff about Photoshop filters is trivia - it isn't what makes the difference. What makes the difference is greater weighting to Outlook, Word and Excel over Access.

Don't take my word for it, go and read the details.

VanHardware biased?

[Anonymous Coward]

this http://www.hardocp.com/index.html#5235-1 article on hardocp sheds a different light on the van article

Re:VanHardware biased?

[Sj0]

All I saw was a page long attack on vans hardware for using data provided by AMD. I don't see how reporting with the information that he's actually given is a bad thing. After reading so many hardware sites with their pseudo-science-like analysis of why certain programs benchmark faster ("here we can see the power of the SSE2 and RDRAM catching up to 3dNow Pro, but without occulsion mapping, will it be able to stay there?"), it's refreshing to see just the facts, and not some lameassed attempt at sounding important by stating theories as fact.

should be open.

[GoatPigSheep]

Obviously, the best bet for cpu benchmarks would be an open-source one compiled using a standard compiler. This is a case where open-source really shines.

Re:should be open.

[ObviousGuy]

Wouldn't a better CPU benchmarks be taken by using the chipmakers' own compilers?

Re:should be open.

[GoatPigSheep]

if you are measuring raw performance of the cpu, then you would want the program to not be optimized for any cpu.

Re:should be open.

[AvitarX]

But you don't want to messure toally raw performance. You want to messure something approximating how it is going to do what you want it to do. so the bench mark should really be compiled however the vendors most often compile there software.

Re:should be open.

[ObviousGuy]

I think if you want real-world performance marks, it makes sense to use real-world apps. If you want to know how much power you can squeeze out of a chip, though, you ought to go to the source and get compilers that are designed to squeeze that power out.

Re:should be open.

[GigsVT]

But there is a chance, a good one, that a generic benchmark will not reflect real world performance due to random chance in the way the compiler works, or the test you are using.

Suppose you decide to use Distributed.net as part of your benchmark suite. Certain processors (MIPS) do not have a hardware implementation of a certain operation (rotate left I think it is), which makes them seem extremely slow when you look at Dnet numbers.

I don't think benchmarking will ever be boiled down to a simple solution, there are always little complexities which call any numbers into question.

Re:should be open.

[GoatPigSheep]

Well in this case the comparison is between two x86 cpu's, the athlon and the pentium4. Both would support standard x86 instructions. If you want to measure how fast the cpu is you would want the program to be unoptimized. Perhaps SSE would be fine since both cpu's support it.

Using optimizations wouldn't be fair unless you had a good idea of the percentage of programs that ARE optimized for one or both cpu's. Many new programs are optimized for both cpu's, such as Cubase SX, a software studio program. I suppose you could use one of those programs as a benchmark in addition to the raw unoptimized open-source one so you can get an idea of how well the cpu performs with or without it's appropriate optimizations. Also, it makes a difference wether there is a free version of the optimized compiler, because if there isn't, there is a higher chance that programs made by individuals at home (who can't afford a 500$ compiler) would not be optimized.

Re:should be open.

[l33t-gu3lph1t3]

If what you say is implemented, then it will be the simple i386 instruction set used for all benchmark programs. This would be utterly useless, as benchmarks are written to simulate REAL LIFE performance. And considering that intense programs are written to utilize every last ounce of performance from the 2 main PC processor types, a simple common codebase would ruin a benchmark program. P4s run Quake3 faster because they're OPTIMIZED for the kind of chores Q3 puts a processor through. Athlons run most office apps faster because they have a great branch prediction unit. When measuring performance, i386 is a joke. SSE1/2, 3DNow!, enhanced FPUs, etc, these are where performance lies...

Re:should be open.

[Ninja Programmer]

Perhaps you should keep browsing Van's page then. He has an idea about making an open source benchmark using a standard compiler

Re:should be open.

[Shadow99_1]

This would seem when Van's COMPREHENSIVE OPEN SOURCE BENCHMARK INITIATIVE (COSBI) would be useful... You could always get in touch with Van about helping out the project...

Re:should be open.

[Ninja Programmer]

No it wouldn't be better. Its better to use a compiler that is actually used in the applications that you run on that platform. In the case of x86, that's going to be Microsoft Visual C++. You could even argue Sun or Microsoft's Java compilers, Visual Basic, or Delphi could be added to the mix.

However, using Intel C++, is not credible since the vast majority of C++ applications running on x86 don't use that compiler.

Re:should be open.

[ObviousGuy]

Sure, in one type of benchmark it would be useful to have the most common compiler used. In one that you wanted to show the full capabilities of the chip, though, it makes sense to use the best of breed compiler to run the tests.

I probably misspoke when I used the word "better". Using the chipmaker's compiler makes for a different type of benchmark, one that it is well suited for. However using the GNU compiler (as an example of an Open Source compiler) to run benchmarks doesn't really measure anything substantial that another test like your own (using the most common compiler) takes care of handily.

Re:should be open.

[fmaxwell]

Wouldn't a better CPU benchmarks be taken by using the chipmakers' own compilers?

It depends on what you want to measure. If you want to measure what the CPU can do for a vendor developing an embedded system around the CPU, where the vendor compiles everything from the OS to the application, then yes, that makes sense.

But if you want to know what the CPU can do in a desktop PC running Windows and off-the-shelf, pre-compiled applications, then you want the benchmark to be compiled using the most commonly used compilers.

Re:should be open.

[Anonymous Coward]

I moderated you down because of you, not your posts.

Re:should be open.

[kolding]

Not really. Sysmark is designed to measure the performance that you would see on "typical" desktop applications. Open benchmarks are frequently only microkernels which tell you a little bit about how the processor will handle one piece of code, but very little about how it will actually work on a real application. Remember Bytemarks? Those were a popular, semi-open benchmark (you could get the code, but you couldn't recompile it and still use it in a benchmark result), but they didn't represent anything real. SPEC does a decent job of indicating who has the best processor/compiler pair, on compute bound engineering type workloads, but it is not incredibly predictive of desktop performance on interactive apps compiled with random compilers.

An open source benchmark would probably not tell much useful at all.

Re:should be open.

[fean]

Hey guys... read the whole article... notice who they cite as the provider for ALL of the information (for the lazy ones, it's an AMD paper that's hidden in their web site)

I love AMD, I only own AMD (and a C3, but that's in my vehicle) processors, but this is rather underhanded... for them to do tests, and quietly release the results so that a web site can regurgitate(sp) the information and make it look independant.

2 wrongs don't make a right... and their first wrong was not cooperating with the benchmark people in the first place.

Re:should be open.

[SmokeSerpent]

There should be a 3-part benchmark.
Part 1: Assembled by a truly unbiased panel
Part 2: Assembled by Intel
Part 3: Assembled by AMD

Re:should be open.

[AvitarX]

That might as well say
part 1) assembled by the Easter Bunny ...

processor strings

[the way, what're you]

Similar to the Quake/Quack debacle, I wonder if it would benchmark differently if the processor reported itself not as "Intel Pentium", but as "Incur Penalty", "Inept Penguin" or "Inane Penises". :)

Re:processor strings

[Ninja Programmer]

On an unpatched installtion of Windows, yes, it would.

This is the idiotic method that Windows Media Encoder decides whether or not SSE instructions might be supported.

Gacy?

[ObviousGuy]

Dudes, what's the deal with this ad?

Couldn't get a picture of Bozo or Marceau? Is the only clown pic you could come up with this one of John Wayne Gacy?

Re:Gacy?

[MaxVlast]

Offtopic yes, creepy? Yes.

Re:Gacy?

[Anonymous Coward]

OSDN needs the cash, so now they take money from advertisers who support child molesters?

Re:Gacy?

[Anonymous Coward]

In the context of the ad, I think having an evil clown (evil because he's filtering your net acess, of course) is what they wanted.

Re:Gacy?

[ObviousGuy]

Yes, filtering web access is just like raping and killing children... :-/

Re:Gacy?

[fmaxwell]

Yes, filtering web access is just like raping and killing children...

Not exactly. There's more screaming when you filter web access.

Not Quite So

[T-Kir]

The thing about the Athlon line of processors is the FPU, which blows the P4 away.

The P4 is better with the memory throughput, considering it was designed for using RAMBUS in mind, hence the more memory bandwidth you throw at it, it utilizes it (SD/DDR RAM is like using a handbrake on it).

The new Thoroughbred Athlons make AMD regain the performance crown, and the Athlon has the Barton revision to go yet before Hammer.

AMD maybe considered a value chip company, but that doesn't mean they cannot produce excellent CPU's.

Re:Not Quite So

[VAXman]

The thing about the Athlon line of processors is the FPU, which blows the P4 away.

According to SPECfp, which is by far the most impeccable FP benchmark in the industry and supported by a huge industry consortium, the Athlon XP 2200+ scores only 624, while the 2.53 GHz Pentium 4 scores 861. What industry supported application supports your claim that Athlon has a superior FPU?

Re:Not Quite So

[Nogami_Saeko]

Apples to apples please - when you clock the Athlon at the same clockspeed as the Intel chip, (which are possible with the new chips that AMD just released) the FPU is far faster on the Athlon chips.

This is what indicates a superior FPU design, not a comparison based on a ~700mhz difference in clockspeed.

Re:Not Quite So

[Hoser McMoose]

"impeccable"?! I dunno about that!

SPEC CFP2000 is by far the most widely supported benchmark, but it has it's share of flaws, just look at Sun's latest SPEC CFP2000 scores and you might notice one of them. For the Sun Blade 1900 (900MHz USIII), Sun had a rather abysmal CFP score, with the Sun Blade 1900 Cu (900MHz USIII), they were suddenly quite competative. Hugely improved compiler right?! Wrong. Actually they only improved by 0-15% on most benchmarks (about what could be expected with a slightly better compiler, ie what you could expect in the real world), however their CFP score improved by over 50%. Why is that? Well ONE single sub-bench of CFP (179.art) was increased by a remakable 560%! The end result is that the USIII looks like a rather fast chip at floating point, when it's actually butt-slow (WAY slower then either the Athlon or the P4) at everything except for one particular Fortran application. This isn't to say that Sun's score is invalid, simply that it doesn't take much to really skew the scores of ANY benchmark, including SPEC CFP.

I've also critized SPEC in the past for being a bit to focused on large datasets. For example, the 171.swim benchmark is essentially a memory bandwidth test taken from some shallow water modeling code. Now, that isn't to say that memory bandwidth isn't important for floating point and scientific applications (if it weren't IBM wouldn't have thrown a whole boatload of bandwidth at their Power4), simply that SPEC seems to have over-emphasized this aspect. I think the reason for this is that SPEC CPU95 was just the opposite, and often critized for emphasizing SMALL datasets too much, and therefore become almost completely a test of the cache architecture of the processor in many cases. For CPU2000, it seems like they over-corrected in my mind.

Long story short, if your floating point work involves small datasets, the Athlons MUCH larger L1 cache and 3 floating point execution units will crunch the code much faster. On large datasets, the P4's higher bus and memory bandwidth and larger L2 cache will crunch the code faster.

You're quite correct in that saying that the P4 has a weak FPU is definitely wrong. The chip actually is VERY good at floating point calculations, better then just about anything out there. Only the Power4 and the Itanium2 are decidely faster FP number crunchers then the P4.

Re:Not Quite So

[VAXman]

Apples to apples please - when you clock the Athlon at the same clockspeed as the Intel chip, (which are possible with the new chips that AMD just released) the FPU is far faster on the Athlon chips.

Not that it matters, but P4 and Athlon performance are approximately equivalent on a per-clock basis: Athlon does 624 / 1800 MHz (0.346) "SPECfp points per megaherz" (whatever that means), and P4 does 861 / 2533 MHz (0.399). Of course since P4 can clock much faster it has better absolute FP performance.

This is what indicates a superior FPU design, not a comparison based on a ~700mhz difference in clockspeed.

Let's throw Itanium 2 into the picture, which does a whopping 1356 SPECfp at an equally astonishing 1.0 GHz, which puts it at this silly "SPECfp points per megaherz" of 1.356 - quadruple that of Athlon/P4.

Does that mean Itanium 2's FPU is four times better than Athlon?

Re:Not Quite So

[Anonymous Coward]

gg intel fanboy, way to stick out with your tardfactor 9

Vans had nothing to do with it.

[l33t-gu3lph1t3]

AMD did the data-mining. Just like a poor fansite, Vans just posted the PR complete with their own pretty graphs made from the numbers AMD ran.

Re:Vans had nothing to do with it.

[Ninja Programmer]

Actually Van has been saying this for some time now. While this last chapter in this saga was mostly accomplished through the leg work of AMD, Van deserves credit for calling this one correctly a long time ago.

surprised ????

[dr.Flake]

Nothing seems to surprise me anymore in this world.

accounting tricks, monopolies, dirty tricks in business.

I guess we should have seen this kinda thing coming a long time ago.

In a way, anything imaginable in this line of business is probably also likely to happen.

whats next?:
The accused, Bill G., was caught red handed manipulating goverment studies concerning TCO and alternative OS's?

Oh wait......

welcome to the real world.

You probably have a P4, right?

[Anonymous Coward]

They're not just keeping up with intel, they are significantly faster in most real-world applications. Over 90% of systems dedicated to rendering and scientific computing sold in the last 2 years were Athlons (they're actually replacing SGI and Sun in thse markets because they are so much cheaper and actually offer better floating-point performance).

All the P4 is good at is moving memory around. And for this it seeds RDRAM (ie, very fast memory). Replace that with DDR and the P4 turns into a very expensive snail.

Even with RDRAM, the P4 is still slower than the Athlons on most real-world tasks. IMO, the only valid reason to buy a P4 is Quake.

I have several Athlon XPs and P4s and the P4s just drag themselves compared to the XPs. In fact, even the PIIIs feel faster than the P4s. And I'm not even talking about the Athlon MPs (that simply wipe the floor with the P4).

Re:You probably have a P4, right?

[Anonymous Coward]

Please define what you mean by "real-world tasks". How exactly does the greater performance of the Athlon FPU benefit most "real-world tasks"?

Re:You probably have a P4, right?

[Teknogeek]

>> IMO, the only valid reason to buy a P4 is Quake.

And that's a BAD thing?

Re:Shut up and keep downloading your warez and mp3

[MaxVlast]

Now I have proof that my P4 makes the Internet go faster!

Processor nonsense

[Anonymous Coward]

If AMD would stick to making totally Intel compatible chips instead of trying to infuse their own personality, we wouldn't have this problem. Hint: my software shouldn't need to know it's running on an AMD chip.

Who cares?

[YahoKa]

Tomshardware, Anandtech, Hardocp ... perhaps they will just be more careful what benchmark programs they use.

Re:Who cares?

[MtViewGuy]

Actually, both Tom's Hardware and Anandtech sites clearly mention during their testing that certain benchmarks favor the Pentium 4 due to things like SSE2 multimedia extensions support and better support in the test program for the way the Pentium 4 processes CPU instructions.

Hopefully, this will force BAPCo to do a benchmark from the ground up that will test functionality common on both the Pentium 4 and Athlon CPU's (like just MMX and SSE support only for now until the new Barton and Hammer core CPU's from AMD get full SSE2 support).

Not really favoring the P4 over Athlon...

[l33t-gu3lph1t3]

Notice that the Via C3 does phenomenally better than the 1.7GHz P4-Celeron. Now consider how utterly weak the floating point unit (FPU) of the Via C3 processor is, its use of SDRAM, its lower bandwidth memory bus than the Celeron. I won't call the P4-Celeron a performance processor, but VIA's C3 should *NOT* be able to beat it in ANY performance benchmark.

What this says is that SySMark is really poorly coded, not "optimized" to favour Intel silicon. Incompetancy isn't evil. It's just...incompetant. This would explain why most serious benchmark runs seem to lack SysMark these days...

Re:Not really favoring the P4 over Athlon...

[Anonymous Coward]

Those VIA results are interesting and remind me of one of my favorites from the real world benchmark hall of fame involving the AMD K5.

The RC5-64 contest offers clients optimized for various architectures. You can test a client by invoking it with the "-benchmark" or "-benchmark2" option. Surprisingly, the AMD K5 client running on a K5-100, absolutely smokes an Intel P5-200 running at twice the speed. The AMD chews through about 325000 keys/sec., whereas the much more costly Intel P5 running at twice the speed only can only manage about 285000 keys/sec.

Of course these results don't mean much now, but when the RC5-64 contest started, you could pick up a K5 with motherboard for about 45 or 50 dollars on pricewatch. It was the poor man's RC5-64 "super computer". It shows that you don't always need "name brand" recognition to acheive good performance.

Re:Not really favoring the P4 over Athlon...

[Anonymous Coward]

Be careful how you interpret that graph. The graph only shows the results on the Paradox DB test. Pentium 4 does far worse on Paradox than it does on any other component of Sysmark. As usual, Van only showed part of the story, the part of the story he could try to embarass Intel with. Hardly an unbiased representation. What's the total Sysmark score for the Via system?

Re:Not really favoring the P4 over Athlon...

[KoolyM]

Where do you get your figures from? I distinctly remember a comparison in (the dutch edition of) this month's CT magazine (which, usually, is quite a quality magazine) that showed Via's C3 to be a far worse performer than a similarly clocked Celeron.

Re:Not really favoring the P4 over Athlon...

[version5]

> What this says is that SySMark is really poorly coded, not "optimized" to favour Intel silicon.

Ah, of course! When software that was essentially developed by Intel to measure the performance of its flagship processor when compared to its competitors magically favors Intel, its pure coincidence and probably the work of incompetent developers.

Come on, how naive do you think we are?

Regarding the Via C3 vs. Celeron 1.7 test: if you look at the front page, you'll notice that they ran this benchmark on systems with different configurations and even different operating systems, so it seems like its a "Wow, look at that!" thing rather than a real benchmark. I hope.

Benchmarking the Benchmarks.

[Taco Cowboy]

Someone ought to do the following:

Benchmarking the various "Benchmarking Programs"

I know that the idea is kinda fuzzy at the moment, but until we all have a clear idea of what we are looking at, I am afraid that some of the "Benchmarking Programs" do not give us true view of what we ought to be seeing.

Re:Benchmarking the Benchmarks.

[rlthomps-1]

Yeah random PC hardware fans can do "meta-benchmarking" and decide if the benchmarks are accurately portrayed.... now if they could only do something like that to the mods here....

Re:Benchmarking the Benchmarks.

[kigrwik]

> Benchmarking the various "Benchmarking Programs"

Yes but, "Quis benchmarkiet ipsos benchmarkiem ?":

Who benchmarks the benchmarks ?

(s/benchmark/custod/g and Google for the original quote :)

Re:Benchmarking the Benchmarks.

[MrFenty]

"Quis custudiet ipsos custudiem ?"

...errm, "who guards the custard ?"

Re:Benchmarking the Benchmarks.

[rodgerd]

There's a reason it's called benchmarketing.

There's also a reason that the credible organisations (TPC, SPEC, etc) have to regularly tune thier benchmarks to work around manufacturer cheats.

Re:Benchmarking the Benchmarks.

[3583 Bytes Free]

Who benchmarks the benchmarks ?

Homer: So I said to him, "Look, buddy, your car was upside down when we got here. And as for your Grandma, she shouldn't have mouthed off like that!"
Lisa: Dad, don't you see you're abusing your power like all vigilantes? I mean, if you're the police, who will police the police?
Homer: I dunno-Coast Guard?

Partialy AMD's Fault

[mtthws]

I hate to say this, I am a big AMD fan, but it is partialy AMD's fault that SysMark favors Intel. They have refused to work with the BABCo people in the past knowing the Intel people have. Is it any suprise that they end up favofing the company that works with them over the one that ignores them? AMD is now supposeidly working with them to make the next version do a more fair job testing their proccesors. So hopefully this will be a non issue in future realeses. It would prob be most fair if AMD and Intel would both let the benchmarking programs be written with out either of their interfiernce, but if one is going to get invloved, then they both realy need to.

Re:Partialy AMD's Fault

[l33t-gu3lph1t3]

Benchmarking isn't about fairness. It's about being better than your competition, and more importantly, LOOKING better than the competition...

Re:Partialy AMD's Fault

[Ninja Programmer]

BapCo's head quarters are on the Intel campus. Its been Intel biased from day 1 (back when AMD was making K5's and thinking about making K6's) and AMD has known this.

The fact is, prior to the release of the Athlon, nearly all benchmarks were biased towards Intel. AMD's strategy when they released that Athlon was to make a CPU so good, it could beat Intel's CPUs even on these benchmarks. Sysmark just happens to be the one benchmark where Intel exercises so much control that it could literally say whatever Intel wanted it to say.

What you are seeing is AMD just starting to switch strategies from "lets just beat them on every benchmark under the sun regardless of bias" to "lets expose the bias where it is as its worse so people can know the truth".

This is all just preparation for the K8 launch I think. If AMD can properly put Sysmark results into perspective, maybe everything that is left will show what a monster K8 is versus any Intel offering. It is indicative that the K8 may not be winning on Sysmark on internal testing, or may not be winning by a sufficient margin.

Re:Partialy AMD's Fault

[Dr. Spork]

I think this speaks volumes: Intel are schmoozing the benchmarkers while AMD are designing kick-ass processors. I hope the stockholders are listening!

Re:Partialy AMD's Fault

[Anonymous Coward]

If they are, they'd be happier with Intel...

Re:Partialy AMD's Fault

[Anonymous Coward]

More than on Intel campus, written by Intel programmers. Sysmark = made by Intel.

Re:Partialy AMD's Fault

[cheezedawg]

Stockholders don't give a crap about technology- they care about money.

That said, considering the new Intel 2.8 GHz P4 beats the new Athlon 2600 in a majority of the benchmarks, I would say that both companies are designing kick-ass processors. The difference is that Intel markets itself tons better.

Not another benchmark...

[mustprotectdata]

Coming from the Unix world, I'm used to comparing machines based on their SPECint and SPECfp performance...

In general the SPEC people have done a better job being platform agnostic than some of the "miscellaneous" PC benchmarks.

Current benchmarks for Intel http://www.spec.org/osg/cpu2000/results/res2002q2/ cpu2000-20020506-01357.html

and AMD http://www.spec.org/osg/cpu2000/results/res2002q3/ cpu2000-20020701-01441.html

Keep in mind that results for more recent AMD CPUs are not shown. If you compare the AMD 2200 with a 2.2G P4 you'll have 734 v's 784, which gives some credence to AMD's claimed rating.

html4me!

Re:Not another benchmark...

[mustprotectdata]

That's actually AMD = 764 v's Intel = 784 so it's even closer than stated above, i.e. within 3%.

Like anyone would be able to tell :-).

And my poor little Sunblade 100 is only 174. No wonder Solaris seems slower than linux.

html 3

Read the linked article

[Anonymous Coward]

As compilers become tuned to exploit this, it's plausible that the Athlon's performance is going to lag quite a bit more than it already does. That there is some benchmark out there that is specifically designed to show off this strength of the P4 is no real surprise to anyone, is it?

That's not the complaint at all. Read the linked article. The complaint is that Sysmark 2002 has been systematically altered relative to Sysmark 2001 so as to favour the P4 over Athlon.

For example, the PhotoShop test in Sysmark 2001 had 13 filters, of which 8 run faster on the Athlon and 5 faster on P4. The Sysmark 2002 PhotoShop test has 6 filters, of which 3 are filters from Sysmark 2001 on which P4 wins and the other 3 are additions on which the P4 also wins. The 8 filters on which the Athlon does better have all been removed.

There are several other examples in the article. Read the article

BTW, an interesting point is that this whole thing is basically an AMD publication that AMD have chosen to proxy via Van's. Van is at least open about it. The AMD presentation containing all the information in that article is linked at the end and is available here

Kyle @ HardOCP covered this yesterday

[Cutriss]

Here's Kyle's 4th Edition post from yesterday. Excerpts from Van's comments are in italics.

VansHardware & AMD: There is a report on VansHardware this morning that visits the differences between BAPCo's SysMark 2001 and SysMark 2002. The report's basic theme is that SysMark 2002 is skewed towards making the Intel Pentium 4 results look better than the AMD CPU results could have looked. It basically shows examples of things that were changed in SysMark 2002 that cherry pick areas in certain programs that the Pentium 4 excels at. While the article might seem to be work done by VansHardware there is something you need to know. All of the data shown in that article has been put together by AMD and not VansHardware. Take note of this one statement in the article.

However, AMD has been able to "pick the lock" on SysMark to gain a much keener understanding into the internal workings of these tests.

VansHardware is not the one with the "keener understanding", AMD is.

The original PDF document from AMD is linked for download so the fact that this data is not Van's is not exactly hidden either.

Also their opening paragraphs state this.

At this moment we will pause from the long march through our benchmark results to revisit the significant issues regarding BAPCo's SysMark 2002 brought up by AMD during our recent meeting with representatives from that chipmaker.

We must state up front that despite the condemning information divulged to us, the AMD spokesmen repeatedly expressed support and guarded optimism for the reformation of BAPCo.

The "significant issues" and "condemming information" shown were not harvested by VansHardware, actually all they do is interject a little bit of commentary.

AMD has verified to me this morning that all of the graphed and tabled data shown on the VansHardware report is data that has been mined by AMD. Does this make the data inaccurate? Of course not, but I am sure that it hardly shows both sides of the story. AMD is not going to supply VansHardware with information that makes Intel look good. VansHardware represents to me, nothing more than an AMD fansite that takes shots at Intel every chance they get. I think they are far from what anyone could consider objective journalist and reporters. Them doing a cut and paste job with AMD's data goes to show that as true in my opinion. Websites get fed information all the time, trust us, we know. It is our jobs to go back and prove data and claims in our labs on our own time, not to repost corporate data, that can be considered far from objective. Independent sites in our hardware community should not be reposting PR spin in such a way as this. There is a fine line here but I think this is stepping across it.

VansHardware does not exactly hide the fact that the data shown is not theirs but rather AMD's, but they certainly did not seem to represent that in an upfront manner so the reader sees the information for being exactly what it is...data released by the AMD PR machine.

I am a huge AMD fan but I just don't like big companies being able to pump their corporate data into our community when it is not presented as such. I think AMD should have the balls to post information like this on their own website and not try and "slip it in" through a back door. In fact, I would consider the information to be much more credible if it were posted on AMD's own website as AMD research.

I know Van has gotten upset here recently with his past employer removing his name from articles he has written. It seems to me that Van has done little to deserve his name being on this article and it should show authored by AMD.

(ED NOTE - This is referring to some allegedly plagiarised articles that Tom's Hardware published after removing Van's name from them)

Also worthy of mentioning is that AMD is now fully working with BAPCo, which they have not done in the past. AMD has had the ability to work with BAPCo for a long time now to make sure their products get represented properly and we are certainly happy to finally see AMD join the party to give the boat a more even keel.

Lastly, another tidbit worth throwing into the mix is that Van Smith, owner of VansHardware, possibly either works for or is contracted to VIA as a CPU validation tester. We are working on a confirmation of this from VIA now. Do we need hardware websites that do work for the companies they end up reporting on? Just another thing to consider when objectivity is in question.

Re:Kyle @ HardOCP covered this yesterday

[Elbereth]

I'm surprised you got moderated as "overrated". What Kyle said is actually rather insightful (and something that I've believed about Van for a loooong time).

Van is just a fanboy with a website. He doesn't do his own research, conduct any tests, or even say anything moderately interesting. It's like giving the pulpit to a slashdot poster during a congressional hearing on the constitutionality of the DMCA. "IT SUXORS!!! DOWN WIT DCMA!!!!111"

Re:Kyle @ HardOCP covered this yesterday

[BrookHarty]

The article was clear at Vans Hardware, he wrote an article using AMDs information... Van Smith should of wrote the article with a little more distance from AMD, but that doesnt alter the facts from AMD.

I didnt see that article over at HardOCP when I posted the news last night. But after reading HardOCP comments, You can see Kyle is really pissed off at Van Smith. Kyle even links to another site Real World Tech where people are talking about Glad someone released the information... Could it be HardOCP is getting ready to release a major article, and Vans Hardware took the spotlight?

There is a hint of back room dealings going on. I picked a new magazine "CPU" that has people from various places. Interesting to see what happens in the next year and major fansites... Heres a list of authors for "CPU" magazine. Rob "CmdrTaco" Malda, Anand Lal Shimpi, Kyle Bennett from Hard OCP, Joan Wood co-founder of Sharky Extreme, Alex "Sharky" Ross, Alex St. John (founder of directx at microsoft), Chris Pirillo (creator of LockerGnome/host on TechTV), Pete Loshin (former editor of BYTE Magazine, runs Internet-standard.com), Lisa Lopuck (Author of Web Design for Dummies).

Best Way to Benchmark?

[d'fim]

Wouldn't the best way to benchmark be to combine approaches? I want to see performance ratings on apps, equalized ratings on raw power, and unequal ratings on chip-optimized raw power. The benchmark suite should give the whole picture, then let me decide which way I want to go with the numbers.

Is this how Sysmark is written?

[VistaBoy]

void Benchmark()
{
cout "AMD Athlon XP: infinity MIPS";
cout "Intel Pentium 4: infinity plus one MIPS";
}

Re:Is this how Sysmark is written?

[Anonymous Coward]

God fucking damn C++ n00b. Something tells me you are some rejected, sad teenager. C++ is homo. Bjarne Stroustrup molested cats in his prime.

Benchmarks...

[Faeton]

It's been said that there are 3 types of lies.

Lies, damn lies, and benchmarks

Q3A

[Anonymous Coward]

How well does the benchmark run Quack 3 arena?

Yes

[Anonymous Coward]

If they used Intel's compiler they could have caught that bug. Since they were using gcc, it ran and crashed immediately.

Pick what you consider for your benchmark

[(H)elix1]

When I dig through reviews on the latest CPU and/or mainboard, I initially groaned at the increasing number of benchmarks folks would put out. It is more than just increasing click-through rates (well maybe not for some, but...) - it lets me see applications that I use. Synthetic benchmarks and politician's promises garner then same level of trust from me.

Anyhow, I game and code but use games to judge where my cash goes. When the P4 came out, I saw it did great job with Quake and I started to get excited about the CPU. Then I saw the benchmarks on the games I actually play - UT, CS, and a few others - and it was not black and white. After the ATI fiasco, Quake is up there with synthetic benchmarks IMHO. As for Photoshop, you can pick what platform you want to 'win' by tuning the filters. Apple does it, their dually box wipes out the competition, the other do it and the tables are turned.

There are great graphs out there that show benchmarks using different sizes of data. Its like comparing a small turbo charged engine to a larger normally aspirated one - so what RPM were you at when you ran your test? BMW's M5 feels slower than an Audi S4 at the start, but get the RPM's up there and it is a different story. Even pickup trucks can beat a Ferrari if you tune the test to take advantage of a sweet spot.

I've done my homework, and my personal cluster is mostly AMD today. Still have one celeron 566@800 as a CS server, but my workstation (Intel Xeon box) was replaced by AMD MP chips. Secondary boxes are all XP chips, but they use to be PII&III's when Citrix and the K5 sucked. They run Oracle, Weblogic, LDAP, and other stuff quite well when I'm working, and one swap of a hard drive later I'm getting some solid fragging in on the same box. In another year or so, if Intel really hold the crown , the price is right, and my boxes are 'only fast enough for web browsing and email', I'll chose them.

This just has to be a result of the lawsuit

[Vlad_the_Inhaler]

A couple of days after some Lawyers get together for a class-action suit alleging that Pentium IVs are slower than the AMD competition, new BAPCO tests 'prove' that the Pentium IV was quicker all along.
Nice one Intel. At the very least, this should muddy the waters with respect to which one is quicker being a matter of opinion.
I use a 7 Watt Via C3 as opposed to one of the 60 Watt P4/Athlone and do not really care either way.

Open Source Benchmarks the way to go?

[Squidgee]

As previously stated here open source seems to be a good way to go.

Though there are hurdles. For one, Intel and AMD may not want whatever 'IP' is in these benchmark programs getting released into the open source world. Also, this would most likely make it easy(ier) for biased sites to edit the code, and post their (tinkered with) results without posting the edited code.

Mind you, those would be some kinks to work out; but an open source benchmark program would, in my opinion, be the only way to truly avoid the biasing toward each chip maker (That is if they honestly used the OSS benchmarker).

Re:Open Source Benchmarks the way to go?

[AvitarX]

If the sights were to really tincker with the results they would be discovered in a matter of seconds. This fact would then be posted all over the internet (at least somewhere) and the smart users would not be fooled. The dumb users would not check benchmarks and have no nead to read the code anyway, so could be fooled just as easy.
There is no way open source would make bencjmark faking any easier. I mean they could post fake tables/graphs and I would be in the dark. I have no Pentium or athlon to test on. In fact I could verify the validity of the open system by recompiling with the options they chose on the compiler they chose and comparing binary files for free/time not for hundreds of dollors.

Intels C++ compiler is free on Linux [more]

[Styx]

for non-commercial purposes, anyway

Re:Intels C++ compiler is free on Linux [more]

[Dave2 Wickham]

Yes, but that optimises it for the Intel processors...

Quote from their site:

The Intel® C++ Compiler 6.0 for Linux makes it easy to get outstanding performance from all Intel® 32 bit processors, including the Pentium® 4 and Intel® Xeon(TM) processors, and the 64-bit Intel® Itanium(TM) processor.The Intel® Compiler provides optimization technology, threaded application support, features to take advantage of Hyper-Threading technology, and compatibility with leading tools and standards to produce optimal performance for your applications. The Intel C++ Compiler delivers leading-edge performance and excellent compatibility, and comes with first-class customer support.

That wouldn't exactly help with fair benchmarks...

Re:Intels C++ compiler is free on Linux [more]

[Dave2 Wickham]

Damnit, trust me not to read it properly!

Anyway, I can't see any free AMD compiler...

But the real question is...

[Space Coyote]

... How well do they do with Photoshop batch processing?

I'm actually partly serious here, I think wider publication of more 'real-world' performance figures is in order. The people who frequent sites like Tom's Hardware and Anand are the only ones who really care about raw benchmark numbers. The rest of the world is more interested in getting their work done more quickly.

Re:But the real question is...

[barawn]

That's what SysMark is supposed to be: they measure "real-world" performance figures - they run a slew of Photoshop filters, and time it, and other crap.

Unfortunately, SysMark's testing strategy is really terrible. I'm even a bit confused how it works: they say that they scale each test based on how long it takes to complete: but is the scaling from a "reference system" or from each system? If it's from a reference system, then it's biased against whatever that reference system is good at (since the difficult bits get weighted more). If it's from each system on the fly, then it's really meaningless, as one poorly-chosen benchmark can skew the whole thing.

Worse yet: in SysMark 2002, AMD claims that BAPCo uses the same benchmark, multiple times: this is just plain bad, because not only does it magnify the importance of this benchmark, it shrinks the importance of all of the other ones. It's just plain idiotic. Take 3 tests, run them 4 times each, and use the results from all of the runs? It's a very very obvious bias - the only reason you would do that is if you wanted to cheat for one specific processor, and you knew which filters it was good at.

Enough of this bs

[f00zbll]

Why isn't there an organization that is truly objective? TCPC (transaction processing benchmark) produces benchmarks that are generally reliable and objective compared to Sysmark.

Not sure open source is the answer, but there needs to be a third party that guides the direction of benchmarks. One that's defined by what consumers use, not by what Intel or AMD defines as good for them.

Re:Enough of this bs

[HimalayanRoadblock]

But buying computers and lab space and electricity costs $$$. If you want provide me with the 3 above I'll gladly run all the benchmarks you want.

optimize for both, separately

[Ledskof]

Since it's honestly difficult to approach a benchmark that isn't biased at all, how about a benchmark that shows both sides of bias? One part of the benchmark is for AMD optimized software and the other for Intel. Then you can see how much of a difference your processor runs software optimized for a different processor. As well, you can truly decide which process you should have based on what kind of software you have. That is if you are willing to give Intel money if that possibilty presents itself, which I'm not willing to do.

I'm starting to feel this would be the only useful industry processor benchmark, because the term "real world" is too subjective.

It would also keep everyone from getting pissed off and calling it biased. Of course it's biased! The only problem is that to be fair, it must support every processor brand that comes out. I don't see a problem with that though. If it's a benchmark as widely accepted as SysMark, then it might actually help that brand grow.

Not to defend Intel but....

[DeadBugs]

HardOCP notes that Vans got their info from AMD so it may be a bit biased. a quote from HardOCP:

" AMD has verified to me this morning that all of the graphed and tabled data shown on the VansHardware report is data that has been mined by AMD"

"AMD is not going to supply VansHardware with information that makes Intel look good. VansHardware represents to me, nothing more than an AMD fansite that takes shots at Intel every chance they get. I think they are far from what anyone could consider objective journalist and reporters."

Re:Not to defend Intel but....

[Chris Johnson]

If they are telling the truth, it doesn't matter how biased they might be. They could believe in black helicopters and Elvis sightings for all it would matter. The question is: is the information they put forth accurate? If so, then Intel is indeed yanking people's chains with benchmarks (as a Mac dude I can't repress a 'oh, THAT'S a surprise' reaction) and the bias is in how the site draws conclusions from this, and how loudly they remind people of stuff like the class action suit over misleading performance claims for the P4.

Which, surprise surprise, they do indeed remind people of! And if this is true, they'd be right that it was a smoking gun w.r.t. that lawsuit, too.

Let them go on being an AMD fanboy site. I don't see INTEL fanboy sites breaking this story.

Ouch

[Spackler]

However, AMD has been able to "pick the lock" on SysMark to gain a much keener understanding into the internal workings of these tests.

Isn't that a violation of the DMCA?

Re:Ouch

[Anonymous Coward]

Aren't you in violation of being cool?

SPEC is open source

[-1, Accurate]

http://www.spec.org

This benchmark is composed of various programs such as gcc and infozip distributed with source code. It's open source. It's respected. And it's already available. There's really little need to put together a new open source benchmark.

Marketing will always find a "good" benchmark. Dev/QA rely on real benchmarks.

Re:SPEC is open source

[VAXman]

Since the P4 creams Athlon in SPEC (both Int and FP), that means it's not good enough for Van/AMD.

How about some new (un)real world benchmarks?

[peteran]

I was thinking on the lines of implementing a new set of benchmarks that are heavily optimized for the specific processor they run on. In the lines of writing an office benchmark optimized for Athlons/PIII/P4/AMR, etc. and counting highest achieved words-per-minute or other productivity related metric.

Seriously though, writing a set of benchmarks that fully exploit the possibilities of a specific processor would be something that I would find interesting.

Somebody Else's Problem field

[fermion]

This reminds me of Douglas Adams Somebody Else's Problem field. It ran for a long time on 9V batter, IIRC. Following the same principle, tell the people what they want to believe, back it up with contrived numbers, and you can take the money and run.

SysMark 2001 and 2002 obfuscate their specific tasks by spitting out one "dumb" number as a score for an entire suite of tests. This makes it impossible to know the relative performance of CPUs on individual tasks.
Which again shows that if people accept a single unexplained statistic as fact, they tend get what they deserve. This reminds me of a mantra from my physics lab: a single number is a guess.

Furthermore, why do we believe that there is a typical 'real world' set of problems that can be benchmarked. Even within a single application, there may be a mode work profile, but there is enough variability that a wide range of profiles would be defensible. The trick is to choose the one that makes your product look good.

No, just nonsense.

[fmaxwell]

If AMD would stick to making totally Intel compatible chips instead of trying to infuse their own personality, we wouldn't have this problem. Hint: my software shouldn't need to know it's running on an AMD chip.

This is so wrong on so many counts...

1. Intel's chips aren't "totally Intel compatible". The Pentium 4 contains instructions that were not present in the Pentium, P2, and P3. Why should your software have to "know it's running on a" Pentium 4 rather than a P3, P2, or Pentium? Hell, there was even a Pentium and a Pentium MMX (the latter adding the MMX instructions).

2. Intel tries every trick possible to patent their instructions to prevent people from implementing them. They do it with hardware, too. Remember when you could plug a K6-2 in place of an Intel Socket 7 CPU? Starting with Slot 1, intel used patents to prevent others from making compatible CPUs, which is why AMD and Intel motherboards are now incompatible.

3. Why should AMD not provide useful processor extensions that improve on Intel's base instructions? That's what provides useful competition and makes the industry grow.

4. What interest do you have in seeing AMD in a constant catch-up mode? In your scenario, Intel gets an advantage every time they release new instructions -- that will take AMD months to implement in silicon. Do you own Intel stock?

5. Why doesn't Intel just stick to providing processors that are 'totally AMD compatible'?

Re:No, just nonsense.

[Anonymous Coward]

Maybe you're missing something with the words "Intel Compatible"... to the best of my knowledge, saying "Intel Compatible" does not mean you have to use the exact same architecture from one processor family to the other, as you seem to be insisting here...

Intel P4's run the same code a Pentium 60 ran way back when, no recompiling needed, sure you can optimize the compile, but you don't HAVE to.

Anyway, that's my piece...

Re:No, just nonsense.

[Anonymous Coward]

And what do AMD chips not run, that a Pentium 60 would run?

Re:No, just nonsense.

[fmaxwell]

Maybe you're missing something with the words "Intel Compatible"...

No, I am simply more intelligent than you are. The original poster was claiming that AMD should never add extensions to the Intel base instruction set and should support all Intel instructions -- including those for which Intel has patents, leaving AMD open to legal action.

Intel P4's run the same code a Pentium 60 ran way back when, no recompiling needed, sure you can optimize the compile, but you don't HAVE to.

So are you saying that AMD Athlon's cannot run the code written for a Pentium 60? If so, to what code are you referring?

We have a situation now where we have the AMD Athlon series and the Intel Pentium 4 series. Both have added extensions relative to the original Pentium. Are you claiming that benchmarks should only support Intel's additional instructions? What's your point?

Re:No, just nonsense.

[Anonymous Coward]

I own Intel stock and all my boxes at home are Athlons.

Maybe I'm bipolar. .anonymouscoward.

Re:No, just nonsense.

[Effugas]

There's an interesting semantic argument to the claim that it's conceptually impossible to patent an instruction: After all, you are being told to do something, which inherently specifies that which you are being told to do.

Wrap your mind around that for a second.

--Dan

Re:No, just nonsense.

[Anonymous Coward]

2. Intel tries every trick possible to patent their instructions to prevent people from implementing them. They do it with hardware, too. Remember when you could plug a K6-2 in place of an Intel Socket 7 CPU? Starting with Slot 1, intel used patents to prevent others from making compatible CPUs, which is why AMD and Intel motherboards are now incompatible.

Actually, AMD licensed Intel's Slot 1 technology and then went on to make a variation of it that wasn't compatible. I don't remember why they did that, except maybe that Athlon CPUs had different power requirements from P2s. They even use the same connectors, except AMD's is keyed backwards relative to Intel's to prevent people from plugging in the wrong CPUs and frying them.

Also, Cyrix (back before becoming a part of Via) licensed Intel's socket interface so you could plug their chips into P3 motherboards. It didn't help, though, that their CPUs sucked massively for almost everything besides embedded apps.

Re:No, just nonsense.

[Hoser McMoose]

Above and beyond what the original poster wrote, take a look sometime at the "errata" or "specification update" sheets that both Intel and AMD document.

These errata sheets describe all the ways in which various processors are not compatible with the IA-32 specification. Last I checked, the P4 had 50-60 documentated incompatabilities, while the PIII had over 80. These are specifically documented situations where Intel is NOT "Intel compatible". The Athlon, meanwhile, had about 15-20.

I'll leave it as an exercise to the reader to figure out if AMD is actually MORE Intel compatible than Intel is, or if Intel just documents their incompatabilities better then AMD.

Ohh, as for point 2 above, the issue was not so much one of Intel patenting Slot 1. AMD's first Athlons used a "Slot A", which was actually just a Slot 1 connector flipped backwards. The real issue though was that Intel had patented the GTL+ bus used for PPro, PII and PIII processors. Part of the legal agreement between AMD and Intel after the 386 and 486 battle was that AMD was NOT allowed to use any Intel bus designs beyond the Pentium bus. The socket/slot used by Intel was immaterial, it was the bus protocol that AMD could not use.

Re:No, just nonsense.

[Hoser McMoose]

AMD never licensed Slot 1, what they did is they "developed" a new slot that was physically identical but flipped around backwards. They called it Slot A.

AMD could NOT use the same motherboard, regarldess of the slot use. The connector was actually of no consequence at all, it was all in the bus protocol used, which Intel would NOT license to AMD at all.

As for Cyrix, they never had a license either, they just used the P3 socket all on their own. They claimed (and still are claiming) that a cross-licensing agreement through their various owners allows them to make use of the GTL+ bus protocol that was used with the PPro, PII and PIII.

Re:No, just nonsense.

[fmaxwell]

Ohh, as for point 2 above, the issue was not so much one of Intel patenting Slot 1. AMD's first Athlons used a "Slot A", which was actually just a Slot 1 connector flipped backwards.

When I referred to the patent of Slot 1, I was referring to the electrical interface, not the physical one.

Other than that minor misunderstanding, your post was insightful and informative. I'm sorry that I don't have moderator points today.

Re:No, just nonsense.

[fmaxwell]

It didn't help, though, that their CPUs sucked massively for almost everything besides embedded apps.

Really an unfair characterization of the C3 processors. They are actually very good in any application where you want low power and reasonable performance. Sure, they suck for playing Quake III, running SETI@home, or ripping DVD to Divx, but that's not what most people do. The majority of people surf the web, do some word processing, exchange e-mail, do some instant messaging, and maybe play some simple games that shipped with Windows. You simply don't need a monstrously powerful CPU for that. Another place they rock is in web servers, FTP servers, etc. The ability to run them with a passive heatsink and no fan means one less point of failure to take down the server.

I never understood benchmarks

[Proc6]

Benchmarks show how fast a given CPU can run that benchmark, which may, but probably will not reflect the kind of computing you're doing.

If you have a computing need that places such a critical importance on speed of CPU that even a few % difference matters, the only real benchmark you should be running is YOUR application under both CPUs.

If you do not have a need like this, and you are just doing general computing, and you can't tell the difference between an Athlon 1600+ and an Intel Pentium 4 1.6, just by using it, then who really cares?

Re:I never understood benchmarks

[Anonymous Coward]

But will youi have to BUY both the Intel AND the AMD part to find out?

An expensive alternative to looking at a benchmark, I think...

Who cares?

[Anonymous Coward]

inexperienced unix sysadmins talk benchmarks, performance, disk space, and bandwidth.
Experienced unix sysadmins talk printer compatibility.

I'm no impressed

[AaronPSU79]

All the data in Van's article came from AMD, so what do you expect?

SPECquake

[ehiris]

Besides SPEC Benchmarks I don't trust any other ones.

They need a SPECquake.

No.

[FreeUser]

Wouldn't a better CPU benchmarks be taken by using the chipmakers' own compilers?

No.

The chipmaker would simply then optimize their compiler for the benchmark(s) in question, rather than for code more generally. In other words, what you suggest would still allow the chipmaker to cheat.

In order to have complete transparency in the benchmarking, both the benchmarks and the compiler should be open source (ideally free software, so that anyone can run and verify the benchmarks as well, allowing repeatable experimentation in the broadest scientific sense). If the chip maker wishes to submit optimizations to such a compiler they would be free to do so, since any such optimizations would in turn be open source (or free software) and subject to peer review.

A good candidate would be gcc, which runs on numerous platforms, and on several operating systems on AMD and Intel hardware.

Cheating would be much harder in this case, perhaps even impossible, something we need given the sordid history of benchmarking by all parties involved (except perhaps AMD? Can anyone recall an instance where AMD has cooked results? I ask because their current chip rating system is extremely conservative ... almost the antithes of what Intel is trying to do. Has this been a longstanding strategy on AMD's part?).

Re:No.

[cnvogel]

I think it's not so easy, at least in the world that we live in.

My naive idea about how chip-features are designed is that the hardware people meet the software people and a discussion goes like this:

We could make a longer pipeline or add more registers or whatnot.... for about the same money/silicon-real-estate/complexity. When would our users and their applications benefit the most.

So the chip is designed with a specific compiler in mind (or at least by people who will have a specific idea of optimizing for a compiler) which in turn is optimized for a specific workload.

And then when you are asked about producing a "real world benchmark" you will most likely create a environment which meets the criteria that went in the design of the compiler which in turn matches your CPU very well..

So I would not rule out that Intel tried to cheat, or at least tried to have a benchmark that matches their CPU best but much of this comes from the way the processors are designed.

[Of course, that's just my naive Idea about the process and I will accept a paid-for trip to intel-HQ with labs-visit and expensive dinner to get a more objective oppinion about that... ;-) ]

Re:No.

[drinkypoo]

gcc would be a shitty choice for your benchmark because so much more optimization goes on for particular CPUs than for others. It might be effective in benchmarking between intel-x86 and amd-x86 but when you got to sparc, say, it would be horrible.

Application benchmarks use (supposedly) the same code as the applications which they mimic. When this is the case, they should use the same compiler which the application developer uses. Anything else is wankery. This is the only way to give an accurate projection of CPU use.

CPU benchmarks should be written in assembler and optimized for each CPU. The idea is to show the ability of the CPU. Then they should ALSO contain a general-purpose benchmark written in C (or similar) which will show what most applications will actually get out of the CPU. For THAT portion, gcc would be a reasonable choice, but still not a good one. I don't know about today, but five years ago most commercial software for Solaris/Sparc was written using the SUNSpro compiler, not gcc, because the code it turned out was dramatically faster than that which gcc produced. On architectures on which gcc is weak, it doesn't make much sense to use gcc for a benchmark.

Re:No.

[BWS]

Their PR Rating (back in the Pentium Days) is the biggest piece of BullShit there ever was..

Re:No.

[Anonymous Coward]

That was Cyrix not AMD, Look it up if you don't believe me.

Re:No.

[Dun Malg]

Their PR Rating (back in the Pentium Days) is the biggest piece of BullShit there ever was..

No, Cyrix's PR Rating was bullshit. AMD marked their CPUs in the normal way back then.

Re:No.

[BWS]

you're wrong.. it was Cyris that started the PR Rating but AMD joined

Re:No.

[Dun Malg]

you're wrong.. it was Cyris that started the PR Rating but AMD joined

Dear obstinant fool: if that be the case, then where pray tell to I find the PR rating on this AMD K6-2 processor from 1998 I currently hold in my hand? There isn't one you obtuse buffoon! In the days of the Pentium-clone processors, only Cyrix used the "Pentium equivalent" Rating. AMD marked their CPUs based on the clock speed. The CPU I have here says: "AMD-K6-2/366AFR". Its clock speed is 366mHz. No PR rating. Just clock speed. Since you seem to be so sure that AMD used the PR rating, please produce for me the markings for just such a CPU. Surely you can find a simple string of characters like that, yes? ignoramus.

Yes, both.

[jhines]

The source should be published, and compiled with a standard (GCC) compiler.

The CPU vendor, or any other compiler vendor can compile the same code, and publish the benchmark as well.

This then opens up the competitive market in compilers.

What I'd like to see is the same code, compled with cross cpu options, IE what happens when the code optomizes for Intel but runs on AMD. How much does that penalize you, useing the wrong optimizations?

As long as the compiler, and options used are disclosed, I don't see a problem.

Come ON people.....

[byran lei]

Outside of the total lusers otherwise known as PC Gamers, is there actually anybody left who take benchmark testing seriously?

Re:Come ON people.....

[Oswald]

...lusers...

This is how you spell "losers," and you expect people to care what you think? Worse yet, your thinking is flawed: even though they are some of the most demanding users, gamers are rarely guilty of buying a processor based on arbitrary cpu benchmarks. They usually seek computers capable of fast, high-quality 3D graphics, and buy based on what's giving the best results in the games they actually play.

Deja Vu - the 286 is comparable to the 68000

[tz]

Back in the mid 1980s, there was a rivalry between intel and motorola on embedded processors.

Part of it was the 286 v.s. the 68000 (which had been around), and Motorola produced a long missive explaining the problem with (64k) segments when trying to do anything useful.

This was in response to benchmarks showing that the 286 was not horribly slower and some cases faster than the 68000.

Nice to know nothing has changed in well over a decade.

The "Ultimate" Benchmarking

[Anonymous Coward]

Since about 1.5 years ago, I have been using all AMDs. No more Intel chips in my house and my business. The decision often boils down to whether I should go with MP or stay with XP.

"I" am the ultimate benchmarker.

like statistics benchmarks are lies

[ToasterTester]

All benchmarks favor whoever requested they be written. So its a crap shoot, choose the one you want to believe in, then go do you own tests, Your own testing is the only tests that matter.

The fastest processor:

[jasonditz]

Why should we even care at this point what the fastest processor is? Except for those of us doing long complex mathematical computations is there really any use for those extra couple of Ghz?

It strikes me that while processors have been getting faster and faster over the past couple of years the software they run hasn't been getting better. My box only runs at 900 Mhz, and I can't tell you the last time I was using more than 20% of the processor. The rest is all wasted (or lying in wait for that next dreaded compile). Truth be told it doesn't seem to run a helluva lot different from my 300 Mhz system (except that its got a nicer video card).

Since the new high-end apps are so much more dependant on video card specs than CPU specs, why should the average person bother to get the latest and greatest CPU anymore?

And even more importantly: Why do I know I will buy AMD's new Hammer processor when it comes out?

Java benchmark

[iamacat]

Each vendor should release a Java VM that they optimize using their own compiler, hand coded assembler and any dirty trick they can think about, including hardware implementation of bytecode. If I can then take real-life Java apps and they actually run faster on a given CPU, the results are accurate - at least for Java apps.

With this method, I don't have to recompile every app used in the benchmark for each CPU and can be still sure that their execution is optimized in every way the vendor can think about.

Intel/AMD architecture bashing

[edwinolson]

I've read a lot of architecture bashing. "P4's 8K L1 is too small to be useful." "AMD's huge L2 is too slow to be useful."

Both of these companies spend *billions* of dollars on producing these processors. Both companies run lots of simulations to determine what design choices best fit with the rest of their design. When you're spending that much time and money developing these CPUs, you can't afford NOT to consider every option.

When it comes down to it, both AMD and Intel have really good engineers, and both companies listen to them when figuring out how to build cpus.

So consider that the P4's 8KB L1 trace cache is so small because that's as big as they could make it while keeping the latency down to 2 cycles-- something that was critical to keeping their double pumped ALUs busy (and thus their IPC up as much as they can)-- and that they could compensate by working a bit harder on a fast L2.

Perhaps AMD decided that they could live with an extr cycle of latency in the L1 because they have enough instrucions in-flight that blocking on a cache access wouldn't hurt them as much as a low hit rate would.

Or, perhaps there are multiple sweet spots in size/hitrate, especially when you factor in die size and cost. Honestly, I don't know the reasons why they made these decisions, and I'd love to find out why-- but I have 100% confidence that all the options were carefully considered.

When it comes down to it, both architectures are performing really well! And for YEARS, they have been competitive with each other. So while you may have your favorite (I, for example, think the P4 SMT and trace cache stuff is pretty neat), you've got to realize that zealously promoting one over another just makes you look silly.

Cheers!

-Ed

Re:Big deal (IPC anyone?)

[tiger_66_y2k]

Hello!? IPC anyone. All that I have to say is that due to the Athlon XP's superior IPC (Instructions Per Clock) the Intel P4's have to be roughly 400MHz faster to be equivilant in real world performance.

Oh, and for those who don't already know the IPC on P4's is 6, while the IPC on an Athlon XP is 9. So theoretically this makes the Athlons 33% faster. Mind you their are many other factors that come into play but here is another one for you.

When Intel released the P4's and realized that they performed worse than the equivilant P3's Intel decided to bump up the FSB & RAM specs, on-die cache and even changed the socked design. To me this seems like Intel was desperate to feel as though they hadn't been wasting their time on a chip that didn't perform as well as its predicessors.

Yet another tid-bit, Intel is up to 266MHz FSB double pumped (effectively 533MHz FSB) and AMD, due to good design and engineering, has been able to keep up/surpass them using half of that. AMD Athlon XP's FSB is 133MHz double pumped (evectively 266MHz) and they are just now to the point where they need to bump it up again.

One last example of the superior engineering at AMD is the Socket 7 design (okay, who all remembers here...). Intel said that it was a useless design after the ~233MHz Pentiums, but AMD lasted competitively using the same socket design until they quit at a whopping 550Mhz. Now, THAT is engineering my friends.

----
I have no sig.

Why you don't trust benchmarks

[MoneyT]

I never have and I never will trust benchmarks. Just like I will never trust any specs that a company puts out untill I try the machine for myself. It's one of the reasons I use Apple computers. As I type this, I'm on a custom built PC, with an AMD Athlon XP 2000 processor. It has 512 MB of memory, an Abit KR7A-133 mo-bo and plenty of other bells and wistles. Now theoreticaly speaking, this computer should be a hell of a lot faster than my 300Mhz iBook. Yet for everything except playing UT, thsi computer seems either equivilent to or sometimes (especialy right after startup) slower than the iBook. My point is, don't trust the numbers, use it yourself and decide for yourself. You're the one using the computer, and 90% o fusing a computer is how it feels to you.

Re:Why you don't trust benchmarks

[Anonymous Coward]

I'm surprised you picked the KR7A-133. If my memory serves me correctly that board uses SDRAM at 133 mhz, which means that your super-duper XP 2000 is choking on limited memory bandwidth. XP's were built for DDR in the same way P4's were built for RAMBUS. I'd also ask what OS you are using and what background processes are hogging your resources.

Re:Why you don't trust benchmarks

[Anonymous Coward]

Awww C'mon, what a bunch of bullshit. Boy haven't you learned that subjective views are just that.... SUBJECTIVE.

Re:Why you don't trust benchmarks

[MoneyT]

Your memory served you wrong then, the motherboard has 4 DDR slots, and of course I'm using DDR RAM

Re:Why you don't trust benchmarks

[MoneyT]

Subjective views may be subjective, but that's what determines the real world. Take for example a scientist, who works out a formula for a chemical reaction. ON paper, the reaction should do what he predicts, so then he does the experiment and discovers that it did do what he predicted. If no other scientist can duplicate his results, his findings are considered invalid.

For me it's the same with computers, you can tell me all you want that it's going to run faster and faster, but untill I see it, I'm not going to believe it.

Check the Math.

[Anonymous Coward]

[The Pentium 4] has 8K of L1 cache... whereas the Athlon has that amount squared and doubled (128K).

8K squared is 64M, not 64K.

But wait, it gets worse!

8K bytes squared is 64M "square bytes." Just like 8K feet squared is 64M square feet. And a square byte would be 64 square bits?

You'd have done better to say, "The Athlon has 16 times more cache..."

Yeah, I know, I'm nitpicking. The devil is in the details.

Re:Big deal (IPC anyone?)

[weewilly]

Um that should read that Intel is up to 133mhz quad-pumped

I told you all

[Sabalon]

I've been saying that as software gets optimized for the P4, it'll start seeming to be the faster processor - Look at Lighwave 7, 3dsmax 4.2sp1, and now this. :)

if(cputype==AMD) {
sleep(1);
}

Speed isn't everything

[Billly Gates]

I bought 3 athlonXP motherboards and an athlonXP1800 last spring. I threw out every single motherboard and the cpu is in my closet. I have downgraded back to my pentiumIII. Why?

I had major stability problems and the athlon burns too hot. First I relized my old powersupply did not have the power to even boot these babies(300 w), I first had to buy a new powersupply, then new ram, then a new case. After this my first motherboard would boot and then crash after it was done posting. I returned it for an abit which was extremely buggy, and my netgear nics wouldn't work, it blue screened multiple times whenever I used my geforce(well known bug), and the apic controller did weird things in linux and sometimes would not shut down properly. It then died totally 2 weeks later. It was exchanged for an msi board that supprisingly worked as expected, well at first. The system would not shutdown properly under linux but I didn't care that much. I then noticed the cpu kept reaching above 45c which could damage the chip over time. A third problem was whenever I enabled the nvidia opengl patch the system would crash quite often. The problem went away if I disabled it. I built the drivers from scratch so it was the right version for my kernel compilied from source. I then spent $60 for the top of the line cpu cooler. I had to use some force to get it to close on the cpu and my screwdriver flew off the cooler and damaged a chip on the board!

The guys at the computer shop said they would only exchange my msi for another exact ones and they were becoming agravaited at me for obvious reasons. I was so angry I just said f*ck this sh*t and didn't bother to replace my motherboard. Even if I get the board replaced, I am faced with the same problems and bugs! I admit I broke the last motherboard and it was totally my fault and not AMD's but after a month and $750 dollars later, I did not care. DDR ram only worked with athlons at the time and I now had two cases and powersupplies which I did not need. I felt like a sucker who just flused $750 dollars down the toilet when I downgraded back. Why should I have to put up with that crap? Anyway the first board was probably defective and the second should of not even left the manufactoring plant. I did look up my bugs online and there were many pissed off consumers who had the same problems with the same exact sets of hardware, so the abit one is a piece of sh*t. Many early chipsets for the athlon processors are buggy. Especially VIA's. There is even a well known linux/nvidia/amd bug that can crash your system if you do any opengl which plauged my msi system.

Are there stable bugfree wonderfull athlon boards out there? Yes I am sure there are. I am not trying to start a flamewar here but rather just share my experience with them. I am thinking about finally getting rid of my pentiumIII. This time with a guinine intel processor.

Is it the fastest or cheapset? No. Do I care? no. I want something close to the fastest that will work with my case, work with my power supply, work with all of my perihperals, and be reliable. All the benchmarking websites do is show how fast the chip is. I want to know how reliable it is. I can find some vendor references to overall reliablity but the boards vary from chipset to chipset. If I put down big bucks to upgrade my system it better well work and keep working for a long time! There is a similiar arguement about buying an expensive sun box over a cheap lintel one. You get what you pay for. This is why many bussinesses choose intel overwhemingly over AMD.

Most of the reliablity problems which plagued the first generations of athlon processors are gone and I admit the first intel 810 chipsets were terrible but there are less bugs with intel chipsets overall. I am willing to spend $250 more this time and feel at ease and look forward to use it for a long time.

Re:Speed isn't everything

[Anonymous Coward]

On the other hand, when I was looking for a new motherboard and processor, I did some research first, read some reviews, and bought a board with the chipset (a VIA one actually) that was the most stable.

And I have never had a problem with it.

Re:Speed isn't everything

[l33t-gu3lph1t3]

Any computer user worth their salt can build an AMD or an Intel system. I have built P2/P3/P4/Athlon/Athlon(socketA)/AthlonXP systems, and I got each one with my first try. I avoid VIA motherboards and use *COMMON SENSE* and I have done very well. Anyone who whines about instability in computer components these days doesn't know WTF they are doing. Except for maybe VIA errata ;)

Re:Speed isn't everything

[RazzleDazzle]

Sounds like your problem is more yourself than anything else. Don't fall victim to media hype or stupid "computer geek's" hype. Maybe you need to read reviews and see what other people say before you rush into a *brand new* product. tomshardware.com is a great hardware review site. I like via chipsets and have never had problems with them before. I also like Soyo motherboards. I have had 3 soyos, and my friends/family have a combined total of 6 soyo dragon motherboards. That is the kind of info you want/need before you spend major money. I will never ever use Intel again. I do my research on quality, speed, scalability and manufacturer. Would you buy a car from me if I told you it could accelerate more quickly than any other car out there?

Re:Speed isn't everything

[Billly Gates]

"Sounds like your problem is more yourself than anything else....you need to read reviews .....I also like Soyo motherboards. I have had 3 soyos...... I do my research on quality, speed, scalability and manufacturer"

How are your Soyo's. Well I am sure there are really nothing wrong with them.

I use to own a soyo myself back in the pentium1 days. They are good no more. Abit and Asus are top of the line in terms of board reliability and yes I did do my homework to know this. Chaintek, shuttle and soyo are the worst. Intel is king when it comes to reliability but there were no amd chipsets for them obviously. Sorry but you get what you pay for. I actually have done pc repair work before and know how to build a basic machine. Quality has gone down in recent years and asus and abit and perhaps soyo at one time did make great stable products. Now because of the chip race between AMD and Intel they finish with half baked bios and no real R&D before they mass produce the boards. My specific problems were later discovered by improper resistors being used in my last msi board as well as the abit one. When a geforece first starts up it loads a whole bunch of power and the resistor does not like the spike. Sorry but I should expect my hardware to work without any bugs. VIA after 2 years is just finally shaking off the bugs which is uneccaptable. Dell and IBM gave the thumbs down an athlons for a good reason. Reliability was too low.

Re:Speed isn't everything

[Anonymous Coward]

The box I'm sitting at was my first build. Soyo Dragon Plus, using the Via KT266A chipset with PC2100 and an XP 1900, in an Antec case with a standard 300w power supply. I had no problems building it and I've had no problems running it- no bsod's, no crashes, nothing. If anything, I think for the $200 I paid for my processor and mobo I've got the upper hand over anyone with a P4 system :)

Re:Speed isn't everything

[NoodleSlayer]

Athlons can run up to 80c within spec, they are quite hardy and most likely you'll end up throwing the machine out long before that piece of silicon burns out.

Re:Big deal (IPC anyone?)

[Anonymous Coward]

Intel P4's have to be roughly 400MHz faster to be equivilant in real world performance.

Hello!? Intel's P4's are around 600 MHz faster than the fastest Athlon. That's why the 2.8 GHz P4 still beats the Athlon 2600+ in a majority of the benchmarks.

Intel's P4 philosophy of increasing the pipeline has turned out to be pretty smart. Intel has jacked up their clock frequency almost at will all year, and they don't have any big roadblocks in the way up to 4+ GHz. AMD on the other hand has been struggling. It took a kludge for them to get the Athlon over 2 GHz. Intel took a hit on IPC, but it has allowed them to increase the clock speeds dramatically.

And you comments about AMD's FSB "keeping up/surpassing" Intel's are pretty dumb. Take a look at any memory performance benchmark and you will see how AMD's slow memory bandwidth is killing them.

Blah blah blah...

[Anonymous Coward]

Yes, AMD chips are faster mhz per mhz. That's what happens when you pump way too much power into a chip and damn near blow the thing up...it runs fast...for a while.

Pentium 4 is a BMW, refined, smooth, and fast. AMD is a turbocharged Honda...runs way too hot, unstable, but is fast as hell until it finally burns up.

Just pick your CPU of choice and shut the hell up about it!!!!

Re:Blah blah blah...

[Bobartig]

Pentium 4 is a BMW, refined, smooth, and fast

don't forget, that BMW, fast as it may be, has tires the size of a quarter, and only holds 6 oz. of gas at a time. P4's do run much cooler than Athlons, but I dont' know how refined that makes you.

Q: Compiler rigged for benchmarks...

[cant_get_a_good_nick]

Which was the compiler company that wrote into it's compiler the ability to recognize a common benchmark that didn't require output, and just converted it to NOOPs? Wow, did that computer ever chomp on those NOOPs fast...

Benchmarks measure speed on benchmark code. It's like horsepower in a car - a car with 300 horsepower isn't necessarily faster than one with 280, or even 200. It just depends, man.

Yawn

[Sebastopol]

Is Intel forcing anyone to use BapCo benchmarks? Aren't there dozens of websites out there with their own? Just look at any chip review on Tom's Hardware or Anandtech, Sysmark is only one of many benchmarks used.

Is this a surprise? Are people actually "outraged"? Please. Are you trying to tell me AMD has never hand-picked their own benchmarks?

Any CPU manufacturer will pick the benchmark that makes them look best but try to play it down, and then use that score extensively as propaganda.

Just look at when Apple used Bytemark to claim a 400% performance boost over intel chips -- of course, they used the 486-compiled version on a P6 core, but they didn't let that little nugget of disniformation get in their way.

Intel, AMD, Apple -- all whores.

nonsense

[RelliK]

SPECfp benchmark is highly dependent on the memory bandwidth. P4, with two channels of RDRAM, has more memory bandwidth than Athlon with a single channel of DDR. Thus, P4 gets a free ride here. Show me an apples to apples comparison here: P4 and Athlon with the same memory (be it SDRAM or DDR). I have yet to see one.

For a truly FPU-intensive benchmark that does not depend on memory bandwidth much, check out 3D Sudio rendering (tomshardware & others have it). Athlon simply smokes P4 on that one. Incidentally, on this benchmark P3 1000MHz outperformed P4 1500MHz.

Re:nonsense

[Anonymous Coward]

Show me an apples to apples comparison here: P4 and Athlon with the same memory (be it SDRAM or DDR). I have yet to see one.

What you are asking for is a crippled P4 system. Don't hate the P4 just because it can handle more memory bandwidth. If it were possible to give an Athlon two channels of RDRAM, that would be interesting to see, but obviously the architecture cannot support it.

This is not a question of fairness, it is simply a design decision that AMD made a few years ago. Athlon is still the best value on a high-performance CPU. But you have to admit, the fastest x86 system on the market is the P4. Maybe that will change when Hammer comes out, but Intel's offering is currently fastest. (You sure have to pay for it, though.)

don't mind him

[RelliK]

he's just trolling for his employer. He always does that.

corrections

[RelliK]

The Pentium IV's really looong pipeline does allow the P4 to run at higher clockspeeds, but the branch prediction you mentioned is instant death.... a single branch prediction requires up to 20 full clock cycles of work to be discarded.

The situation is not quite as dire due to P4's trace cache (you actually addressed that later in your post). Nevertheless, your point stands.

On Intel SMP setups, even on P4 Xeons (Which, IMO, are inferior to P3 Tualatin chips by the same company) when one CPU accesses main memory, it locks main memory for the other CPUs. All other CPUs have to sit and twiddle their transistors while the main memory is on use by only one CPU. On AMD SMP setups, ALL processors can simultaneous access memory, merely sharing the bandwidth simultaneously. So, if one CPU is only using 100MB of memory bandwidth, the rest can be used by other CPUs at that time.

P4 Xeons (as well as P3s) have a shared memory bus. That is, multiple CPUs share the bandwidth of the 400MHz or 533MHz bus when accessing memory. However, Athlon has a point-to-point channel for each CPU. That is, each Athlon CPU has the full bandwidth of the 266MHz (soon to be 333MHz) memory bus, regardless of how many CPUs there are in the system. This means that beyond 2-way SMP systems, Athlon has a significant advantage in memory bandwidth over P4.

What a boatload of crap

[Anonymous Coward]

Its amazing to me how much horsecrap gets thrown around.

Here it is folks...i've worked with BAPCO...I actually know something!

It is NOT headquartered at an Intel facility. The sysmark benchmark is NOT written by Intel engineers. The tools are not intentionally written to "favor" Intel and show poorly for AMD.

BAPCO consists of dozens of companies who offer up representatives to determine, develop, code and distribute their benchmark tools. There are numerous processes in place to assure that nobody "owns" or "manipulates" the results. I've worked with BAPCO reps from a number of companies, you'd be better off calling their mothers a name than suggesting they arent interested in benchmark purism.

So enjoy the conspiracy theories, but 98% of what I've read here is complete and utter crapola.

If AMD want BAPCO tools to do something special for them, they oughta contribute the manpower and money to the effort other companies do. Or shut the hell up. And that goes for their fanboy sites like Vans as well.

Re:What a boatload of crap

[Anonymous Coward]

Its amazing to me how much horsecrap gets thrown around.

Here it is folks...i've worked with BAPCO...I actually know something!

It is NOT headquartered at an AMD facility. The sysmark benchmark is NOT written by AMD engineers. The tools are not intentionally written to "favor" AMD and show poorly for Intel.

BAPCO consists of dozens of companies who offer up representatives to determine, develop, code and distribute their benchmark tools. There are numerous processes in place to assure that nobody "owns" or "manipulates" the results. I've worked with BAPCO reps from a number of companies, you'd be better off calling their mothers a name than suggesting they arent interested in benchmark purism.

So enjoy the conspiracy theories, but 98% of what I've read here is complete and utter crapola.

If Intel want BAPCO tools to do something special for them, they oughta contribute the manpower and money to the effort other companies do. Or shut the hell up. And that goes for their fanboy sites like Vans as well.

Re:Sour Grapes

[Anonymous Coward]

It's diappointing that Van didn't do much more than comment on a AMD PR release. Van has been saying that something was up with Sysmark 2001 and 2002 for quite some time now, I think its more of a case that since AMD joined Bapco(?) and now has access to the test criteria of Sysmark, Van finally has the proof he has been unable to obtain by himself.

With people stealing articles from him (As far as I am concerned), I think he was just in a little bit of a rush in releasing information which justifies his opinion.

On the other hand I have a sour taste in my mouth after reading Kyle's comments. No-where did Van claim the results weren't from AMD. Kyle should have kept his comments to himself. I have given up on Tom's Hardware, and was just statrting to read HardOCP, but looks like I will be limited to AnandTech and TechReport now. What a shame.

males running the show

[zoftie]

what this reminds me of, is of natural competition:
"my penis is bigger then yours, because I have a bigger car". It is faulty logic, as many do agree nowadays, but it does not stop people from repeating the excersize. Similarly here, benchmarks are nothing but fictitious statement, taking some program that is written by people(source of most errors in logic), and ouput of it as good indicator of what cpu is good at. Nevermind chipsets, auxiliary hardware and many other things. If people who buy computers in large quantities would take these benchmarks for any guilding value, they are to be called suckers. Those with real brains, will take two instances of same cpu put them into same box, boot the computer and let two computers do the same thing many computers will do, when purchased in larger quantity. If you don't do this, you would be 'easy' to be taken advantage of, by computer businesses.
2 canadian cents.

Learn to count.

[Anonymous Coward]

> It has 8K of L1 cache, the same amount found in the ancient 486 processor, whereas the Athlon has that amount squared and doubled (128K)

Ridiculous (same kind of sh*t that we have in benchmarks).

8K = 65536 bits. 128K = 1048576 bits.

So 8Kbits^2 * 2 = 1 Gigabyte of cache.

Troll?

[Anonymous Coward]

Dear Slashdot moderator: You are a fuckhead.

Tomshardware attacks integrity of vanshardware

[geoswan]

There have been some more developments around Van's Smith's review of the Bapco benchmark's. Tom Pabt's, the owner of tomshardware.com, has written an editorial condemning the journalistic integrity of Smith, and Kyle Bennett of hardocp.com.

Here are part 1 , part 2 , part 3 , part 4 and part 5 . Pabst's accusation is that Smith and Bennett have both written articles where they claimed to have discovered flaws in the benchmarks that make one manufacturer's product look good, when they were really being coached by the that manufacturer's rivals.

Here is Smith's rebuttal .

Van Smith used to work for Tom Pabst. In my opinion the quality and utility of tomshardware.com has gone down since Van Smith departed.

And, about this fight, I would say that Dr Pabst (he is an MD) hasn't learned the value of civility. In my opinion, in a fight like this one, people can't really follow the details, so they base their assessment of who is right, by looking to see who remains more civil.

It's time to wake up and watch out benchmarks!

[Anonymous Coward]

http://www.aceshardware.com/read_news.jsp?id=55000 490