Slashdot Mirror

I think you may be on to something. Apple's decision will really tell a lot about their plans for the future. They've made good money as a niche player, offering compelling performance for professionals in music, video, and publishing. They've also appealed to conspicuous consumers who wanted something "different" from the typical Wintel offering. Finally, they've also appealed to those wanting greater ease of use. That market has really deteriorated as those people tend to also want something cheap (see the educational market.) Their choice of processor could signal either a move to solidify one or more of these markets, or an attempt for more popular appeal.

Itanium would be an excellent choice to continue to appeal to the high end professional crowd. It has always been high on raw performance, just check out some benchamrks. It was mostly a victim of bad timing. It came along to challenge Big (UNIX) Iron from Sun and IBM, but it did this during a time when smaller servers running Linux were displacing those same servers. Still it's not hard to imagine some dual-CPU Itanium2's with properly compiled versions of Photoshop, Maya, Quark, etc. really performing admirably. And when it comes to laptops, Itanium can be used there too. Intel has been working on the Millington core, a low voltage version of Itanium designed to run on blade servers. Blade servers have to run cool, just like laptops.

On the other hand, if Apple really just wants to focus on mass appeal, expanding their "different" niche, then they are better off going for cheaper chips, a la the Pentium family. Maybe dual core P4s for desktops and Pentium M's for laptops? Certainly their success with the iPod and Mini, plus their never ending attempt to publicize "switchers" suggest that this is what Apple really wants. Maybe the Mini has shown them that they can sell a lot of computers if they just lower their prices.

Reading you link...

Unfortunately the ASF rejected the work presented here and SGI terminated the project. It has a new home now on SourceForge, courtesy of SGI, and it seeks a new owner and a fresh start. Want Apache to run faster? Want to take a different approach to appealing to the ASF? Perhaps simply presenting the ASF with a different spokesperson would improve the odds of adoption. Apply to the current owner for project ownership or to sponsor me to revive the project.

Nonetheless, this project's aggressive optimizations make Apache/1.3 up to ten times faster and Apache/2.0 up to four times faster on the SPECweb96 benchmark.

And in the FAQ:

12. Will the ASF adopt the patches?
We contributed the patches to the Apache Software Foundation but the ASF has refused to include the patches in future releases of Apache/1.3 or 2.0, citing "unnecessary" typecasts and complication associated with the warning-free 64-bit port, and incompatible license terms with the state-threaded MPM.

Its more a historical summary than a precise review.

Apache was slammed in a ( soon to be infamous ) review done by Mindcraft back in 1999 ( some post-incident slashdot coverage here ) as being slower than IIS at serving static content ; though much faster at content generated on the fly via php/perl/python and the like. The tests were biased heavily in favour of IIS, but after the BS was filtered out and the report distilled down to the bare facts, it was admitted that IIS did indeed have the edge in serving some specific content types. After a bit of grumbling, a few side projects like the tux webserver & kernel httpd were born, then the best ideas and strengths of each were rolled out with apache 2.x, and it has never looked back.

Incidentally, its a perfect example of the strengths of opensource development: critiques even if intended as flamebait or non-constructive criticism might point out legitimate weaknesses -> community is galvanised into action -> weaknesses are repaired -> project is stronger -> profit!

spec.org should have fairly impartial albeit dry results, or you can dig up old apache articles on /. like this or this.

Google is also your friend, post Mindcraft articles and mailing-lists generated a lot of 'how do we speed up apache' type discussion.

you have to go back to 2nd quarter 2003 to find a hardware vendor using IIS. Intrestingly, that vendor was Dell showing off a PowerEdge 2650 running IIS5 as well as Rehat Linux 8 (link http://www.spec.org/web99/results/res2003q2/). On this hardware, IIS perfomed 3% better than RHCA (TUX) on Redhat Linux. So, if speed is what you want, hardwave vendors have given up on IIS about two years ago. If features is what you need, then it seems Apache is the choice. Seems to me, IIS has no role to play any more. There's always a better and cheaper choise elsewhere.

It sounds like you are looking for a way to normalize and compare system (and application?) performance across different hardware platforms.
Say hello to SPEC. This is exactly what the organization was formed for. Take a look at their CPU benchmarks. I know you're looking for more of a snapshot, and less of a benchmark - but I would think SPEC is a good place to start.

Basically, it all comes down to what you need. Personally, for anything and everything I do at this time, Opteron based systems are the best solution for me. In the past, Sun systems have figured prominently in solutions I worked on, because they were the best fit dollarwise for the requirements.

Apple don't release industry standard benchmarks, why?

If CPU speed is irrelevant to processor power, then why do we keep talking about it?

Because the marketing droids need a metric to convince the masses to buy the latest shiny thing. If people used standardized benchmarks on processors (say, SPEC's CINT2000 or CFP2000), then it would be too easy to see the benefits and shortcomings. However, the masses don't really care about any of that, they just want what the TV commercials say is "better."

Just as a would-be car enthusiast thinks that the most important element of an automobile is the engine, and that all performance characteristics are centered upon it, a would-be computer enthusiast would see the CPU (and its clock speed) as the most important element, and the component most worthy of an upgrade whenever possible. However, a wise mechanic, similarly to a wise techie, understands that there are many, many elements to make a machine function optimally. Ignoring an auto's suspension, brakes, aerodynamics, etc. makes an auto less-than-optimal, just as ignoring RAM size and speed, hard disk size and speed, L1 cache size, etc. makes a computer less-than-optimal.

All that being said, however, there's no way that we can expect the unwashed masses to instantly grow a brain and realize that they're being duped by marketing con-artists. We, as the educated, can attempt to educate those close to us, but the truth is that there will most likely always be a separation between the people who can see through the advertising double-speak and those who can't.

For instance:
Hummer commercial: An educated person might see through the double-speak and ask, "What about the gas milage?"
Intel commercial: An educated person might ask, "What about the cache size?"

If CPU speed is irrelevant to processor power, then why do we keep talking about it?

Because the marketing droids need a metric to convince the masses to buy the latest shiny thing. If people used standardized benchmarks on processors (say, SPEC's CINT2000 or CFP2000), then it would be too easy to see the benefits and shortcomings. However, the masses don't really care about any of that, they just want what the TV commercials say is "better."

Just as a would-be car enthusiast thinks that the most important element of an automobile is the engine, and that all performance characteristics are centered upon it, a would-be computer enthusiast would see the CPU (and its clock speed) as the most important element, and the component most worthy of an upgrade whenever possible. However, a wise mechanic, similarly to a wise techie, understands that there are many, many elements to make a machine function optimally. Ignoring an auto's suspension, brakes, aerodynamics, etc. makes an auto less-than-optimal, just as ignoring RAM size and speed, hard disk size and speed, L1 cache size, etc. makes a computer less-than-optimal.

All that being said, however, there's no way that we can expect the unwashed masses to instantly grow a brain and realize that they're being duped by marketing con-artists. We, as the educated, can attempt to educate those close to us, but the truth is that there will most likely always be a separation between the people who can see through the advertising double-speak and those who can't.

For instance:
Hummer commercial: An educated person might see through the double-speak and ask, "What about the gas milage?"
Intel commercial: An educated person might ask, "What about the cache size?"

What is it with all these enthusiast sites that run huge suites of 'benchmarks' on CPU's that measure just about everything except actual CPU performance? Are they unable to come up with the scratch to buy a copy of the SPEC benchmark suite? Is it really a surprise that Dual Core CPU's don't help much for running games or MS Office? Did anybody really need to test this? By contrast, it would be nice to know what the SPECRate FP and SPECRate Int is for these chips. For thwt matter, does anybody really care about the performance of a chip that you can't buy? For future reference, if you want to have a pretty good idea of the 'guaranteed not to exceed' performance for a CPU that you can actually purchase, go to http://www.spec.org/ and get the numbers without the ads.

Evas lib (part of Enlightenment) has both OpenGL and software benching

OpenGL:
FRAME COUNT: 69952 frames
TIME: 20.000 seconds
AVERAGE FPS: 3497.584 fps
EVAS BENCH: 58.293

Software:
FRAME COUNT: 4899 frames
TIME: 20.003 seconds
AVERAGE FPS: 244.911 fps
EVAS BENCH: 4.082

SPECviewperf is a very comprehensive OpenGL benchmark. Your own applications should be your personal benchmark though, if you only run Doom3 or UT2K4 then these other benchmarks don't really matter.

... tweak their drivers to perform better on benchmarks.

Another one: just look at the old Dhrystone benchmark and all of the over-the-top "optimizations" that were used to get better compiler/processor results. The SPEC organization, created in a direct attempt to deal with this very kind of problem, still must update its bendhmarks regularly in order to deal with loopholes (and changing technology in general). A good example was when a particular benchmark (matrix300; ref is 2/3 the way down) was defeated because it took no input. That made it possible in principle to collapse the entire program to a constant, and at some point, somebody did. That last link also gives a good description of why initially good benchmarks go stale.

And while those two examples are old enough to show that this has been going on a long time, there are plenty of examples far older.

Benchmarking has always been an arms race.

... tweak their drivers to perform better on benchmarks.

Another one: just look at the old Dhrystone benchmark and all of the over-the-top "optimizations" that were used to get better compiler/processor results. The SPEC organization, created in a direct attempt to deal with this very kind of problem, still must update its bendhmarks regularly in order to deal with loopholes (and changing technology in general). A good example was when a particular benchmark (matrix300; ref is 2/3 the way down) was defeated because it took no input. That made it possible in principle to collapse the entire program to a constant, and at some point, somebody did. That last link also gives a good description of why initially good benchmarks go stale.

And while those two examples are old enough to show that this has been going on a long time, there are plenty of examples far older.

Benchmarking has always been an arms race.

Monday Forbes reports Intel told software companies they should license a multi-core chip as one processor. Also on Monday, Intel compared their new Itanium to the "best published RISC" machine. Their graph indicates a 64-processor Itanium is about the same SpecIntRate as a 64-processor RISC machine. Now the funny part is for the RISC result they used the 32 chip Power5 SpecIntRate as 64-processors. So 64 Itanium-2 chips are really about the same as 32 Power-5 chips. So while Intel advocates per-chip licensing, they use per-core benchmarking. It is also interesting to note that this new Itanium-2 SpecIntBase of 1590 is just a bit faster than a 2 Ghz Pentium-M and much slower than a 2.6 Ghz Athlon-64-FX.

You can compare CPU benchmarks here.

AMD is beating the crap out of Intel.

Exactly what math do you want to do?

Go see:

SPEC FPU results

Look at the details, you'll see that different processors have different strengths depending on the tasks. To see what tasks they are you can look at:

SPEC FPU

Another thing the Intel compiler does work well for AMD too. But you may have to force it to recognize it as AMD to turn on even more optimizations. Apparently you get a boost as a nonIntel CPU, but if you disable the "intel-only" detection you can get even more of a boost.

Exactly what math do you want to do?

Go see:

SPEC FPU results

Look at the details, you'll see that different processors have different strengths depending on the tasks. To see what tasks they are you can look at:

SPEC FPU

Another thing the Intel compiler does work well for AMD too. But you may have to force it to recognize it as AMD to turn on even more optimizations. Apparently you get a boost as a nonIntel CPU, but if you disable the "intel-only" detection you can get even more of a boost.

Quad opterons scale a LOT better. They make most present quad xeons look pretty bad. There are few reasons to buy a quad xeon server - often it is better to buy two dual xeons. But quad opterons appear to be a different matter.

Check out: SPEC int rate

Sure they don't do as well as IBM POWER5, but most other stuff don't either ;).

As a CPU buff, I ordered a back-issue of Microprocessor Report where they discussed the introduction of the Alpha in glowing terms. The radical chip architecture and speed-at-any-price mentality was new at the time, but quickly proved itself to be the superior chip design approach. For most of the 1990s, the Alpha was the fastest chip on the market in both integer and floating point operations.

Alpha was a Risc chip's risc chip. The IBM Power architecture has dozens of operations and permutations; the Alpha has a handful. This contributed not only to the Alpha's speed, but also to its insatiable demands for memory. DEC introduced a code-translator that allowed the Alpha to run x86-32 binaries at native speeds, but warned that memory requirements would grow substantially. The software never became cost effective.

But, towards the turn of the millennium, something strange happened: the Pentium Pro architecture (happily renamed PII and PIII) inched towards the lead in integer operations. The P4 actually surpassed the Alpha chips. Intel had, by then, hired away some of the Alpha designers and began to adopt its performance enhancing strategies. How could Intel catch up to the Alpha when Intel was burdened with an architecture as convoluted as x86?

Strangely, the x86 architecture can also be a benefit to chip design. Because x86 compresses commonly used instructions into tiny, awkward byte codes, the P4 generation of chips requires less memory and fewer cache misses - and the convoluted opcodes can be decoded quickly by the processor prior to dispatch. In the long run, Alpha's simplified instruction set proved to be less useful than machine-code x86 compatibility; and x86 chips are now little more than Alpha chips sitting behind an x86 instruction decoder. The Alpha design lives on in every CPU you buy, whether it be AMD or Intel.

For further reading, check out CPU performance numbers on http://www.spec.org and read the commentary on Microprocessor Report.

My great idea is to visit this URL:

praise me! praise me!!!!!

(yes, SPEC might not be as comprehensive as you'd like, but given the choice between SPEC and some stupid 19 year old indian kid, i know who i'd choose....)

If you care about floating-point performance, regardless of how much I dislike the boxes, Itanium has the best SPECfp results (by far) -- look at the SPEC website
if you want to see what I mean.

Opteron has the best overall integer performance at the moment, and probably the best price/performance ratio. I believe it will thrash a Xeon on integer or FP, due to the doubling of the internal register file for both. But for max-floating-point ops per second, Itanium is still king. It's true for hspice in my line of work (which mostly consists of FP matrix math), it should hold true for you too.

I would try to get a set of eval systems from someone like HP, if I was you. They sell all 3 architectures -- tell them you will buy whichever one runs your research app the fastest and they will probably be willing to do the demo (or could get you some time ssh'd into one to at least try it out on your code).

-c

Try any from SPEC, for example. Maybe you're thinking about x86 because otherwise, the Itanium2 is way out of the Opteron's league (as well as price range, but that is besides the point.)

A single rating # would be just as useless as MHz numbers. There is no such thing as "true speed". Different users have different workloads. If you spent your time recompiling KDE, look at the gcc rating from the SPECint suite. If you want a decent framerate in Doom 3, wait for it to be released and run on different systems. Then upgrade. And if you just want Windows to boot faster -- clean up your autostart registry keys and forget about speed ratings.

• Sun Fire V440 --> $16,395
  
• HP DL580 G2 --> $34,374
  

• Sun Fire V440 --> 702/1054/26.5/33.0
  
• HP DL580 G2 --> 1491/1208/61.6/30.7
  

It's called SPEC. SPEC may not be perfect, but it's pretty good.

Really, the technical community needs to sit down and figure out a universal cross-platform benchmarking method.

Well, there's SPEC and TPC. Other than that, benchmarks are both overrated and the best metric we have for evaluating performance. Then you have cases when a CPU is optimized for a particular benchmark to inflate performance numbers (hence the term benchmarketing).

No, this does not signal that Itanium is doomed. Have a look at www.spec.org and look at the CPU2000 scores. Itanium is starting to kick some serious tail.

However Itanium is not a desktop chip-- its too big. 64-bit x86 will be a consumer product for desktops.

If you haven't seen any documentation it's only because you haven't looked hard enough!

There have been a number of tests comparing Opteron and Athlon64 processors in 32-bit and 64-bit mode under Linux and even a few that use Windows (beta version of WinXP). Here are a few links:

First off, some SPEC CINT2000 numbers: 32-bit OS + 32-bit apps, 64-bit OS + 32-bit apps and 64-bit OS + 64-bit apps. Unfortunately there are no similar CFP2000 numbers since GCC Fortran isn't up to the task, so you end up with lots of different variables making it nearly impossible to compare.

There is also this areticle at Ace's Hardware, and this little bit on Anandtech. Other tests exist.

Long story short, 64-bit support on the Opteron can and often does improve performance, even on apps that don't require lots of memory or use 64-bit integers. The extra registers help.

As for compilers, Microsoft plans on supporting AMD64 in their Visual.net 2003 compiler (beta versions are available now) and GCC supports the instruction set now. That makes up the compilers used for the vast majority of applications. As you mention, PGC is also doing a compiler, and it seems that Sun will support AMD64 with their compiler as well. Here is the AMD64 Developer Resource Kit page which lists all sorts of software with support for the AMD64 instruction set.

If you haven't seen any documentation it's only because you haven't looked hard enough!

There have been a number of tests comparing Opteron and Athlon64 processors in 32-bit and 64-bit mode under Linux and even a few that use Windows (beta version of WinXP). Here are a few links:

First off, some SPEC CINT2000 numbers: 32-bit OS + 32-bit apps, 64-bit OS + 32-bit apps and 64-bit OS + 64-bit apps. Unfortunately there are no similar CFP2000 numbers since GCC Fortran isn't up to the task, so you end up with lots of different variables making it nearly impossible to compare.

There is also this areticle at Ace's Hardware, and this little bit on Anandtech. Other tests exist.

Long story short, 64-bit support on the Opteron can and often does improve performance, even on apps that don't require lots of memory or use 64-bit integers. The extra registers help.

As for compilers, Microsoft plans on supporting AMD64 in their Visual.net 2003 compiler (beta versions are available now) and GCC supports the instruction set now. That makes up the compilers used for the vast majority of applications. As you mention, PGC is also doing a compiler, and it seems that Sun will support AMD64 with their compiler as well. Here is the AMD64 Developer Resource Kit page which lists all sorts of software with support for the AMD64 instruction set.

If you haven't seen any documentation it's only because you haven't looked hard enough!

There have been a number of tests comparing Opteron and Athlon64 processors in 32-bit and 64-bit mode under Linux and even a few that use Windows (beta version of WinXP). Here are a few links:

First off, some SPEC CINT2000 numbers: 32-bit OS + 32-bit apps, 64-bit OS + 32-bit apps and 64-bit OS + 64-bit apps. Unfortunately there are no similar CFP2000 numbers since GCC Fortran isn't up to the task, so you end up with lots of different variables making it nearly impossible to compare.

There is also this areticle at Ace's Hardware, and this little bit on Anandtech. Other tests exist.

Long story short, 64-bit support on the Opteron can and often does improve performance, even on apps that don't require lots of memory or use 64-bit integers. The extra registers help.

As for compilers, Microsoft plans on supporting AMD64 in their Visual.net 2003 compiler (beta versions are available now) and GCC supports the instruction set now. That makes up the compilers used for the vast majority of applications. As you mention, PGC is also doing a compiler, and it seems that Sun will support AMD64 with their compiler as well. Here is the AMD64 Developer Resource Kit page which lists all sorts of software with support for the AMD64 instruction set.

We've had benchmarks for months, actually meaningful benchmarks. They show that the G5 is a nice, competitive chip, but it's merely keeping up with AMD performance-wise. And G5 systems are behind Opteron systems in terms of bang-for-the-buck and features.

If you check the published SPEC benchmarks for the Opteron 148 against Apple's claimed SPEC results for the G5, you'll see that a dual G5 is not faster than the Opteron. It is pretty telling, incidentally, that Apple still has not actually submitted official SPEC results for the G5's--they really don't seem comfortable with the comparison on a real benchmark.

Of course, a dual Opteron will have other advantages for many users: you can get it in 1U rack mounts, it runs a lot more application software, and it's cheaper.

Running five application programs does not constitute a meaningful benchmark of the CPU. We don't know how those applications are written, what CPUs they are compiled for, what compilers they used, etc. Most likely, none of those applications have been tuned for Opteron, wherease they have received extensive tuning for PPC and AltiVec over the years. The differences may be something as trivial as cache conflicts. All those "benchmarks" tell you is that if you must run the current version of Bryce and AfterEffects, you may get more bang (but not necessarily more bang-for-the-buck) out of a G5 for the time being.

Because I don't use Photoshop and don't do much media processing, I'm not all that interested in how those application-level benchmarks play out. The things I care about are integer-based. Although I've done some crossplatform benchmarking of my typical workload before, it's time-consuming to get that all set up the right way. I may return to that at some point, but I've found that SPEC CPU2000 roughly corresponds, at least in the integer tests. The rate tests don't measure communication costs between processes on multiple processors, so they're a bit of a best case for multiprocessor systems. Still, they can be revealing.

Apple publishes a base 17.2 SPECint_rate for the dual 2GHz G5 (not submitted to SPEC? they point here. Among the published scores at spec.org is a dual Opteron 248 running Linux and using gcc.

It scores 27.4.

Xi Computing charges $200 more for the dual 248 over the 246 the parent article was writing about. So deduct ~10% of MHz from that 27 if you want both boxes to be dual 2GHz....

Because I don't use Photoshop and don't do much media processing, I'm not all that interested in how those application-level benchmarks play out. The things I care about are integer-based. Although I've done some crossplatform benchmarking of my typical workload before, it's time-consuming to get that all set up the right way. I may return to that at some point, but I've found that SPEC CPU2000 roughly corresponds, at least in the integer tests. The rate tests don't measure communication costs between processes on multiple processors, so they're a bit of a best case for multiprocessor systems. Still, they can be revealing.

Apple publishes a base 17.2 SPECint_rate for the dual 2GHz G5 (not submitted to SPEC? they point here. Among the published scores at spec.org is a dual Opteron 248 running Linux and using gcc.

It scores 27.4.

Xi Computing charges $200 more for the dual 248 over the 246 the parent article was writing about. So deduct ~10% of MHz from that 27 if you want both boxes to be dual 2GHz....

I know this isn't quite on topic, but I wonder how the latest Alpha design would fair. The alpha was the first mass produced 64 bit chip that had any commercial success. It was introduced in the early nineties. IN fact Linus had one. Basically the curret EV78 is a 6 or 7 year old design, but in most serious tests of processor power it has done quite good. It's amazing that such an "old" design still works so well. The last SPEC numbers I can find are here. Considering the platorm has been ignored and basically orphaned, it's suprising that this chip still powers many of the worlds top rated super computers.

How does all this relate to the G5 and Opteron? Well AMD gets it's bus design from the Alpha lineage. The G5 is built by IBM, who I believe is building the alpha cores as well (I could be wrong, I can't keep up). The irony? Every current intel pentium chip is quality control checked by machines with alpha processors. Funny world huh?

AngryPeopleRule

I know this isn't quite on topic, but I wonder how the latest Alpha design would fair. The alpha was the first mass produced 64 bit chip that had any commercial success. It was introduced in the early nineties. IN fact Linus had one. Basically the curret EV78 is a 6 or 7 year old design, but in most serious tests of processor power it has done quite good. It's amazing that such an "old" design still works so well. The last SPEC numbers I can find are here. Considering the platorm has been ignored and basically orphaned, it's suprising that this chip still powers many of the worlds top rated super computers.

How does all this relate to the G5 and Opteron? Well AMD gets it's bus design from the Alpha lineage. The G5 is built by IBM, who I believe is building the alpha cores as well (I could be wrong, I can't keep up). The irony? Every current intel pentium chip is quality control checked by machines with alpha processors. Funny world huh?

AngryPeopleRule

AMD used version 7.0 to compile it's entries for SpecInt performance, and I'm guessing that they didn't just pick it because they thought it had a cute name.

Compiler:
Intel C++ 7.0 build 20021021Z
Microsoft Visual Studio .NET 7.0.9466 (libraries)
MicroQuill Smartheap Library 6.0

There are some benchmarks available at the Spec website. You have to do a little bit of digging, but AMD has submitted a lot of results with identical systems running 32-bit and 64-bit Linux.

On average, running AMD64 code will buy you about a 5% boost in performance over running IA-32 code on the Opteron/Athlon64. Of course, some applications will see a much larger improvement (some apps are more than twice as fast), while others will actually be slower (64-bit pointers = more data to load from memory and fewer pointers that can be stored in cache). Note that this actually has just about nothing to do with 32 vs. 64-bit for most cases, but rather that AMD64 doubled the number of general purpose registers (16 vs. 8 in IA-32). Considering that the lack of registers was one of the last major downsides to the x86 instruction set (as compared to something like PowerPC, MIPS, SPARC, etc.), this was a very good idea. Of course, even 16 visible registers is rather low, most other ISAs have 32+ visible registers.

Could be because the Opteron is one of the fastest chip in the world at executing Java code right now, and that's when running in IA-32 (aka 32-bit x86) mode?

Check out the results for SPEC JBB2000. On a per-processor basis, AMD's Opteron chips are second only to Intel/HP Itanium2 based systems, and the Opterons are quite a bit cheaper. Actually, when combined with the new x48 Opteron chips announced alongside the Sun deal, AMD should make up most of the current 8% difference between the two chips.

So, they get better performance than anything IBM has to offer (even the full-fledged Power4 can't match the Opteron in Java if the above test is to be believed) and a much lower price tag than what Intel is looking for. Seems like a pretty good choice if you ask me.

Are there reliable/meaningful ways to compare throughput on these machines?

Well, one basis for comparison, a fairly reasonable one, is the SPEC benchmarks. Since IBM went to their Power4 CPU's for the Z series boxes you could compare SPEC benchmarks. However, as far as I know IBM has not submitted SPEC benchmarks on the Z series boxes. Another valid comparison is TPC benchmarks. TPC-C uses a standardized OLTP system that probably would be a valid comparison.

I think the thing to understand is that mainframes are, these days, just another server when talking about CPU power, transactional capabilities and so forth. The one place where they really shine is reliability and availability. It really is true that you get what you pay for. There is a reason why mainframes are expensive. They are engineered for a degree of reliability not found in less expensive systems. However, for most shops the difference between 4 9's of uptime, which is quite achievable by a Sun or HP cluster, and 5 9's of uptime is only about 45 minutes, on an annual basis. Is it really worth the cost of a mainframe sysplex instead of a Sun cluster to achieve the extra 45 minutes? Probably not.

It's also annoying that a CICS application that served 10000 terminals with sub-second response times is denigrated by people whose web-based app takes 10 seconds or more to do a simple update and return output to the user.

Well, here you are really comparing apples to oranges. CICS applications served up across an SNA network to dumb terminals is not really comparable to web applications (you don't specify what platform, but let's assume Java) across a TCP/IP network to modern PC's. Simply comparing CPU power and network throughput won't do the trick. Admittedly a "simple" update and output return should perform as well as a CICS application to dumb terminals. But only IF the system was designed properly, from end to end for performance. For example, we recently implemented an image management system on a sun cluster. We have 500 users hitting the system, retrieving about 150 insurance claims images per minute. We are able to achieve a response time of about .75 seconds with it. We designed it for response time because we lose money when our claims examiners sit and wait for the image. The user accesses a java based web app, images are housed on Hitachi storage, and the application and DB2 database are housed on a cluster based on SunFire 6800's. Uptime requirements are 99.9%. So high performance and reasonable reliability can be done on midrange boxes. It's a matter of design.

The real question is, do you have the right system for the job?

Yes, even honest data can be misunderstood, taken out of context, or abused.

Example misunderstanding: the comment that said that compiler optimizations are "cheating". Nope, compiler optimizations are allowed by design. The SPEC CPU benchmarks are specifically intended to test CPU, memory hierarchy (caches & main memory), and compilers. (Of course, compiler optimizations are expected to be generally useful, and not unfairly "target" a specific benchmark. That's a judgment call which SPEC has sometimes had to work hard to make.)

Now, one can argue about whether Apple's attempt to hold the compiler constant was a reasonable thing to do, given that the CPU2000 benchmarks are intended to test all three of cpu/memory/compiler. But they at least have told us what they did, and how they did it, and in my book that goes a long way toward supplying the ounce of honest data.

For those who suggest that improvements should be made to the SPEC benchmarks in one way or another, please be reminded that you can join SPEC (discounts for academic/non-profit memberships) and you can contribute to new benchmarks (and earn modest financial compensation if your benchmark is accepted.)

SPEC has a long history of welcoming technical contributions by technical people (but is less responsive to complaints - we know it's impossible to please everyone).

Disclaimers: I am not employed by Apple, nor do I own an Apple computer. I am employed by another company that is a member of SPEC. My opinions are my own and have not been approved or disapproved by my employer, by SPEC, or by Apple.
-john henning

Yes, even honest data can be misunderstood, taken out of context, or abused.

Example misunderstanding: the comment that said that compiler optimizations are "cheating". Nope, compiler optimizations are allowed by design. The SPEC CPU benchmarks are specifically intended to test CPU, memory hierarchy (caches & main memory), and compilers. (Of course, compiler optimizations are expected to be generally useful, and not unfairly "target" a specific benchmark. That's a judgment call which SPEC has sometimes had to work hard to make.)

Now, one can argue about whether Apple's attempt to hold the compiler constant was a reasonable thing to do, given that the CPU2000 benchmarks are intended to test all three of cpu/memory/compiler. But they at least have told us what they did, and how they did it, and in my book that goes a long way toward supplying the ounce of honest data.

For those who suggest that improvements should be made to the SPEC benchmarks in one way or another, please be reminded that you can join SPEC (discounts for academic/non-profit memberships) and you can contribute to new benchmarks (and earn modest financial compensation if your benchmark is accepted.)

SPEC has a long history of welcoming technical contributions by technical people (but is less responsive to complaints - we know it's impossible to please everyone).

Disclaimers: I am not employed by Apple, nor do I own an Apple computer. I am employed by another company that is a member of SPEC. My opinions are my own and have not been approved or disapproved by my employer, by SPEC, or by Apple.
-john henning

Yes, even honest data can be misunderstood, taken out of context, or abused.

Example misunderstanding: the comment that said that compiler optimizations are "cheating". Nope, compiler optimizations are allowed by design. The SPEC CPU benchmarks are specifically intended to test CPU, memory hierarchy (caches & main memory), and compilers. (Of course, compiler optimizations are expected to be generally useful, and not unfairly "target" a specific benchmark. That's a judgment call which SPEC has sometimes had to work hard to make.)

Now, one can argue about whether Apple's attempt to hold the compiler constant was a reasonable thing to do, given that the CPU2000 benchmarks are intended to test all three of cpu/memory/compiler. But they at least have told us what they did, and how they did it, and in my book that goes a long way toward supplying the ounce of honest data.

For those who suggest that improvements should be made to the SPEC benchmarks in one way or another, please be reminded that you can join SPEC (discounts for academic/non-profit memberships) and you can contribute to new benchmarks (and earn modest financial compensation if your benchmark is accepted.)

SPEC has a long history of welcoming technical contributions by technical people (but is less responsive to complaints - we know it's impossible to please everyone).

Disclaimers: I am not employed by Apple, nor do I own an Apple computer. I am employed by another company that is a member of SPEC. My opinions are my own and have not been approved or disapproved by my employer, by SPEC, or by Apple.
-john henning

The PC World benchmark says the Athlon is faster in the real world. Well looking at SPEC published benchmarks for the Athlon FX-51:

2GHz G5 (from Apple, not on SPEC): 840 (float), 800 (int)

Athlon 64 3200+ (from AMD, on SPEC): 1266 (float), 1180 (int)

That's about 50% faster for the Athlon, the G5 may have been the fastest personal computer when it was launched but it isn't any more.

Look, Sun makes great hardware above the low end, but an old K6-2 beats a Blade 100 desktop in perceived performance and compile speeds. The IIe chip is low power -- in more ways than one. If you don't have a CPU-bound process, like say, a web server for mostly static pages, a Netra X1 or V100 works great, but it's not a fast CPU.

OK. Price/performance. Let's see. SPEC2000 results, Sun Blade 100 (650Mhz US IIe, fastest IIe available in a system) gets 246 integer, 276 floating point. An Opteron 146 (2.0Ghz), on an Asus SK8N board, gets 1262 integer, 1300 floating point.

Just in case you meant the US IIIi, as used in the new V210, V240, V250, and Blade 1500, the results on a V210 (server chassis, 1002 Mhz) are 555 integer, 841 floating point. If and when Sun can get the IIIi up to 2Ghz, that would not quite match the Opteron for integer ops, and just beat it for floating point. Of course, by that time, the Opteron will probably be up to 3Ghz and smoke any available IIIi.

Any more bullshit to sling about price/performance?

Benchmarks from www.spec.org, as published by the vendors. Configurations of the boxes are detailed there.

absurd figures? dude, the itanium 2 really does get over 2000. Just look:

here

where are the measured G5 figures?

The originally proposed quick fix is to simply redesign the the Sun servers to accept the SPARC64-V. An even better proposal, now leaked by the press, is to simply discontinue the Sun-designed servers and to sell re-branded Fujitsu designed servers. The latter proposal is a much faster path to solving the server-performance problem at Sun but leads to lower profit margins. Clearly, the situation at Sun is dire, so you can be assured that one of these proposals will be adopted. (Please read "Sun posts deeper loss for quarter". Having lower profit margins is better than having no profit margins. Right now, the second proposal appears to be winning.

Sun Microsystems will most likely fire more than 50% of its processor development team. The single biggest cause of Sun losing marketshare so rapidly is the UltraSPARC III. It has horrible performance. Check "SPEC" and "TPC".

How does this deal help Fujitsu? It can now sell more servers and get more cash. Fujitsu has the upperhand and should force Sun to accept the second proposal: Sun exits the highend server market and sells rebranded Fujitsu-designed servers. To avoid being dependent on Sun, Fujitsu should move rapidly to jettisoning the Solaris OS in favor Linux. Fujitsu is rapidly being shaped into a company like IBM: high-performance servers and computing services are the mainstays of the business.

As a side note, Fujitsu rejects hiring foreign workers (the equivalent of H-1Bs). Their SPARC64-V and SPARC64-VI were designed and built almost exclusively by native talent. When Fujitsu hires workers, Fujitsu most values the quality of "willingness to work", not "best match of skills"; Fujitsu will take the time to train its employee. Fujitsu is a traditional Japanese company that emulates most of the values that once characterized traditional American companies. Sun, by contrast, encourages the employment of H-1Bs; the UltraSPARC III and the UltraSPARC IV were built substantially by former or current H-1B workers. Sun seeks only "best match of skills" and, along with Intel, claims that they absolutely need H-1B worker even during a period of 8% unemployment among native Americans in Silicon Valley.

... from the desk of the reporter

1. i assume he used the tar that ships with solaris... and we all know that the solaris tar sucks badly. if he wants to benchmark the fs and not the app (tar) he should have installed gtar on the solaris box.
2. the http benchmarks seems specious... i would have preffered to have seen a specweb99 bench... something that i could compare to real, existing benchmarks.

However, this single biggest problem also has an easy solution. Sun merely needs to jettison its SPARC processor R&D team and to adopt the SPARC64-V and the SPARC64-VI. The latter is a dual-core chip just like the well-regarded Power4. Sun could easily redesign its server boards within a month to accept the SPARC64 chips.

The SPARC64-V and SPARC64-VI are radically different from the UltraSPARC III. The former were designed and built almost exclusively by native talent (i. e. Japanese citizens). The UltraSPARC III was built by H-1B workers because Sun, Intel, and other companies claim that they cannot find enough native talent (i. e. American citizen) who are good enough -- even during a period 8% unemployment in Silicon Valley.

The issue here is mismanagement at Sun. Specifically, the management up to Scott McNealy himself refuses to adopt the SPARC64-V/VI. Why would any company refuse to adopt a processor that outperforms its own processor, that is readily available, that executes an identical instruction set , and that would immediately boost the performance of all the servers sold by said company? Why? The answer is deliberate mismanagement at Sun.

... from the desk of the reporter

So we have chips running at 1.1GHz, 553 MHz, 800MHz or 533MHz. And almost all people seem to think that these numbers are some kind of speed indicators, even at slashdot. Jesus!

How about a link to some actual benchmark figures? Have anyone seen any? I just looked at SPEC, but there was nothing there. What is the speed of these chips, really?

As a side note, I believe that the really interesting news is that the new EDEN chips use a nanoBGA packaging and are to be used in the new nanoITX form factor which is smaller than miniITX. At 120x120mm, the new motherboard is just slightly smaller than a CD enclosure!

Sun's mistakes are well documented, but the biggest one is believing that what made them successful in the past would make them successful in the future."

If that is the biggest mistake, then the second biggest mistake is the processor-design team. According to "Sun's processor plans slip a notch", the schedule of the UltraSPARC processors has slipped again. The processor-design team has 2 characteristics: chronically behind schedule and chronically behind the performance curve. Right now, the UltraSPARC III is being crushed, performance-wise, by the Power4+ and the SPARC64-V, according to SPEC".

Yet, McNealy stubbornly clings to the UltraSPARC III. If he knew how to run Sun, he would immediately scrap the UltraSPARC III and successors and tell his server team to use the SPARC64-V. He could come out with an E15K that just barely competes against the p690 in about 2 months. The SPARC64-V is instruction-set compatiable with the UltraSPARC III and vastly outperforms it, and modifying the E15K and other Sun servers to use the SPARC64-V is a simple matter.

Time is extremely critical. Sun itself claims that it will lose about 10 cents per share for the first quarter. 10 cents per share means a loss of about $300 million. Extrapolating to the full fiscal year means a loss of about $1.2 billion. In order to compensate for that loss, Sun will need to fire about 6000 employees.

The only conceivable reason that McNealy refuses to abandon the UltraSPARC III is that he fervently supports a workforce weighted in favor of H-1B workers. Sun has many H-1B employees, and they built the UltraSPARC III. By contrast, Fujitsu uses native workers (i.e. Japanese citizens), and they built the SPARC64-V. (IBM also prefers American citizens or permanent residents, and they built the Power4).

McNealy better put aside his ego and go with the SPARC64-V. It is the fastest, safest route to boosting Sun's fortunes. In the future, he should consider giving preference to American workers, not H-1B workers. There is no evidence to suggest that H-1B workers are better than American ones; indeed, H-1B workers might actually be destroying Sun as evidenced by the horribly designed UltraSPARC III.

Most importantly, the SPARC64-V will buy time for McNealy. Maybe 1 year or 2 years of breathing room. Then, he can make the hard decision of spinning off the processor-development group and transforming Sun into a niche player that focuses on two areas: software applications and highend-servers that use Fujitsu processors (or, gasp, IBM processors) designed by native talent. Other possibilities have been thoughtfully outlined by Merrill Lynch, the premier American investment company.

... from the desk of the reporter