Slashdot Mirror

That's right, this site gives the impression
that P3 1.13 Ghz is the fastest CPU in the industry.
In fact, both Alpha and U-Sparc-III are TWICE faster.

Well, no. By far the most credible cross-platform CPU benchmark is SPEC; if you had bothered to check the SPEC CPU2000 scores you would find that a 1GHz P3 is precisely as fast as a 900MHz US-III on SPECint2000_base (438) and only 30% slower in SPECfp2000_base (327 vs. 427). Assuming linear scaling (not generally a great idea but close enough in this case), a 1.13GHz P3 will be 13% faster than the USIII-900 in SPECint, and 16% slower in SPECfp. In other words, they will be essentially equal.

Now, the Alpha actually is quite a bit faster than the P3, with SPECint/fp scores of 514/591 for an 833MHz chip. That makes it 17%/80% faster than our 1GHz P3, and an estimated 4%/60% faster than a 1.13GHz P3.

Now, on the other hand, both the Alpha and the US-III systems tested cost many times more than the i840 1GHz P3 system; not only do the chips cost a good deal more, but they get the benefit of much faster (and more expensive) buses to memory, etc. This makes quite a difference even in the SPEC CPU tests, and if the chips could somehow be placed on equivalent platforms, the P3 would easily win SPECint outright, and might be rather competitive even on SPECfp. (In case you were wondering, the reason the P3 sucks at SPECfp is because it is saddled with the register-starved x87 floating-point implementation for backwards compatability reasons; the P4 will go quite a ways towards solving this problem--at least as far as newly compiled code goes--with its SSE2 instructions.)

On the third hand, as has been pointed out, the 1.13 GHz P3 does not exist, and never did. (Intel "launched" what amounted to several engineering samples which turned out not to work properly anyways. Other than a couple dozen sent off to review sites and OEMs for validation, no 1.13 GHz P3's ever left the company.)

On the fourth hand, a chip which does currently exist in much higher quantities, the 1.5 GHz P4, looks like it will quite forcefully take the SPECint crown away from Alpha, and depending on Intel's progress in optimizing their compilers for SSE2, might even take the SPECfp crown as well. We'll get to find out when it is officially released in about a month or so.

On the fifth and final hand, though, the real advantage of the Alpha and US-III is their platforms, which give them much greater i/o throughput--often more important than CPU power for server applications anyways--and allow them to scale to configurations of 32 and 64 CPUs and beyond; Intel has a long way to go to compete on these measures.

With all other variables equal (bus speed, HD speed, etc.), a 500 MHz PowerPC G3 is approximately twice as fast as a 1 GHz Intel PII/P!!! at doing hard-core integer number crunching (Photoshop, some 3D games in software mode, d.net) according to the SPEC integer math benchmark. And the vector unit in G4 is quite a bit easier to code for than the vector unit in PIII, giving even more speed and boosting Team Slashdot's RC5 rating.
<O
( \
XPlay Tetris On Drugs!

http://www.spec.org/osg/web99/results/res2000q2/

Vendor: Dell
Model: PowerEdge 6400/700
Processor: 700MHz Pentium III Xeon
# Processors: 4
Memory: 8 GB
Disk Subsystem: 7 9GB 10KRPM drives
Operating System: Windows 2000 Advanced Server
File System: NTFS
HTTP Software
Vendor: Microsoft
HTTP Software: Internet Information Server 5.0
Conforming Simultaneous Connections (Median): 1598

Vendor: Dell
Model: PowerEdge 6400/700
Processor: 700MHz Pentium III Xeon
# Processors: 4
Memory: 8 GB
Disk Subsystem: 5 9GB 10KRPM drives
Operating System: Red Hat Linux 6.2 Threaded Web Server Add-On
File System: ext2
HTTP Software
Vendor: Red Hat
HTTP Software: TUX 1.0
Conforming Simultaneous Connections (Median): 4200

Of course, it all means nothing, I'm sure...

...just mere numbers.

Right?

t_t_b
--
I think not; therefore I ain't®

Vendor: Dell
Model: PowerEdge 2400/667
Processor: 667MHz Pentium IIIEB
# Processors: 1
Operating System: Red Hat Linux 6.2 Threaded Web Server Add-On
File System: ext2
HTTP Software
Vendor: Redhat
HTTP Software: TUX 1.0
Conforming Simultaneous Connections (Median): 1270

Vendor: IBM
Model: Netfinity 7600
Processor: 700 MHz Pentium III Xeon
# Processors: 1
Operating System: Microsoft Windows 2000 Advanced Server
File System: NTFS
HTTP Software
Vendor: Microsoft
HTTP Software: IIS 5.0
Conforming Simultaneous Connections (Median): 968

Vendor: Dell
Model: PowerEdge 4400/800
Processor: 800MHz Pentium III Xeon
# Processors: 2
Operating System: Red Hat Linux 6.2 Threaded Web Server Add-On
File System: ext2
HTTP Software
Vendor: Red Hat
HTTP Software: TUX 1.0
Conforming Simultaneous Connections (Median): 2200

Vendor: Compaq Computer Corporation
Model: ProLiant DL360
Processor: 800EBMHz Pentium III
# Processors: 2
Operating System: Windows 2000 Server
File System: NTFS
HTTP Software
Vendor: Microsoft
HTTP Software: IIS 5.0
Conforming Simultaneous Connections (Median): 1020

Vendor: Compaq Computer Corporation
Model: ProLiant DL380
Processor: 933EBMHz Pentium III
# Processors: 2
Operating System: Windows 2000 Server
File System: NTFS
HTTP Software
Vendor: Microsoft
HTTP Software: IIS 5.0
Conforming Simultaneous Connections (Median): 1098

...of course, I'm sure this all means nothing.

Mere numbers...

Yeaah..

t_t_b
--
I think not; therefore I ain't®

Check out this IIS setup and this TUX setup. The hardware is identical in each case with the exception of the SCSI controllers, and that shouldn't matter because in the IIS case the data set is small enough to fit in memory. The TUX setup is 2.5 times faster despite having a larger dataset.

Check out this IIS setup and this TUX setup. The hardware is identical in each case with the exception of the SCSI controllers, and that shouldn't matter because in the IIS case the data set is small enough to fit in memory. The TUX setup is 2.5 times faster despite having a larger dataset.

Apache may well ble slower than IIS (and probably is, as their main focus is portability and correctness, while many performance-tuning tricks must be applied manually), but IIS is by far slower than Tux as tested by Dell. Links:

• Tux
• Specweb, Q3
• Specweb, Q2

Apache may well ble slower than IIS (and probably is, as their main focus is portability and correctness, while many performance-tuning tricks must be applied manually), but IIS is by far slower than Tux as tested by Dell. Links:

• Tux
• Specweb, Q3
• Specweb, Q2

More specifically go to Third Quarter 2000 SPECweb99 Results

I sent in a few links to LinuxToday for a story about the SpecWEB99 numbers from Dell recently. Here is this link to the resulting story.

The bottom line is that Dell has been submitting systems to the Special Performance Evaluation Corp (SPEC) to be included in the SpecWEB99 results. Dell using the Red Hat TUX 1.0 web server has kicked the butt of all other submissions with the single exception of the IBM eServer pSeries 680 with 12 CPU's running Zeus. The Dell / Red Hat Tux 1.0 (8 CPU) numbers come in at 6387 simultaneous connections. The IIS/5.0 systems highest score is on an IBM Netfinity 6000R (4CPU) comes dragging in at 1582.

Hope this helps.

Tony

Go to the SPEC web site. Then search for the SPECweb99 results Dell submitted.

Apparently, they are using Redhat's Tux server, not Apache. I don't know whether they are related but the combination kicks IIS's ass. You can't get much more definitive than the manufacturer's own tests using the recognised industry benchmark.

This page on Dell's site might also be of interest.

The current WebSpec99 numbers for Linux and Tux blows away all other competitors.

Look at: http://www.spec.org/osg/web99/r esu lts/res2000q2/

Show your boss that Linux has twice the performance of Windows.

Tux is a kernel module that integrates the html protocol directly into the kernel. It can serve static as well as dynamic web pages and can be integrated with Apache.

Check it out at: http://www.redhat.com/tux/

This shows how a bleeding edge webserver, TUX 1.0, running on a tweaked 2.4.0-testX box can outperform a virtually identical box running IIS 5.0. Curiously enough, these are DELL boxes, and the tests were performed by DELL.

I understand that it is hoped that the advanced features of TUX 1.0 will eventually make their way into Apache.

This shows how a bleeding edge webserver, TUX 1.0, running on a tweaked 2.4.0-testX box can outperform a virtually identical box running IIS 5.0. Curiously enough, these are DELL boxes, and the tests were performed by DELL.

I understand that it is hoped that the advanced features of TUX 1.0 will eventually make their way into Apache.

http://slashdot.org/articles/00/07/05/0211257.shtm l

http://www.spec.org/osg/web99/results/res2000q2/

While HP has not published Spec2k numbers yet, it is doing quite well with the PA 8600 and the upcoming 8700 thank you.

Check again.

HP 9000 Model N4000 552 MHz PA-RISC 8600
SPECintbase2000: 367 (equivalent to a P3-800)
SPECfpbase2000: 338 (barely better than a P3-1000)

As for the 8700, what I said was not, "HP is dead meat," but rather "HP desperately needs to release a new processor"--namely the 8700. Unfortunately, they are not expected to do so for another year or so, in the second half of 2001. And when they do it will be little more than a process shrink of the 8600, itself just a critical path tweak of the 8500, a process shrink of the 8200, which was a tweak of the now 5 year-old PA-RISC 8000. In other words, the 8700 will be no more a new processor than was the Coppermine P3 (a process shrink and reworking of the Katmai P3, itself a tweak of the Deschutes P2, which was a process shrink of the Katmai P2, which was a tweak of the original P2, which was a process shrink and relayout of the 6 year-old PPro). Worse yet, all indications are that the 8800 and 8900 will be yet more tweaks and shrinks, rather than new cores. Don't get me wrong--the PA-8x00 has turned out to be a great core. But diverting most of their chip engineers over to help Intel with IA-64 looks like a bad move on HP's part these days. Maybe in a year and a half McKinley will change our minds, though.

SPECfp2000 results are available for the UltraSparc-III 900 MHz. It scored a 482. Pretty damn quick, especially when you consider that its score is more than 50% higher than the Pentium-III 933 Mhz, which got a 305.

Of course, if you consider cost, it nearly evens out.. but people don't buy Suns cause they are cheap.

SPECfp2000 results are available for the UltraSparc-III 900 MHz. It scored a 482. Pretty damn quick, especially when you consider that its score is more than 50% higher than the Pentium-III 933 Mhz, which got a 305.

Of course, if you consider cost, it nearly evens out.. but people don't buy Suns cause they are cheap.

Check www.spec.org; in short, Intel just got kicked into touch. A new US III 900MHz workstation is getting over twice the FP performance of a roughly equivalent Intel chip.
--

Obviously industry standard benchmarks are no match for an AC who claims to have a rendering package, but look here
--Shoeboy

Yes, the MIPS processor only runs at 400 Mhz, but the PIII's don't approach the all-around FP speeds of the R12K until the Intel reaches about 1 Ghz. Its a good illustrator of the Mhz lie. On integer benchmarks, the MIPS isn't quite as far ahead, but the 400 Mhz R12K is similar to a 733 PIII.

I am tired of seeing Intel put out more and more vaporware. RDRAM, IA-64, etc, etc..

You can buy RDRAM right now if you want to. Hardly vapour.

Engineering prototypes of the IA-64 have been around for a while, with every indication that they will ship. Doesn't look very vaprous to me.

IA-64 has 1/5 the performance of an alpha under gcc, which is not optimised for the alpha. (likely the kind that is 3x an Athlon or more for a P3)

Firstly, GCC is not the best compiler in the world. When comparing an Alpha to an IA-64 chip, I'd use Intel's compiler on the IA-64 and Compaq's compiler on the Alpha. Both companies have a history of writing compilers that were extremely well optimized for their platforms.

Secondly, I don't see much support for your figures. See my next point.

Even a 2 year old alpha can beat most P3s (1.5 -2x P3 MHz = alpha MHz in performance)

Not really. Alpha chips are about even in everything except floating point (where the Alpha blows *everyone* out of the water - Sun, HP, IBM, Motorola, etc).

They do this with the higher speed grades of their chips that were released _recently_. Older chips used the same design but were clocked more slowly, and don't blow away present chips.

Check http://www.spec.org for reasonably accurate benchmark information. They use the fairest system for evaluation that I've seen (standard test code supplied by SPEC, compilation and system tweaking handled by the companies owning the platforms being tested).

As far as the performance of the Alpha or an Athlon vs. the P4 goes... The P4 is still in the final debugging stages. Wait six months and look for SPEC marks.

Personally, I'd like to see SPEC marks for the G4. Apple has been allergic to SPEC of late.

• SPEC JBB2000
• Volano
• Java Grande Forum - not a benchmark, but these folks work on improving Java for high-performance computing

In SPECfp2000, an Origin 2000 with one CPU -enabled- beats a 1 GHz Pentium III.

http://www.spec.org

Here Is the spec results of a Origin 200 360mhz R12k.
Here is a dell PIII 733.
You'll find the PIII system has better intger proformance and the Origin better floating point proformance. But, the tests are run only using _one_ cpu... and from I read in previous posts the origin is focusing on really fast IO.

Here Is the spec results of a Origin 200 360mhz R12k.
Here is a dell PIII 733.
You'll find the PIII system has better intger proformance and the Origin better floating point proformance. But, the tests are run only using _one_ cpu... and from I read in previous posts the origin is focusing on really fast IO.

Are these results even relevant outside of benchmarks?

The thing that I find the most disturbing about the SPEC99 results is that Linux/TUX cleaned everyones clock so easily. It wasn't just the other IIS machines but the full blown UNIX boxes. The machine that came the closest was the IBM RS/6000 with 8 processors running Zeus, and it only had a performance score of 3216.

It is hard to say if RedHat/TUX would best NT in the orginal Mindcraft test. I would tend to believe that it probably would.

What I would like to see is how a Linux box of the same configuration except without TUX would fair. That might be a better indication of exactly how much faster TUX is. If you noticed the only machines that ran Linux were the Dell machines, and all of them had TUX on them.

The OS's I would like to see tested are the following:
NT 4.0 with IIS
NT 4.0 with Apache
Win2000 with IIS
Win2000 with Apache
Linux 2.2 kernel, Apache w/ TUX
Linux 2.2 kernel, Apache w/o TUX
Linux 2.4.pre kernel, Apache w/ & w/o TUX
Solaris 8 with iPlanet
Solaris 8 with Apache
OpenBSD with Apache

If you read the full report from Mindcraft they also tested Solaris 7 x86 version and FreeBSD. Where NT scored >2000, FreeBSD scored ~1200, and Solaris scored >6000.

Oh, in case you are wondering Mindcraft did infact help out with the SPECweb99 tests. The information on how their machine faired in the test it is with the rest of the results.
Of course it did only have one NIC instead of 4.

Oh, in case you are wondering Mindcraft did infact help out with the SPECweb99 tests. The information on how their machine faired in the test it is with the rest of the results.
Of course it did only have one NIC instead of 4.

Your ServerBench comments - maybe it's a valid result, maybe not. Note that the 2.2 kernel was used, while the SPECweb99 result used the 2.4 kernel.

Your SPECweb99 comments though make no sense whatsoever, they show basic misunderstandings of SPECweb99. In SPECweb99 the tester specifies a 'requested connections' value, and the server either meets it, or it doesnt. The Windows 2000 server maxed out at 1598, the Linux server at 4200 connections. You can request 1 million connections as well, but the testrun will not be compliant. Please check out this link for more information about SPECweb99. It's a complex benchmark.

And please do not ignore the lower-end benchmarks as well - single-Xe on server with 2GB RAM is not all that uncommon these days.

1) the maximum filesize in the SPECweb99 benchmark is 900kb, this is why there is a 1MB limit set. Your claim that there are 1MB objects in the benchmark is false.

2) the CGI executable is mandated by the SPECweb99 Run Rules. A process must be created and destroyed. But the total amount of CGI requests is 0.1%! All the other 99.9% of the workload was handled with IIS 'low application priority' modules, which is a DLL loaded into IIS's address space, not a .EXE.

3) the IIS object cache was set to 2GB (not 2MB). It's set to 2GB because Windows 2000 + IIS has a serious limitation, threads (such as the IIS threads) can only address 2GB. This is a design flaw in Windows 2000, which hunts them in the enterprise now.

4) are you really seriously promoting the idea that the top 4 PC OEMs (Dell, IBM, Compaq, HP) and Microsoft did not tune IIS to the max and somehow conspired in making Linux+TUX numbers look good?

Fact is, the only reason why the TUX result was compared to the same Dell system is that the Dell system also happened to have the fastest Windows 2000 results. Your whole line of argumentation is obviously flawed if you compare IBM's similar Windows 2000 SPECweb99 result to the TUX result.

1) the maximum filesize in the SPECweb99 benchmark is 900kb, this is why there is a 1MB limit set. Your claim that there are 1MB objects in the benchmark is false.

2) the CGI executable is mandated by the SPECweb99 Run Rules. A process must be created and destroyed. But the total amount of CGI requests is 0.1%! All the other 99.9% of the workload was handled with IIS 'low application priority' modules, which is a DLL loaded into IIS's address space, not a .EXE.

3) the IIS object cache was set to 2GB (not 2MB). It's set to 2GB because Windows 2000 + IIS has a serious limitation, threads (such as the IIS threads) can only address 2GB. This is a design flaw in Windows 2000, which hunts them in the enterprise now.

4) are you really seriously promoting the idea that the top 4 PC OEMs (Dell, IBM, Compaq, HP) and Microsoft did not tune IIS to the max and somehow conspired in making Linux+TUX numbers look good?

Fact is, the only reason why the TUX result was compared to the same Dell system is that the Dell system also happened to have the fastest Windows 2000 results. Your whole line of argumentation is obviously flawed if you compare IBM's similar Windows 2000 SPECweb99 result to the TUX result.

Reality is that these results are all well-tuned. The reason for the differences you noticed is that OS tuning is very different, even on the same hardware. One OS works better with large buffers, one with smaller buffers. Some parameters might not make any difference to performance, but are at some non-default value (and thus have to be reported to SPEC).

If you think that those Windows 2000 systems are not tuned well enough then more power to you, i'm sure you'll be hired immediately by any of these companies, good SPECweb99 performance is a top priority for every hardware vendor.

Reality is that these results are all well-tuned. The reason for the differences you noticed is that OS tuning is very different, even on the same hardware. One OS works better with large buffers, one with smaller buffers. Some parameters might not make any difference to performance, but are at some non-default value (and thus have to be reported to SPEC).

If you think that those Windows 2000 systems are not tuned well enough then more power to you, i'm sure you'll be hired immediately by any of these companies, good SPECweb99 performance is a top priority for every hardware vendor.

Reality is that these results are all well-tuned. The reason for the differences you noticed is that OS tuning is very different, even on the same hardware. One OS works better with large buffers, one with smaller buffers. Some parameters might not make any difference to performance, but are at some non-default value (and thus have to be reported to SPEC).

If you think that those Windows 2000 systems are not tuned well enough then more power to you, i'm sure you'll be hired immediately by any of these companies, good SPECweb99 performance is a top priority for every hardware vendor.

Reality is that these results are all well-tuned. The reason for the differences you noticed is that OS tuning is very different, even on the same hardware. One OS works better with large buffers, one with smaller buffers. Some parameters might not make any difference to performance, but are at some non-default value (and thus have to be reported to SPEC).

If you think that those Windows 2000 systems are not tuned well enough then more power to you, i'm sure you'll be hired immediately by any of these companies, good SPECweb99 performance is a top priority for every hardware vendor.

> Not very close, most busy sites don't have all static content.

You should understand the benchmark before you make blanket statements about it. SPECweb99 has 30% dynamic content (maybe fairly lightweight dynamic content, but it's a start).
See http://www.spec.org/osg/web99

What does this mean? Vendors obviously try to maximize # of connections, but they have to keep the bitrate above 320kbits to have a valid benchmark run. You can test with 1 million connections as well, but you'll get an invalid run because the kbit rate will be somewhere around 0.1kbits/sec. This is why you see almost identical kbits values (and all are a bit above 320kbits/sec), but different connections and ops/sec values. I hope this explains things.

See the SPEC-enforced Web99 Run Rules, there are alot of very strict requirements for a result to be accepted by SPEC.

oops, wrong URL, the right link for the source code is here. It's standard user-space code, you can output any dynamic page with TUX as well.

You assume that IBM, HP, Mindcraft and Dell are all in a big conspiracy to make Windows 2000 numbers look bad - are you kidding? The reality is that there is fierce competition for best SPECweb99 numbers, and Linux/TUX is just plain faster.

The other flaw in your argument is this TUX dynamic module. Check out the source code, TUX does dynamic modules. (besides, the SPECweb99 workload includes 30% dynamic load, so all SPECweb99 webservers must support dynamic applications.)

You assume that IBM, HP, Mindcraft and Dell are all in a big conspiracy to make Windows 2000 numbers look bad - are you kidding? The reality is that there is fierce competition for best SPECweb99 numbers, and Linux/TUX is just plain faster.

The other flaw in your argument is this TUX dynamic module. Check out the source code, TUX does dynamic modules. (besides, the SPECweb99 workload includes 30% dynamic load, so all SPECweb99 webservers must support dynamic applications.)

You assume that IBM, HP, Mindcraft and Dell are all in a big conspiracy to make Windows 2000 numbers look bad - are you kidding? The reality is that there is fierce competition for best SPECweb99 numbers, and Linux/TUX is just plain faster.

The other flaw in your argument is this TUX dynamic module. Check out the source code, TUX does dynamic modules. (besides, the SPECweb99 workload includes 30% dynamic load, so all SPECweb99 webservers must support dynamic applications.)

'TUX' comes from 'Threaded linUX webserver', and is a kernel-space HTTP subsystem. TUX was written by Red Hat and is based on the 2.4 kernel series. TUX is under the GPL and will be released in a couple of weeks. TUX's main goal is to enable high-performance webserving on Linux, and while it's not as feature-full as Apache, TUX is a 'full fledged' HTTP/1.1 webserver supporting HTTP/1.1 persistent (keepalive) connections, pipelining, CGI execution, logging, virtual hosting, various forms of modules, and many other webserver features. TUX modules can be user-space or kernel-space.

The SPECweb99 test was done with a user-space module, the source code can be found

here. We expect TUX to be integrated into Apache 2.0 or 3.0, as TUX's user-space kernel-space API is capable of supporting a mixed Apache/TUX webspace.

TUX uses a 'object cache' which is much more than a simple 'static cache'. TUX objects can be freely embedded in other web replies, and can be used by modules, including CGIs. You can 'mix' dynamically generated and static content freely.

While written by Red Hat, TUX relies on many scalability advances in the 2.4 kernel done also by kernel hackers from SuSE, Mandrake and the Linux Community as a whole. TUX is not one single piece of technology, rather a final product that 'connects the dots' and proves the scalability of Linux's high end features. I'd especially like to highlight the role of extreme TCP/IP networking scalability in 2.4, which was a many months effort lead by David Miller and Alexey Kuznetsov. We'd also like to acknowledge the pioneering role of khttpd - while TUX is independent of khttpd, it was an important experiment we learned alot from.

Other 2.4 kernel advances TUX uses are: async networking and disk IO, wake-one scheduling, interrupt binding, process affinity (not yet merged patch), per-CPU allocation pools (not yet merged patch), big file support (the TUX logfile can get bigger than 5GB during SPECweb99 runs), highmem support, various VFS enhancements (thanks Al Viro), the new IO-scheduler done by SuSE folks, buffer/pagecache scalability and many many other Linux features.

So why not make sure Linux wins a benchmark in an area where they know Linux is popular already? Fudged numbers and fudged hardware aside (we'll assume they were honest), it would certainly seem a logical thing for them to do. As is evident from the results page, they blew away the competition with TUX 1.0. I've been unable to find any information on this (please enlighten), but it appears that it's a kernel patch, because the options are set with the kernel interface. Is this the khttpd that was discussed after the Mindcraft fiasco?

In any case, if it's in kernelspace, it's most likely not a full-featured HTTP server like Apache, Zeus, IIS. So it can spit out static pages as fast as you'd possibly need. Big deal. Fireworks accelerate faster than space shuttles, but you wouldn't create dynamic content with fireworks. (Erm... where was I?)

My point is, Dell has proven that a specially designed static page server is faster than servers with more features. That doesn't really tell us anything we didn't know. It doesn't demonstrate that one OS is better than another, nor does it make deployment decisions any easier (except for fools).

> the unjustly-maligned Mindcraft benchmark

No, the malignance was just. Even though Mindcraft II addressed some of the obvious technical problems with the first round, they still ran an extremely odd benchmark on an extremely odd selection of hardware, which left them open to charges of having tuned the test to provide the desired results. (I.e., "Here's one we can win!") These charges were confirmed by the suite of benchmarks run by c't at about the same time, where Linux won on almost every test, even though there was a realistic and reasonable variety between the specific c't tests, rather than a single bizarre test as in Mindcraft. (Another poster has given a link to those results.)

Even though Red Hat was foolish enough to participate in Mindcraft II [*] and thereby gave the benchmarks an appearance of legitimacy, many of us said in advance that we would not accept the results if they did not use a more relistic benchmark on a more realistic selection of hardware. I, for one, still stand by that.

It's absurd to put any stock in a benchmark that is sponsored by a company with a direct interest in the outcome and that does not even reflect a standard benchmark.

[*] Or not, as the case may be. Perhaps they were just trying to get a close look at the behavior so that they could get started on their "add on". Indeed, this may be what happened - see the details and notice the "Each NIC IRQ bound to a different CPU; Each TUX thread's listening address bound to 1 NIC's associated network", which sound like a direct response to Mindcraft.

> I am slightly curious whether this "web server add-on" is available to consumers

The linked page says that the "HTTP Software", "Operating System", and "Supplemetal System" (whatever that is) will be available in August 2000, so it does sound a bit vaprous.

--

From http://www.spec.org/osg/web99/:

SPECweb99 is the next-generation SPEC benchmark for evaluating the performance of World Wide Web Servers. As the successor to SPECweb96, SPECweb99 continues the SPEC tradition of giving Web users the most objective and representative benchmark for measuring a system's ability to act as a web server. In response to rapidly advancing Web technology, the SPECweb99 benchmark includes many sophisticated and state-of-the-art enhancements to meet the modern demands of Web users of today and tomorrow:

• Standardized workload, agreed to by major players in WWW market
• Full disclosures available on this web site
• Stable implementation with no incomparable versions
• Measurement of simultaneous connections rather than HTTP operations
• Simulation of connections at a limited line speed
• Dynamic GETs, as well as static GETs; POST operations.
• Keepalives (HTTP 1.0) and persistent connections (HTTP 1.1).
• Dynamic ad rotation using cookies and table lookups.
• File accesses more closely matching today's real-world web server access patterns.
• An automated installation program for Microsoft Windows NT as well as Unix installation scripts.
• Inter-client communication using sockets.

It certainly looks like they are testing Dynamic content as well as static. Check out http://www.spec.org/osg/web99/results/ api-src/ for the source for the dynamic content.

From http://www.spec.org/osg/web99/:

SPECweb99 is the next-generation SPEC benchmark for evaluating the performance of World Wide Web Servers. As the successor to SPECweb96, SPECweb99 continues the SPEC tradition of giving Web users the most objective and representative benchmark for measuring a system's ability to act as a web server. In response to rapidly advancing Web technology, the SPECweb99 benchmark includes many sophisticated and state-of-the-art enhancements to meet the modern demands of Web users of today and tomorrow:

• Standardized workload, agreed to by major players in WWW market
• Full disclosures available on this web site
• Stable implementation with no incomparable versions
• Measurement of simultaneous connections rather than HTTP operations
• Simulation of connections at a limited line speed
• Dynamic GETs, as well as static GETs; POST operations.
• Keepalives (HTTP 1.0) and persistent connections (HTTP 1.1).
• Dynamic ad rotation using cookies and table lookups.
• File accesses more closely matching today's real-world web server access patterns.
• An automated installation program for Microsoft Windows NT as well as Unix installation scripts.
• Inter-client communication using sockets.

It certainly looks like they are testing Dynamic content as well as static. Check out http://www.spec.org/osg/web99/results/ api-src/ for the source for the dynamic content.

It's being replaced by SPEC's "CPU 2000" benchmark, but AMD don't seem to be reporting CPU 2000 results yet.