I would like to discuss how the benchmark could be improved. In its current form: 1) It is a client/server test with web clients talking to an Apache2 web server, 2) The server runs SuSE Linux Enterprise Server, 3) The server's database tables are built on MySQL, 4) The transaction is a gasoline credit card purchase, 5) The test measures power consumed at 7 different transaction activity levels: Idle, 5 different constant transaction rates and the maximum that the server will deliver, 6) At each activity level the benchmark collects power used for 30 minutes, 7) The test reports its data for all levels tested, 8) The transactions are coded so that as the level of user activity increases larger and larger areas of the database tables are accessed. This means that at lower user counts the disk I/O is cached and the test is calculation intensive while at higher user counts the database working set may exceed the kernel disk cache size and thus the test is limited by physical disk I/O.
Many real world servers process web transactions against RDMSs and are idle evenings and weekends. This test lets people: 1) compare the maximum throughput of different machines, 2) Review the power consumed at maximum throughput, 3) Review the power used at various intermediate transaction arrival rates, and 4) Review the power consumed at idle.
Would this be a better benchmark with Oracle rather than MySQL or RedHat rather than SuSE? Would it be a better test without the client/server network traffic? Would it be better if is was based on floating point calculations that did not do any database access or disk I/O? What can be done to make this a more useful benchmark? Neal Nelson
You are correct that NNA hopes to generate some business from these tests but I feel that your other speculations are wrong. The "worlds-fastest" web site is not a put up job by AMD. It is true that several recent sets of test results show AMD as superior but that could change, possibly as soon as Xeon servers are available with DDR2 RAM. The test methodology is valid and when Intel starts building servers that use less power, the test will report that.
I don't believe that MySQL or SuSE linux has a bias toward AMD. I know that the NNA code (some of which was listed in the white paper appendix) was not written to favor any vendor.
Regarding the Netburst test, an Inquirer story in the same time frame reported that although the Core chips had been out for a while, 4 out of 5 processors being shipped by Intel at that point in time were Netburst processors. So it was reasonable to test Netburst because 4 out of 5 people buying Intel machines were getting Netburst. The report clearly identified the processors involved and both were models and revisions that were actively being sold as new products at the time of the test. I also feel that the test was worthwhile because I believe it was the first publicly released client/server transaction test where the CPU clock, memory size, disk drives, host adapter, operating system, application code and system tunables were all exactly the same. Neal Nelson
I read your reply but I do not see any recommended settings. Can you provide some link to a "howto" or some published paper where this information is provided for the Xeon and the Opteron? Thanks, Neal Nelson
Your statement "They tweaked all kinds of shit...." is incorrect. Appendix B in the white paper lists the two changes that we made to the Apache2 configuration files. One increased the number of user sessions and the other turned off logging. As noted in the text of the white paper we also set the BIOS fan speed control to automatic. I hardly think that these changes can be accurately described as tweaking "all kinds of shit". If you will send me a list of what you consider to be the proper tweaking changes for both the Xeon and the Opteron I will run a partial set of tests and publish: 1) Xeon before your changes compared to Xeon after your changes, 2) Operon before your changes compared to Opteron after your changes, and 3) Xeon after your changes compared to Opteron after your changes. Neal Nelson
Stanley, You say "The only widely available chipsets for these Xeons use FB-DIMMs," I want to do a comparison test of Xeon DDRII to Xeon FB-DIMM. I did a search for a motherboard with the Xeon Woodcrest and DDR-II. I could not find one anywhere. Do you know if anyone makes a motherboard for the Xeon Woodcrest with DDR-II? If so, who? Thanks, Neal Nelson
These tests were not run with the new G stepping. If someone can loan me a pair of the new chips for about a week I will re-run the tests and promptly publish the results.
Neal Nelson
Is anyone aware of any other published power efficiency data? It would be pretty easy to plug in a "Kill A Watt" or "Watts Up" device and measure the power at idle. Is there any data for other server configurations? Has anybody compared an Intel "Desktop Server" with DDRII to a Xeon based server with FB-DIMMS? Has anybody reported idle power under Windows versus idle power under Linux on the same machine?
Slashdot Mirror
I would like to discuss how the benchmark could be improved. In its current form: 1) It is a client/server test with web clients talking to an Apache2 web server, 2) The server runs SuSE Linux Enterprise Server, 3) The server's database tables are built on MySQL, 4) The transaction is a gasoline credit card purchase, 5) The test measures power consumed at 7 different transaction activity levels: Idle, 5 different constant transaction rates and the maximum that the server will deliver, 6) At each activity level the benchmark collects power used for 30 minutes, 7) The test reports its data for all levels tested, 8) The transactions are coded so that as the level of user activity increases larger and larger areas of the database tables are accessed. This means that at lower user counts the disk I/O is cached and the test is calculation intensive while at higher user counts the database working set may exceed the kernel disk cache size and thus the test is limited by physical disk I/O.
Many real world servers process web transactions against RDMSs and are idle evenings and weekends. This test lets people: 1) compare the maximum throughput of different machines, 2) Review the power consumed at maximum throughput, 3) Review the power used at various intermediate transaction arrival rates, and 4) Review the power consumed at idle.
Would this be a better benchmark with Oracle rather than MySQL or RedHat rather than SuSE? Would it be a better test without the client/server network traffic? Would it be better if is was based on floating point calculations that did not do any database access or disk I/O? What can be done to make this a more useful benchmark? Neal Nelson
You are correct that NNA hopes to generate some business from these tests but I feel that your other speculations are wrong. The "worlds-fastest" web site is not a put up job by AMD. It is true that several recent sets of test results show AMD as superior but that could change, possibly as soon as Xeon servers are available with DDR2 RAM. The test methodology is valid and when Intel starts building servers that use less power, the test will report that.
I don't believe that MySQL or SuSE linux has a bias toward AMD. I know that the NNA code (some of which was listed in the white paper appendix) was not written to favor any vendor.
Regarding the Netburst test, an Inquirer story in the same time frame reported that although the Core chips had been out for a while, 4 out of 5 processors being shipped by Intel at that point in time were Netburst processors. So it was reasonable to test Netburst because 4 out of 5 people buying Intel machines were getting Netburst. The report clearly identified the processors involved and both were models and revisions that were actively being sold as new products at the time of the test. I also feel that the test was worthwhile because I believe it was the first publicly released client/server transaction test where the CPU clock, memory size, disk drives, host adapter, operating system, application code and system tunables were all exactly the same. Neal Nelson
I checked both the Xeon and Opteron servers, and just as Wesley said ondemand governor was already set on both boxes. Neal Nelson
I read your reply but I do not see any recommended settings. Can you provide some link to a "howto" or some published paper where this information is provided for the Xeon and the Opteron? Thanks, Neal Nelson
Your statement "They tweaked all kinds of shit ...." is incorrect. Appendix B in the white paper lists the two changes that we made to the Apache2 configuration files. One increased the number of user sessions and the other turned off logging. As noted in the text of the white paper we also set the BIOS fan speed control to automatic. I hardly think that these changes can be accurately described as tweaking "all kinds of shit". If you will send me a list of what you consider to be the proper tweaking changes for both the Xeon and the Opteron I will run a partial set of tests and publish: 1) Xeon before your changes compared to Xeon after your changes, 2) Operon before your changes compared to Opteron after your changes, and 3) Xeon after your changes compared to Opteron after your changes. Neal Nelson
Stanley, You say "The only widely available chipsets for these Xeons use FB-DIMMs," I want to do a comparison test of Xeon DDRII to Xeon FB-DIMM. I did a search for a motherboard with the Xeon Woodcrest and DDR-II. I could not find one anywhere. Do you know if anyone makes a motherboard for the Xeon Woodcrest with DDR-II? If so, who? Thanks, Neal Nelson
These tests were not run with the new G stepping. If someone can loan me a pair of the new chips for about a week I will re-run the tests and promptly publish the results. Neal Nelson
Is anyone aware of any other published power efficiency data? It would be pretty easy to plug in a "Kill A Watt" or "Watts Up" device and measure the power at idle. Is there any data for other server configurations? Has anybody compared an Intel "Desktop Server" with DDRII to a Xeon based server with FB-DIMMS? Has anybody reported idle power under Windows versus idle power under Linux on the same machine?