Phoronix Releases Linux Benchmarking Distribution

Bitnit writes "Phoronix has released a major update to their automated Linux benchmarking software, the Phoronix Test Suite, and more interestingly they have released their own distribution that's designed for hardware testing and benchmarking on Linux. With PTS Desktop Live they provide this Linux distribution that's to run only from a live environment off a DVD / USB key and then allows their benchmarking software to run — and only that — on this standardized software stack, which makes hardware comparisons a lot easier."

Comments

Problems...

[Darkness404]

A lot of speed issues have to do with drivers and not the actual hardware itself. I wonder if this takes that into the equation.

Re:Problems...

[jd]

Well, to an extent you do. A bad driver is much easier to upgrade/replace than bad hardware, especially if the hardware is built into the motherboard. In the case of audio (a classic Linux problem), if ALSA has a bad/missing driver you are still free to use OSS, and vice versa. If X has a bad video driver, there may be one in GGI or in the Linux framebuffer that you can use. On the other hand, if the chipset is crud, all the software options in the world won't help you.

I'm afraid!

[Parker Lewis]

"The only required dependencies for the Phoronix Test Suite on Linux systems is PHP 5.x CLI"

Wrong benchmarks

[Anonymous Coward]

While the suite is good for benchmarking regressions between kernel versions (across distros for example), it's not a very good distribution comparing tool in my opinion.

I'd like to see benchmarks such as:

* Time from bootloader to login window/desktop
* Time from desktop to webbrowser fully loaded Google or something
* Time it takes to open an Excel sheet with OpenOffice
* Amount of swap space used under normal desktop cirmumstances (some ~15 FF tabs, few sheets, docs, mail reader, etc)

It doesn't matter if it differs +-2 fps in some game, or a MB/s on storage throughput. It's the whole integration and system together that determines what the distribution is etc.

Re:Wrong benchmarks

[ta bu shi da yu]

So what's stopping you? :-) They give instructions on how to do this here. The only thing it probably won't do well is bootup testing, but then you probably want something like Bootchart for that.

Re:Speed - running off the CD

[roger_that]

Once the test is loaded, it runs from memory, and CD speed will not (should not) affect the test results.

VirtualBox

[owlman17]

Sounds great! I'll download it and try to run it in a VM.

Re:VirtualBox

[Cormacus]

nooo . . . but it would give you an idea of how much of a performance hit you take when running in a VM. Which could be interesting in and of itself.

Re:VirtualBox

[geekboybt]

"You may use the Software to conduct internal performance testing and benchmarking studies, the results of which you (and not unauthorized third parties) may publish or publicly disseminate; provided that VMware has reviewed and approved of the methodology, assumptions and other parameters of the study. Please contact VMware at benchmark@vmware.com to request such review." From http://www.vmware.com/download/eula/server.html

Re:Speed - running off the CD

[Aim Here]

Why would it invalidate them? The results will be slower for everyone, since the LiveCD would standardise the software environment.

Anyways, it only matters if the suite reads from the disk during a test for some boneheaded reason.

Great!

[Simian Man]

Now anyone can conduct highly inaccurate bench-marking tests and publish the results every few months!

Re:Great!

[ta bu shi da yu]

But if we average out the results over a year the inaccuracy will surely decrease!