Phoronix Releases Linux Benchmarking Distribution

Bitnit writes "Phoronix has released a major update to their automated Linux benchmarking software, the Phoronix Test Suite, and more interestingly they have released their own distribution that's designed for hardware testing and benchmarking on Linux. With PTS Desktop Live they provide this Linux distribution that's to run only from a live environment off a DVD / USB key and then allows their benchmarking software to run — and only that — on this standardized software stack, which makes hardware comparisons a lot easier."

Comments

Problems...

[Darkness404]

A lot of speed issues have to do with drivers and not the actual hardware itself. I wonder if this takes that into the equation.

Re:Problems...

[TheRaven64]

Does it matter? As a user, I don't care if poor performance is caused by bad drivers or by bad hardware.

Re:Problems...

[jd]

Well, to an extent you do. A bad driver is much easier to upgrade/replace than bad hardware, especially if the hardware is built into the motherboard. In the case of audio (a classic Linux problem), if ALSA has a bad/missing driver you are still free to use OSS, and vice versa. If X has a bad video driver, there may be one in GGI or in the Linux framebuffer that you can use. On the other hand, if the chipset is crud, all the software options in the world won't help you.

I'm afraid!

[Parker Lewis]

"The only required dependencies for the Phoronix Test Suite on Linux systems is PHP 5.x CLI"

Re:I'm afraid!

[Anonymous Coward]

Did anyone see this? What the fuck?

Errors on me when I run the gui

[ta bu shi da yu]

chris@ubuntu:~$ phoronix-test-suite gui

Fatal error: Class 'GtkWindow' not found in /usr/share/phoronix-test-suite/pts-core/objects/gtk/pts_gtk_window.php on line 23

Not working too well... :(

Re:Errors on me when I run the gui

[Anonymous Coward]

perhaps they could port to QT . . .

Re:Errors on me when I run the gui

[RalphSleigh]

I think the PHP Gtk binding are still a compile from source mission on ubuntu, there was not a package available when I needed it a few weeks ago. Plus if you leave the module in your default php config, it breaks php (as in won't run even non Gtk scripts) whenever it it run from somewhere without a window manager available (ssh, cron, etc).

All in all a pain in the ass.

Re:Errors on me when I run the gui

[Kryai]

I think the PHP Gtk binding are still a compile from source mission on ubuntu, there was not a package available when I needed it a few weeks ago. Plus if you leave the module in your default php config, it breaks php (as in won't run even non Gtk scripts) whenever it it run from somewhere without a window manager available (ssh, cron, etc).

All in all a pain in the ass.

I'm about to give up installing the requirements for a GUI on CentOS 5.3 trying to get php-gtk installed is quite the venture.

Wrong benchmarks

[Anonymous Coward]

While the suite is good for benchmarking regressions between kernel versions (across distros for example), it's not a very good distribution comparing tool in my opinion.

I'd like to see benchmarks such as:

* Time from bootloader to login window/desktop
* Time from desktop to webbrowser fully loaded Google or something
* Time it takes to open an Excel sheet with OpenOffice
* Amount of swap space used under normal desktop cirmumstances (some ~15 FF tabs, few sheets, docs, mail reader, etc)

It doesn't matter if it differs +-2 fps in some game, or a MB/s on storage throughput. It's the whole integration and system together that determines what the distribution is etc.

Re:Wrong benchmarks

[ta bu shi da yu]

So what's stopping you? :-) They give instructions on how to do this here. The only thing it probably won't do well is bootup testing, but then you probably want something like Bootchart for that.

Re:Wrong benchmarks

[Fieryphoenix]

Read the title again. This is not a Linux Distribution Benchmark. It is a Linux Benchmarking Distribution.

Speed - running off the CD

[yossarianuk]

LiveCD/DVD's are always slower then a proper HD install.

Will this not make peoples bechmark results slower ? This would invalidate the benchmark results.

Re:Speed - running off the CD

[roger_that]

Once the test is loaded, it runs from memory, and CD speed will not (should not) affect the test results.

Re:Speed - running off the CD

[Aim Here]

Why would it invalidate them? The results will be slower for everyone, since the LiveCD would standardise the software environment.

Anyways, it only matters if the suite reads from the disk during a test for some boneheaded reason.

VirtualBox

[owlman17]

Sounds great! I'll download it and try to run it in a VM.

Re:VirtualBox

[Anonymous Coward]

Now there's a good idea. I mean, running in a VM wouldn't skew the results at all.

Re:VirtualBox

[Cormacus]

nooo . . . but it would give you an idea of how much of a performance hit you take when running in a VM. Which could be interesting in and of itself.

Re:VirtualBox

[geekboybt]

That's the point, at least in my case. My Master's thesis/project involves benchmarking multiple VM systems - Xen, OpenVZ, and KVM. Unfortunately, VMware's out at this point due to licensing issues, though I'm probably going to run them anyway for my own personal use.

Re:VirtualBox

[atomic-penguin]

Um, is there anything that explicitly prohibits you from benchmarking and publishing the results of performance on VMWare ESXi or GSX? Both of them are free downloads from vmware.com with a valid e-mail address.

Re:VirtualBox

[Bert64]

Yes, the license terms which you must agree to before you can go on with the download...

Re:VirtualBox

[Anonymous Coward]

ESXi is free, as well as Microsoft Hyper-V Server.

Both VMWare and Microsoft have free VMs.

Re:VirtualBox

[geekboybt]

"You may use the Software to conduct internal performance testing and benchmarking studies, the results of which you (and not unauthorized third parties) may publish or publicly disseminate; provided that VMware has reviewed and approved of the methodology, assumptions and other parameters of the study. Please contact VMware at benchmark@vmware.com to request such review." From http://www.vmware.com/download/eula/server.html

Re:VirtualBox

[atomic-penguin]

Actual license terms verbiage:

You may use the Software to conduct internal performance testing and benchmarking studies, the results of which you (and not unauthorized third parties) may publish or publicly disseminate; provided that VMware has reviewed and approved of the methodology, assumptions and other parameters of the study. Please contact VMware at benchmark@vmware.com to request such review.

To the parent poster, IANAL however, if nobody outside your University or dissertation committee ever even reviews your paper, that would certainly be acceptable use under these terms. Otherwise you can shoot them an e-mail, explain your situation and ask permission, you might even be surprised by the response.

Re:VirtualBox

[atomic-penguin]

So you e-mailed them at the given address, and they told you that you could not use VMWare benchmark results in your graduate dissertation?

Great!

[Simian Man]

Now anyone can conduct highly inaccurate bench-marking tests and publish the results every few months!

Re:Great!

[ta bu shi da yu]

But if we average out the results over a year the inaccuracy will surely decrease!

Re:Great!

[Anonymous Coward]

True. Now we can see, how much the choice of file system effects on the mp3 encoding speed.

A GIMP work-a-like ... eh

[noshellswill]

Un-fathomable, un-usable and un-speakable. Fit only for Slackware and Debiolian retros.

Phoronix == phoenix + moron

[Anonymous Coward]

A moron that rises from the ashes every few months with a fucked up benchmark.