OS X 10.8 vs. Ubuntu On Apple Hardware, Benchmarked

An anonymous reader writes "OS X 10.8 has been benchmarked against Ubuntu Linux with some interesting results. From the tests on a Apple Mac Mini and Apple MacBook Pro, OS X Mountain Lion was clearly superior when it came to the graphics performance, but the rest of the time the operating systems performed quite closely with no clear winner. OS X also seems to have greater performance issues with solid-state drives than Linux."

Comments

surprise surprise

[shentino]

Apple hardware performs better when run by Apple device drivers.

News at 11.

Re:surprise surprise

[CAIMLAS]

The graphics drivers are written in house at NVidia and AMD. Apple doesn't actually write their own drivers. And the GPUs are just bog standard AMD, NVidia, and Intel GPUs (expect for some of the graphics switching.). There is not reason Linux should be at a disadvantage.

What, you mean aside from the fact that Linux drivers for all those respective device manufacturers don't really get a whole lot of attention from the developers compared to Windows and OS X?

But really, that's not important. What people need to pay attention to is that this is something done by Phoronix. This means you need to consider a couple things:

* This benchmark is almost meaningless. Time and time again, I have seen them (falsely) correlate data with an assumption.
* The review was done by someone who doesn't really know what they're talking about.
* These are synthetics. Without context or understanding of what the benchmarks are doing (there is no explanation) or what may have led to the
* The discrepancies are, in most cases, severe enough that you have to assume (at least) one of the following: their benchmark suite was not properly/identically configured for all architectures, or there are drastic implementation discrepancies within the benchmark tool they're using (eg. it wasn't designed but with a specific use case in mind).

The reason there is "no clear winner" is because it's all rubbish. They're throwing 100 things at two different targets and comparing what misses and saying "no conclusion". Really? You'd have better consistency with an ink blot test of random participants, with ink blots generated by a true random number generator.

Some of the graphics benchmarks don't stand out; the ones that stand out the most are the computational ones involving (very) standard libraries or frameworks which then contradict later results.

For instance, CompileBench and Threaded I/O Tester: OS X falls flat on its face. The threaded I/O tester I believe, because I've seen the same with db and server performance. But earlier, they've got bgbench giving OS X four times the performance for postgresql as Linux. Is that even rational, given that even FileMaker has shyed away from OS X as a preferred platform due to threading and filesystem performance?

Then, they go on to fail to explain these things and why they're fundamentally inconsistent. Not just "this doesn't quite line up, we can write it off due to different library version overhead" but in line with "this car goes faster because its engine is smaller". What?

On a more personal level, I have used their suite of benchmark tools and come away fairly underwhelmed by the results. They're inconsistent and inexplicable, such as those seen in this review.

Here's a hint, benchmarkers:

* when you benchmark something, you must compare things and try to figure out why they are performing as they do.
* If there are gross discrepancies which belay a reasonable expectation or contradicts other information, investigate it, because it's probably important
* Be sure of what you're comparing. If you've got (more or less) identical binaries on different platforms and the hardware, you're just comparing the kernel. Is that what's happening here? Are their tools linked against native libraries (which would, you know, be an honest benchmark of said platform) or do they use their own stack?

Anyway, I could go on, but you get the idea. This benchmark is stupid on its face. The only benchmarks I'd trust from this roundup here are those that are straight up "measure something real" (frames per second in x, time to complete concrete task y). They make a very different picture than when the synthetics are thrown in to the whole: overall OS X performance is pretty abysmal, but is marginally better at graphical things than Linux. This fits pretty close with my (personal) observation that OS X is about 10-15% slower than linux on general things, markedly slower on threaded things, and a dog at file manipulations while having a firm grasp on display management/graphical stuff - so it might just be my "uneducated Apple-hating bias" speaking. :)

Summary of tests?

[GoNINzo]

15 pages of a review, with a poor summary of the results, results in the most number of page views. It would have been nice if they had some sort of summary or benchmark to compare the two against rather than individual tests spread across this. Perhaps a summary chart?

Also, comparing a well tuned video device driver versus the (usually) hastily written Linux one is a poor comparison.

I really doubt people choose a mac over Linux over this kind of test. There more solid reasons to choose one or the other.

Re:Why is Linux's SSD performance so terrible?

[Sparticus789]

PEBKAC. When I upgraded my Ubuntu laptop to SSD, boot time was under 10 seconds and my battery life while surfing the internet went from 3 hours to almost 5 hours. Not all SSDs are made the same, you have to research the performance of each, power draw, etc.

That being said, I bought the SSD with the second-lowest power usage and middle-of-the-road performance.

Re:Why is Linux's SSD performance so terrible?

[cpu6502]

That's Not what the article said. It said OS X had performance issues with solid state drives.

Also I'm kinda curious: Why would spend twice as much to buy an Intel Mac PC if they're just running linux? I'd buy a regular PC for 1/2 to 2/3rd the cost.

Re:Why is Linux's SSD performance so terrible?

[0123456]

I made the mistake of "upgrading" two Ubuntu 12.04 desktops to solid state drives, only to find the performance increase was trivial.

If a process isn't disk-intensive, an SSD will make no difference. If it's not seek intensive, a cheap SSD may actually be worse; if I remember correctly, sustained reads from my 'Green' hard drive are 80-100MB per second, whereas one of my SSDs only gets about 40MB per second.

The big benefit is reduced seek time, and a lesser benefit from faster sustained reads on the more modern and/or expensive SSDs. It won't make games run faster unless they're streaming from disk, or improve CPU-intensive 3D rendering, or anything much else that doesn't require a lot of disk seeks.

Worthless...

[Thinine]

Yet another worthless benchmarking from Phoronix (Moronix, amirite?). They switch between compilers, compiler versions, and even use Xcode itself for some of these comparisons, which make it essentially worthless. Add to that absolutely zero investigations of the reason for differences between the platforms (aside from the obvious mention of graphics drivers) and this is yet another piece of benchmark porn from a site dedicated to it.

Re: Why an Intel Mac PC to run linux?

[sl956]

Also I'm kinda curious: Why would spend twice as much to buy an Intel Mac PC if they're just running linux? I'd buy a regular PC for 1/2 to 2/3rd the cost.

I looked for a silent small footprint linux pc. I was unable to find one. That's why I bought a Mac Mini. It runs Linux flawlessly... and silently thanks to the fanless design and SSD.

People wanting an HD screen on a laptop might also have to buy Apple hardware even though they plan to use only Linux.

Re:Why is Linux's SSD performance so terrible?

[omnichad]

Hope you mean TRIM and not defragmenting, which occurs when a file is deleted on an SSD, not when one is written. You don't defragment an SSD, as there's no gain at all.