OS X 10.8 vs. Ubuntu On Apple Hardware, Benchmarked

An anonymous reader writes "OS X 10.8 has been benchmarked against Ubuntu Linux with some interesting results. From the tests on a Apple Mac Mini and Apple MacBook Pro, OS X Mountain Lion was clearly superior when it came to the graphics performance, but the rest of the time the operating systems performed quite closely with no clear winner. OS X also seems to have greater performance issues with solid-state drives than Linux."

Comments

surprise surprise

[shentino]

Apple hardware performs better when run by Apple device drivers.

News at 11.

Re:surprise surprise

[maccodemonkey]

For graphics, what Apple device drivers?

The graphics drivers are written in house at NVidia and AMD. Apple doesn't actually write their own drivers. And the GPUs are just bog standard AMD, NVidia, and Intel GPUs (expect for some of the graphics switching.). There is not reason Linux should be at a disadvantage.

And if they did I'd expect worse performance. Back when Apple used to write their own drivers they were totally awful. Apple has less experience writing graphics drivers, I'm not actually sure why you'd expect Apple written drivers to perform better.

Re:surprise surprise

[shentino]

I love how you're biased in calling linux a hassle to program and saying that one must know how to script to use it properly.

There are actually, contrary to rumor, a few user friendly distros out there that don't require a PhD in computer science to make use of.

And Microsoft at least HAS been caught hiding APIs that gives its own programs a performance advantage.

My comment isn't about which is better, anyway. It's about which ones cheat on their benchmarks by giving themselves a proprietary boost not available to the competition.

See also the scandal of either nvidia or ati making its own hardware's performance deliberately go down the crapper when it detected the competition's chips.

Re:surprise surprise

[CAIMLAS]

The graphics drivers are written in house at NVidia and AMD. Apple doesn't actually write their own drivers. And the GPUs are just bog standard AMD, NVidia, and Intel GPUs (expect for some of the graphics switching.). There is not reason Linux should be at a disadvantage.

What, you mean aside from the fact that Linux drivers for all those respective device manufacturers don't really get a whole lot of attention from the developers compared to Windows and OS X?

But really, that's not important. What people need to pay attention to is that this is something done by Phoronix. This means you need to consider a couple things:

* This benchmark is almost meaningless. Time and time again, I have seen them (falsely) correlate data with an assumption.
* The review was done by someone who doesn't really know what they're talking about.
* These are synthetics. Without context or understanding of what the benchmarks are doing (there is no explanation) or what may have led to the
* The discrepancies are, in most cases, severe enough that you have to assume (at least) one of the following: their benchmark suite was not properly/identically configured for all architectures, or there are drastic implementation discrepancies within the benchmark tool they're using (eg. it wasn't designed but with a specific use case in mind).

The reason there is "no clear winner" is because it's all rubbish. They're throwing 100 things at two different targets and comparing what misses and saying "no conclusion". Really? You'd have better consistency with an ink blot test of random participants, with ink blots generated by a true random number generator.

Some of the graphics benchmarks don't stand out; the ones that stand out the most are the computational ones involving (very) standard libraries or frameworks which then contradict later results.

For instance, CompileBench and Threaded I/O Tester: OS X falls flat on its face. The threaded I/O tester I believe, because I've seen the same with db and server performance. But earlier, they've got bgbench giving OS X four times the performance for postgresql as Linux. Is that even rational, given that even FileMaker has shyed away from OS X as a preferred platform due to threading and filesystem performance?

Then, they go on to fail to explain these things and why they're fundamentally inconsistent. Not just "this doesn't quite line up, we can write it off due to different library version overhead" but in line with "this car goes faster because its engine is smaller". What?

On a more personal level, I have used their suite of benchmark tools and come away fairly underwhelmed by the results. They're inconsistent and inexplicable, such as those seen in this review.

Here's a hint, benchmarkers:

* when you benchmark something, you must compare things and try to figure out why they are performing as they do.
* If there are gross discrepancies which belay a reasonable expectation or contradicts other information, investigate it, because it's probably important
* Be sure of what you're comparing. If you've got (more or less) identical binaries on different platforms and the hardware, you're just comparing the kernel. Is that what's happening here? Are their tools linked against native libraries (which would, you know, be an honest benchmark of said platform) or do they use their own stack?

Anyway, I could go on, but you get the idea. This benchmark is stupid on its face. The only benchmarks I'd trust from this roundup here are those that are straight up "measure something real" (frames per second in x, time to complete concrete task y). They make a very different picture than when the synthetics are thrown in to the whole: overall OS X performance is pretty abysmal, but is marginally better at graphical things than Linux. This fits pretty close with my (personal) observation that OS X is about 10-15% slower than linux on general things, markedly slower on threaded things, and a dog at file manipulations while having a firm grasp on display management/graphical stuff - so it might just be my "uneducated Apple-hating bias" speaking. :)

Why is Linux's SSD performance so terrible?

[MrEricSir]

I made the mistake of "upgrading" two Ubuntu 12.04 desktops to solid state drives, only to find the performance increase was trivial.

What gives? The difference between magnetic drives and SSDs on OS X is incredible. Is this a driver issue, or what?

Re:Why is Linux's SSD performance so terrible?

[Sparticus789]

PEBKAC. When I upgraded my Ubuntu laptop to SSD, boot time was under 10 seconds and my battery life while surfing the internet went from 3 hours to almost 5 hours. Not all SSDs are made the same, you have to research the performance of each, power draw, etc.

That being said, I bought the SSD with the second-lowest power usage and middle-of-the-road performance.

Re:Why is Linux's SSD performance so terrible?

[Hatta]

Linux caches disk reads pretty aggressively. If you have plenty of RAM, you might only notice a difference the first time you start an app.

Re:Why is Linux's SSD performance so terrible?

[cpu6502]

That's Not what the article said. It said OS X had performance issues with solid state drives.

Also I'm kinda curious: Why would spend twice as much to buy an Intel Mac PC if they're just running linux? I'd buy a regular PC for 1/2 to 2/3rd the cost.

Re:Why is Linux's SSD performance so terrible?

[0123456]

I made the mistake of "upgrading" two Ubuntu 12.04 desktops to solid state drives, only to find the performance increase was trivial.

If a process isn't disk-intensive, an SSD will make no difference. If it's not seek intensive, a cheap SSD may actually be worse; if I remember correctly, sustained reads from my 'Green' hard drive are 80-100MB per second, whereas one of my SSDs only gets about 40MB per second.

The big benefit is reduced seek time, and a lesser benefit from faster sustained reads on the more modern and/or expensive SSDs. It won't make games run faster unless they're streaming from disk, or improve CPU-intensive 3D rendering, or anything much else that doesn't require a lot of disk seeks.

Re:Why is Linux's SSD performance so terrible?

[willy_me]

I would guess it is because OS X defragments the drive as it is being written. The overhead is largely not noticed when writing to a traditional hard drive while. Due to SSDs greater speed, it will make it appear that OS X has performance issues. The thing about performing inline defragmentation is it improves speed as the computer ages and as the HD begins to fill. Because all of the benchmarks were performed with fresh systems, the benefits of a defragmented drive would not be noticed.

The question I have is with the low seek times of SSDs, is there still a need to defragment drives? Probably, but to what degree as it surely is not as important as when one is using a traditional hard drive.

Re:Why is Linux's SSD performance so terrible?

[Sparticus789]

Defragmenting SSDs is not recommended, as it causes unnecessary use of the storage transistors. The speed associated with a SSD is a result of any block of data being accessible at any time, no hardware movement required.

Re:Why is Linux's SSD performance so terrible?

[omnichad]

Hope you mean TRIM and not defragmenting, which occurs when a file is deleted on an SSD, not when one is written. You don't defragment an SSD, as there's no gain at all.

Re:Why is Linux's SSD performance so terrible?

[CharlyFoxtrot]

Footprint, noise (or lack thereof), the ability to run all major OS (OSX, Windows, Linux) on the same machine, low power usage and nice looking sturdy construction. If you're going to be putting it on a desk the Mini is a nice little package.

Summary of tests?

[GoNINzo]

15 pages of a review, with a poor summary of the results, results in the most number of page views. It would have been nice if they had some sort of summary or benchmark to compare the two against rather than individual tests spread across this. Perhaps a summary chart?

Also, comparing a well tuned video device driver versus the (usually) hastily written Linux one is a poor comparison.

I really doubt people choose a mac over Linux over this kind of test. There more solid reasons to choose one or the other.

Worthless...

[Thinine]

Yet another worthless benchmarking from Phoronix (Moronix, amirite?). They switch between compilers, compiler versions, and even use Xcode itself for some of these comparisons, which make it essentially worthless. Add to that absolutely zero investigations of the reason for differences between the platforms (aside from the obvious mention of graphics drivers) and this is yet another piece of benchmark porn from a site dedicated to it.

Re: Why an Intel Mac PC to run linux?

[sl956]

Also I'm kinda curious: Why would spend twice as much to buy an Intel Mac PC if they're just running linux? I'd buy a regular PC for 1/2 to 2/3rd the cost.

I looked for a silent small footprint linux pc. I was unable to find one. That's why I bought a Mac Mini. It runs Linux flawlessly... and silently thanks to the fanless design and SSD.

People wanting an HD screen on a laptop might also have to buy Apple hardware even though they plan to use only Linux.

Wrong interpretation of the results, favors OSX

[tstrunk]

If you read the whole article you will see that there are many computing intensive benchmarks, where Linux outperforms OSX by nearly a factor of two. Saying that there is no noticeable difference is simply wrong (see Page 11, Page 12).

Lots of graphs, worthless analysis

[CadentOrange]

The compilation benchmarks are not comparable as the compilers are different, not only in version number but in architecture! OS X ships with llvm-gcc, which is a different compiler from GCC. Think of it a LLVM pretending to be GCC (accepting GCC options, etc) for backward compatibility. This would explain the huge discrepancies between the results of the compilation benchmarks

Disk performance is another thorny issue. The Postmark benchmark shows Ubuntu 12.04 being 3x faster than OS X 10.8 (246 tps vs 80 tps), yet the postgresql database benchmark shows OS X to be 3x faster than Ubuntu. No explanation is even attempted. Why? Readers would like to know! How can OS X be faster at a database benchmark when a raw disk benchmark shows it to be a lot slower than Ubuntu?! Perhaps there's something screwy with the configuration of Postgres on Ubuntu? Does this mean that OS X is *THE* choice for hosting busy databases? My suspicion is that this is due to fsync (http://www.postgresql.org/docs/8.1/static/runtime-config-wal.html). If fsync is enabled, the database waits for the transaction log to be flushed to disk every time a transaction is committed. It's basically down to defaults, and who knows what the default values are for Postgresql on OS X vs Ubuntu?

The graphs raise far too many questions that are not addressed. Many of them should have raised warning flags, like the one about disk performance vs actual database performance. As such, the results are thoroughly suspect and no reasonable conclusions can be drawn. Pity, because they clearly have the kit just not the knowhow.

Re: Why an Intel Mac PC to run linux?

[CharlyFoxtrot]

I have the older style Mini and when the HDD goes to sleep and it runs on SSD-only it's damn near completely silent. The fan will only come on when really stressing the cpu.