Slashdot Mirror


Ubuntu 8.10 vs. Mac OS X 10.5.5 Benchmarks

An anonymous reader writes "As a sequel to their Is Ubuntu Getting Slower? Phoronix now has out an article that compares the performance of Ubuntu 8.10 to Apple's Mac OS X 10.5.5. They tested both the x86 and x86_64 spins of Ubuntu and threw at both operating systems a number of graphics, disk, computational, and Java benchmarks, among others. With the Mac Mini used in some of the comparisons, 'Leopard' was faster, while in others it was a tight battle."

32 of 328 comments (clear)

  1. We musn't fight each other... by PinkyDead · · Score: 3, Funny

    Surely we should be united against the common enemy.

    --
    Genesis 1:32 And God typed :wq!
    1. Re:We musn't fight each other... by rvw · · Score: 5, Insightful

      Surely we should be united against the common enemy.

      It's not fight, it's play. And when one system wins in terms of speed or usability, both systems win in terms of a weaker common enemy.

    2. Re:We musn't fight each other... by aliquis · · Score: 5, Insightful

      This thread will end up getting moderated flame-bait, but what would that common enemy be? Personally I think Windows is rather ok now, Windows 7 will probably be even better, who knows, maybe even better than snow leopard.

      The only thing I see as an enemy is ideas which are pushed down my throat no matter what if I want them or not. I want to use my data and my applications in the way I feel like, not be forced to a single method just because someone else thought it was the best one. But that is true for all operating systems and no special "enemy."

      I like many things in OS X and in applications for it because it makes sense and makes using the computer more comfortable, I don't like some other things because they don't let me do the things I want to do.

      The huge amount of applications for Windows makes it rather likely that you can find one which fits your purpose, some for the window managers and such in the free unix-like oses.

    3. Re:We musn't fight each other... by Anonymous Coward · · Score: 3, Insightful

      It's not really about Windows, it's about Microsoft. I don't care if Windows is coded by the best programmers in the world, the problem comes from management and their shoddy business tactics.

    4. Re:We musn't fight each other... by Rik+Sweeney · · Score: 3, Funny

      OK, so who are the following:

      Unicron: A giant Steve Balmer head?
      Galvatron: OS X?
      Hot Rod: Ubuntu?

    5. Re:We musn't fight each other... by Koiu+Lpoi · · Score: 5, Funny

      Yeah, one produces great hardware and software while locking you in, and the other is a failing monopoly! Oh, wait, those don't sound the same at all. I'm going to have to ask you to explain yourself.

  2. I don't get it really by zappepcs · · Score: 5, Informative

    It's a lengthy read, and there isn't much in there to say that Ubuntu has any real work to do. Seems like they were comparing two Ferrari race cars and commenting on the differences in interiors... to use a car analogy.

    I've just upgraded 8 systems to 8.10 and am quite happy. I was concerned over real world issues about the upgrade from early reports. The old IBM T22 with 256MB RAM was my test case. Guess what? The upgrade went as fast as my Wireless G card would allow it, after a reboot, and then an update last night, it is working a bit better than with 8.04 from a layman's point of view. Yes, it can drag now and then, but is resource limited severely. After the upgrade I did not have to tweak anything, and any problems I was having prior are now fixed. I appear to have fscked up a setting on the wireless networking, but now it's all good. As far as I am concerned, with two older laptops upgraded, and 3 older desktops upgraded, all with ZERO defects, Ubuntu continues to impress me. I will continue to give out CDs free to anyone that wants to improve their computing life.

    Now, if you just have to have the 'perfect' gaming machine... go ahead and worry about little things. As for the rest of the world, 8.10 is rocking awesomeness.

    1. Re:I don't get it really by zappepcs · · Score: 5, Interesting

      Yeah, I forgot to mention that 8 upgrades cost me nothing but time. One thing that I wish Canonical would do is to set up a donation fund where I could donate say $50 per install and know that all the apps that come with Ubuntu would get a reasonably fair share of that money. Is anyone at Canonical listening?

      AS it is now, I have to donate separately to those projects which I feel that I use enough to donate to. Trouble is that some projects which I do use are not readily recognizable as such. The Samba project is one such case. Ubuntu and others more-or-less hide its use from the user so they would be unaware that they are using it. I think this would go a long way toward helping various projects. Even if all Samba got from my $50 was $0.75. That is still a donation. In my case it would be eight times that. Yes, I do contribute to F/OSS projects, EFF, and several other groups who have my best interests at heart... well, our interests coincide.

      Another thing that Canonical could do, short of setting up such a fund, is write a small app that lists the apps being used on any installation and allow the user to save the list to disk which would include the designated donation web page for that project. That's not quite as good as a donation fund, but would still help the smaller projects by announcing their use and value.

      I like a good value as much as the next guy, and there is something satisfying about paying a very fair price for someone's work when it is valuable to yourself. I just wish it was easier to do.

  3. More of a summary by ojintoad · · Score: 5, Informative

    Apple's Mac OS X 10.5.5 "Leopard" had strong performance leads over Canonical's Ubuntu 8.10 "Intrepid Ibex" in the OpenGL performance with the integrated Intel graphics, disk benchmarking, and SQLite database in particular. Ubuntu on the other hand was leading in the compilation and BYTE Unix Benchmark. In the audio/video encoding and PHP XML tests the margins were smaller and no definitive leader had emerged. With the Java environment, Sunflow and Bork were faster in Mac OS X, but the Intrepid Ibex in SciMark 2 attacked the Leopard. These results though were all from an Apple Mac Mini.

    Also worth mentioning are the collection of posts from the last thread that convincingly argued various problems with the Phoronix Benchmarks.
    Example 1
    Example 2
    Example 3

    Speed tests are good, let's make sure we're doing them right

    1. Re:More of a summary by Milyardo · · Score: 4, Insightful

      Apple's Mac OS X 10.5.5 "Leopard" had strong performance leads over Canonical's Ubuntu 8.10 "Intrepid Ibex" in the OpenGL performance with the integrated Intel graphics, disk benchmarking, and SQLite database in particular. Ubuntu on the other hand was leading in the compilation and BYTE Unix Benchmark. In the audio/video encoding and PHP XML tests the margins were smaller and no definitive leader had emerged. With the Java environment, Sunflow and Bork were faster in Mac OS X, but the Intrepid Ibex in SciMark 2 attacked the Leopard. These results though were all from an Apple Mac Mini.

      Also worth mentioning are the collection of posts from the last thread that convincingly argued various problems with the Phoronix Benchmarks. Example 1 Example 2 Example 3

      Speed tests are good, let's make sure we're doing them right

      Every one of those examples are fail at reasoning weaknesses in the Phoronix Test Suite and this is why:

      Example 1

      If you look closely you'll notice that (a) the benchmarks were run on a Thinkpad T60 laptop, and (b) there were significant differences on some benchmarks like RAM bandwidth that should have little or no OS components.

      If you look closely you'll notice that (a) the laptop the benchmarks are run on effects in no way, the validity of the benchmark as long as they are run consistently on the same laptop and (b) some benchmarks like RAM bandwidth have theoretical limits that are not effected at all by the Operating System but in actual practice, is entirely limited by the operating system you are using.

      Example 2

      Some of the benchmarks were hardware testing, and those showed variation. They should not, unless the compiler changed the algorithms used to compile the code between distros.

      All of the benchmarks were testing the hardware and should have showed variation. The compilers used on all the benchmarking applications are all the same. But the compilers used to build the Operating Systems are all completely different versions. Therefore the compiler on each distro will compile the same "algorithm" slightly different way. That is assuming there were no changes between implementation of packages between distros (of which there were actually hundreds of thousands of changes in the code itself, build options, and runtime configurations)

      Example 3

      The test suite itself: The Phoronix test suite runs on PHP. That in itself is a problem-- the slowdowns measured could most likely be *because* of differences in the distributed PHP runtimes.

      The Phoronix-Test-Suite Only uses its PHP back-end to aggregate benchmarking information. If a compilation with GCC took 5 seconds, its going to take 5 seconds no matter what version of the PHP runtime is used to to start the sub-shell that GCC runs in. It's take the same amount of time if you invoked GCC from bash, from perl, python, java, tcl, C, or C++. It doesn't matter because GCC is its own process just like every other benchmark.

      What exactly are they testing? The whole distro?

      Yes.

      The kernel?

      Yes again, since that is a part of the distro

      If they're testing the released kernel, then they should run static binaries that *test* the above, comparing kernel differences.

      No, what wouldn't prove anything as most of the binaries with ea

    2. Re:More of a summary by chrb · · Score: 3, Insightful

      If you look closely you'll notice that (a) the laptop the benchmarks are run on effects in no way, the validity of the benchmark as long as they are run consistently on the same laptop and (b) some benchmarks like RAM bandwidth have theoretical limits that are not effected at all by the Operating System but in actual practice, is entirely limited by the operating system you are using.

      a) Well that depends on what you mean by validity of the benchmark. If you only run tests on a single laptop, then any statistically significant results you find apply only to that single laptop - not even that laptop model, but that specific laptop. Who knows, maybe this specific laptop has some faulty memory or hard disk? Maybe the PSU is under-powering the system leading to slow down? The point is that without wider testing, you just don't know. To draw general results, you need randomised testing across different hardware platforms.

      b) You should see very little variation between operating systems when hardware is the limiting factor. "RAM bandwidth" is certainly not "entirely limited by the operating system you are using".

      All of the benchmarks were testing the hardware and should have showed variation.

      I believe you were missing the OP's point: when hardware is the limiting factor in a test, then there should be very little variance in the test result. If you are seeing a lot of variance, then you need to quantify why, because it is unexpected.

      Wrong. You isolate it down to one independent variable, its called the scientific process. And there was only one independent variable involved, the distro. Everything else is dependant on that variable.

      You then need to go and find out why you're seeing the results you see. Scientists also constantly question their own test methodology - you need to verify that the results you observe are indeed caused by significant differences between the systems under test, or by the test setup itself. And you also need randomised tests, otherwise your results can't be generalised. Oh, and you don't need to isolate it to one variable - see Factorial experiments.

  4. Summary by GauteL · · Score: 5, Insightful

    ... for those that can't be bothered to read this lengthy yet information sparse piece.

    1. MacOS X is faster in graphics intensive benchmarks.
    2. The other benchmarks are fairly even with Ubuntu coming out on top more often than OS X (one notable exception is SQLite).

    This is hardly anything new. OS X has a well optimised graphics system with good drivers for the intel chips (which up until now was used in both Macbooks and Mac Minis).

    Also SQLite is AFAIK integral to many features of OS X, and for this reason it makes sense for Apple to have optimised for it.

    Overall the benchmarks suggests that Linux (not just Ubuntu) needs some work on the graphics system and the Intel drivers. What a shock.

    1. Re:Summary by Gilmoure · · Score: 3, Informative

      Firefox 3 doesn't work in networked OSX environments...

      Firefox 3 is the default browser on Macs at my work (OS X 10.4 and 10.5) and we have about 1500 networked. How is Firefox 3 supposed to not be working?

      --
      I drank what? -- Socrates
    2. Re:Summary by Chaos+Incarnate · · Score: 4, Informative

      Firefox 3 works fine if profiles are on the local machine under OS X, but has issues if they're on a remote machine.

      --
      Benford's Corollary to Clarke's Law: "Any technology distinguishable from magic is insufficiently advanced."
  5. it's not simply the OS, it's the distro by slashnot007 · · Score: 4, Insightful

    For things like compilations, there's a bunch of file opens, caching the compiler and loader, gobs of Mallocs, and so forth that probably intersect the OS. Then there's the driver and video layer tests that look at frames per second. Leopard had 2 to 4 times faster frames per second. then there's the supporting distro services. Tests of My SQL were 4 times faster on the mac. And then there's things like the optimzation of VMs like JAVA where again Leopard excels. THese are clearly optimization problem and can be improved. the purpose of comparing it against a mac is not simply to say "oh yeah mac is faster than unbuntu", but rather to give a bench that shows how much room for optimization ubuntu has. Conclusion is that in almost every aspect Ubuntu is severely unotimized. Since older Linux seemed to be more optimized it suggests that feature bloat is probably either screwing up the design of linux or no one is paying attention to optimizing those features.

  6. SQLite inserts? by zoid.com · · Score: 3, Interesting

    What's up with the SQLite inserts? Is EXT3 really that bad? I would be interested in seeing PostgreSQL benchmarks.

    1. Re:SQLite inserts? by coolsnowmen · · Score: 3, Informative

      http://developer.apple.com/documentation/Darwin/Reference/ManPages/man2/fsync.2.html

      You are right, and that is not what I expected.

      Note that while fsync() will flush all data from the host to the drive (i.e. the "permanent storage
                device"), the drive itself may not physically write the data to the platters for quite some time and it
                may be written in an out-of-order sequence.

                Specifically, if the drive loses power or the OS crashes, the application may find that only some or
                none of their data was written. The disk drive may also re-order the data so that later writes may be
                present, while earlier writes are not.

                This is not a theoretical edge case. This scenario is easily reproduced with real world workloads and
                drive power failures.

                For applications that require tighter guarantees about the integrity of their data, Mac OS X provides
                the F_FULLFSYNC fcntl.

  7. Give credit where credit is due by Anonymous Coward · · Score: 5, Insightful

    Ubunu isn't getting slower, Mac OSX is getting faster.

    Do any of you recall Mac OSX 10.0?

    The day I installed Apple's first "modern" OS, I thought X marked the spot of Apple's demise.

    Apple has done an admirable job bringing MacOS into the 21st century, and their future looks promising.

  8. Apple's Moving Aggressively On Performance by glennrrr · · Score: 5, Insightful

    Next year, we will be seeing how much the extreme emphasis Apple is placing on performance will affect comparisons like these. Apple has figured out that since they can no longer hope to use differences in the CPU to differentiate themselves with generic Windows boxen, they will be using Microsoft's extreme backwards compatibility needs against them when it comes to fully using all the cores--whether they be in a CPU or a GPU--in a computer, and making full use of the 64-bit instruction set. GPGPU programming can give a huge performance boost to certain algorithms and the cleaner, more register rich, 64-bit instruction set is intrinsically faster in addition to allowing larger data sets.

    That's why they stopped selling non 64-bit capable computers a couple years ago, and why the new MacBooks have much improved integrated graphics. That's why they are moving their developers to include 64-bit compiles as part of newly shipped universal binaries. Next year is when all this latent potential gets switched on.

    Linux has the opportunity to do the same; perhaps more opportunity as it has less of a legacy binary issue, although Linux has to deal with a multitude of graphics chips, Apple only has to optimize for a handful.

    1. Re:Apple's Moving Aggressively On Performance by TheRaven64 · · Score: 5, Interesting

      Leopard has already improved performance in a lot of areas. In particular, mmap performance no longer sucks. I had an interesting experience where I wrote two back ends for some of my code, one with aio and one with mmap. On FreeBSD they were within 5% of each other. On OS X 10.4, the mmap one was an order of magnitude slower (and aio ran slower on the OS X machine, but that's not a very fair test - the OS X machine had a faster CPU but a slower disk). I extended this pathological case to a simple program which mmap'd a 2GB file and then touched one byte in every page in turn, 200 times. On 10.4, this completely killed the machine until it finished running, and didn't finish when I left it overnight. On 10.5, the machine stayed responsive (some slowdown, but not much), and finished in about an hour. Now that it's certified as fully complying with SUS'03, Darwin is a pretty nice OS, although too many parts of it (audio subsystem, for example) are closed for it to really be useful without OS X.

      --
      I am TheRaven on Soylent News
    2. Re:Apple's Moving Aggressively On Performance by scorp1us · · Score: 5, Interesting

      So many people are going to call BS on this, but...

      I was in the mac store the other day, and I swear I could tell the difference between the new Mac books with the NVIDIA chips and the ones without. From just looking at the scaling performance of the doc as you mouse over it, it looked so much more solid.

      I tend to be very sensitive to visual artifacts. I hated my MythTV box because of the tearing (memory bus issue) and blocking on Comcast (so glad I now have FIOS, which still blocks, but only for static or oceanscapes).

      Things like a dock where it feels "solid" (better servicing of repaints) just give a better impression of stability and performance, even if its just a simple scale operation. Having no flicker in position or delay in rendering make an impression on people who may not even be aware of what they are seeing.

      --
      Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
  9. Re:Ubuntu -- Obama Linux Distro by andrewd18 · · Score: 4, Insightful

    It came from nothing to something in a very short period of time.

    I wouldn't call Debian "nothing".

  10. Re:Ubuntu if you want to by ShieldW0lf · · Score: 5, Insightful

    Mac OS X doesn't have to accommodate variances in the hardware it is running on in the same way that Linux or Windows has to do. Therefore, it can exploit the hardware better. It's the same principle that applied to game developers targeting the XBox rather than a standard PC. Standard PCs might be more powerful, but the XBox is a non-moving target, so you don't need to write to the lowest common denominator, and can exploit the particular strengths of the hardware better. So, it's unreasonable to expect an OS that is written to work on multiple platforms to compete in this fashion.

    --
    -1 Uncomfortable Truth
  11. Re:Why is Fruit of the Loom so popular? by Dystopian+Rebel · · Score: 4, Funny

    Distros are like underwear

    Gentoo has the nastiest skid marks.

    --
    Rich And Stupid is not so bad as Working For Rich And Stupid.
  12. Re:Ubuntu -- Obama Linux Distro by tlacuache · · Score: 5, Funny

    Also, both are brown.

  13. Ubuntu fast enough for me by Eravnrekaree · · Score: 4, Interesting

    I have not noticed performance problems from Ubuntu. Sometimes I think these small differences are pretty much unnoticeable to the common user. I would say that while Linux always seems fast and snappy to me, its Windows which has a truly noticeable sluggish feel.

    I certainly do not think it is a good trade off in an OS to sacrifice features for an increase in speed which really is not noticeable. In most cases this is not necessary as many parts of a system can be made optional. The schedular and some core kernel systems effect the speed of the whole system, but most other components are optional, like X, like drivers, like Gnome, and so on.

    Which also is the nice thing about X: the designers of X decided not to try to build in a bunch of heavy user interface junk into the X server, ironically which many people criticise. Excluding memory leaks in some drivers not related to X itself, the X protocol and server system is actually very efficient by todays standards and does not use much memory. Most memory usage is in caching and in bad drivers full of crappy code. Therefore you can run our own window manager without carrying a bunch of stuff you wont use. But the eye candy is there if you want it. People should choose how many features and memory or how little they wish to use.

  14. Re:Ubuntu if you want to by ShieldW0lf · · Score: 5, Informative

    Mac OS X doesn't have to accommodate variances in the hardware it is running on in the same way that Linux or Windows has to do.

    It doesn't, huh? You mean like three generations of PowerPC CPUs, a second CPU architecture (x86), all the different flavors of HDDs, DVD drives, video cards, and other installable and peripheral devices you can add third-party, and then just about every bit of hardware Apple has come out with since the G3 processor debuted?

    Therefore, it can exploit the hardware better. It's the same principle that applied to game developers targeting the XBox rather than a standard PC. Standard PCs might be more powerful, but the XBox is a non-moving target, so you don't need to write to the lowest common denominator, and can exploit the particular strengths of the hardware better.

    Okay. The XBox uses a motherboard. Apple has several models using a variety of motherboard and hardware dating back to who-knows-when that it has to account for. You're comparing a console's static array of hardware to an entire production line. That's hardly the same thing.


    Ok, you've expressed how much more variance there is in the Apple product line compared to the XBox. Now, just for the sake of completeness, why don't you express how much more variance in the supported hardware for Ubuntu compared to Apple.

    --
    -1 Uncomfortable Truth
  15. It's not the distro either, it's the compiler. by ciroknight · · Score: 5, Interesting

    Older Linuxes are built on GCC 3.x or GCC 4.1.x. Since 4.2.x, GCC has produced absolute garbage code when the Gentoo flags are not enabled.

    Since most distros don't ship with --funroll-loops -O19 --ZOMG-MAKE-CODE-FAST, almost everyone has experienced a huge code speed drop. Meanwhile, Apple, knowing that all of their x86 machines support SSE2 or better has no qualms doing said incantations and benefiting from the speedups in autovectorization and other areas where the GCC hackers and Apple have been spending time.

    This leads us to the conclusion that a) Older Linuxes were better optimized (by the compiler, not the coder), b) Newer Linuxes are able to benefit but... c) Newer Linuxes are not benefiting because of their one-size-fits-all nature.

    --
    "Victory means exit strategy, and it's important for the President to explain to us what the exit strategy is." G.W.Bush
  16. Re:Ubuntu if you want to by Stormx2 · · Score: 4, Interesting

    It used to, but doesn't any more. Lack of demand.

    And incase you're saying that PPC support is more important than the tens of thousands of devices linux supports, think again. I've never had a hardware setup ubuntu hasn't worked with. It can be tricky at times, but my digital camera works straight away, my phone does, my MP3 player, web camera, usb headset, usb/wireless mouse, etc etc etc. The hardware support in linux absolutely vast, and support for non-peripheral hardware is going to be pretty tight.

    So yeah, linux supporting everything does mean lowest-common-denominator development, as the grandparent said.

  17. Re:Ubuntu if you want to by ShieldW0lf · · Score: 3, Informative

    It used to, but doesn't any more. Lack of demand. And incase you're saying that PPC support is more important than the tens of thousands of devices linux supports, think again. I've never had a hardware setup ubuntu hasn't worked with. It can be tricky at times, but my digital camera works straight away, my phone does, my MP3 player, web camera, usb headset, usb/wireless mouse, etc etc etc. The hardware support in linux absolutely vast, and support for non-peripheral hardware is going to be pretty tight. So yeah, linux supporting everything does mean lowest-common-denominator development, as the grandparent said.

    It's worth mentioning that G3 support was dropped in Leopard, and that PowerPC support was dropped entirely from the developer version of Snow Leopard.

    --
    -1 Uncomfortable Truth
  18. Re:7-zip benchmark? WTF?? by Hatta · · Score: 3, Informative

    7z is actually pretty impressive. I recently converted a bunch of ROMs from 7z to gz for use with mednafen. I don't remember exactly, but the gzipped roms were close to 50% bigger than the 7z. FWIW, 7z uses both LZMA and bzip2 compression as appropriate. The only real reason not to use it, is that you can't expect people to have 7z installed just yet. But bzip2 overcame that obstacle, and 7z should too. This comparison should interest you.

    It's also worth noting that the gui windows 7-zip program is the best archive manager I've ever used on that platform. Easy to use, powerful (great context menu entries), supports most existing formats, and is entirely free. There's no reason 7z shouldn't take over the world. But I guess that makes me a loudmouth advocate, huh.

    --
    Give me Classic Slashdot or give me death!
  19. Re:Ubuntu if you want to by nxtw · · Score: 4, Informative

    Mac OS X doesn't have to accommodate variances in the hardware it is running on in the same way that Linux or Windows has to do. Therefore, it can exploit the hardware better. It's the same principle that applied to game developers targeting the XBox rather than a standard PC. Standard PCs might be more powerful, but the XBox is a non-moving target, so you don't need to write to the lowest common denominator, and can exploit the particular strengths of the hardware better. So, it's unreasonable to expect an OS that is written to work on multiple platforms to compete in this fashion.

    The Xbox was always sold with the same chipset. There was no need to have interchangable drivers.

    OS X Intel has been sold (so far, and at least) on systems with Intel 945GM, 945PM, GM965, GMS965, PM965, 5000X, 5400 and nVidia 9400M system chipsets. That's two (plus one low-power variation) Intel mobile chipsets with Intel integrated graphics, two Intel mobile chipsets with PCI Express x16 graphics, two Intel server/workstation chipsets, and one nVidia integrated mobile chipset. It supports two generations of Intel GPUs (GMA 950 and X3100), at least three generations of GeForce GPUs (7, 8, and 9-series), and at least four generations of Radeon GPUs (X1k, HD2k, HD3k, and HD4k).
    There aren't a whole lot of hardware-specific assumptions or optimizations that can be made without making things only work on on that hardware... which is why hardware-specific code is in interchangeable drivers.

    Apple can't even assume that every Intel Mac has a 64-bit dual core CPU; the first Intel Macs had Core Solo or Core Duo CPUs. Apple can assume that every Intel CPU will have SSE2 and SSE3, though, so many floating-point operations are performed using SSE instructions instead of the x87 FPU. But software can be compiled to use SSE on any other operating system as well. (SSE3 is featured on nearly all Intel and CPUs.)

    They could write code/compiler optimizations that result in faster execution on some CPUs, but they're already supporting the P6 microarchitecture (the original Core Duo was close in design to the Pentium M) and the Core microarchitecture, and Nehalem will be here soon.