Firefox 3.5RC2 Performance In Windows Vs. Linux

pizzutz writes "Andy Lawrence has posted a Javascript speed comparison for the recently released Firefox 3.5RC2 between Linux (Ubuntu 9.04) and Windows(XP SP3) using the SunSpider benchmark test. Firefox 3.5 will include the new Tracemonkey Javascript engine. The Windows build edges out Linux by just under 15%, though the Linux build is still twice as fast as the current 3.0.11 version which ships with Jaunty."

Comments

Don't benchmark it on Ubuntu

[petrus4]

Ubuntu typically has everything but the kitchen sink running in the background; it's even worse than XP for frivolous defaults.

Get Slackware, or something else minimalistic, where you're likely to have a marginal amount of memory left after the operating system and residents are loaded in. ;)

Re:Don't benchmark it on Ubuntu

[larry bagina]

Firefox isn't slower because of ubuntu, it's slower because the microsoft's C compiler is better than gcc.

Re:Don't benchmark it on Ubuntu

[Freetardo Jones]

It has nothing to do with Ubuntu. Here are benchmarks from Firefox on Fedora: The issue is just as bad on Fedora: http://www.tuxradar.com/content/browser-benchmarks-2-even-wine-beats-linux-firefox. That's only from a few months

Re:Don't benchmark it on Ubuntu

[Anonymous Coward]

It's because it's an interesting post, while your previous one was flamebait.

Re:Don't benchmark it on Ubuntu

[K. S. Kyosuke]

Surprising? Because your post *did* smell of something burning, while the MSVC claim has at least some merit - MSVC really got better. I guess it had to, it would be a shame if MS threw a couple of megabucks on it with no results at all.

Re:Don't benchmark it on Ubuntu

[Locke2005]

I am SHOCKED and AMAZED that a compiler specifically implemented for x86 (with assistance from Intel) produces more efficient x86 machine code than a compiler that is based on a general purpose architecture with just a back-end code generator for x86. Next you'll be telling me that a Swiss army knife isn't as good for skinning animals as a Bowie knife and that an amphibious vehicle is neither as fast on land as a Ferrari nor as fast in the water as a cigarette boat!

Re:Don't benchmark it on Ubuntu

[mR.bRiGhTsId3]

You know all that stuff that's running in the background. Well, it's running in the background, which means that most of it should be swapped out and it shouldn't wake up very often to do stuff.

Re:Don't benchmark it on Ubuntu

[Anonymous Coward]

So how hard is this to test? Anyone want to run firefox win32 binaries on linux with WINE? Since the executable code will be faithfully executed by the CPU we can eliminate the compiler differences from the equation.

Somebody else do it. :) I'm busy.

Re:Don't benchmark it on Ubuntu

[Anonymous Coward]

Visual C is not "compiler specifically implemented for x86". It supports (and supported in the past) lots of architectures -- x86, x64, Itanium, Alpha, MIPS (and MIPS16), PowerPC, ARM (and Thumb), Hitachi SuperH, Infineon TriCore, several other embedded CPUs as well.

Of course x86/x64 are main targets, but my guess it is so for GCC as well :-)

Re:Don't benchmark it on Ubuntu

[R2.0]

"Next you'll be telling me that a Swiss army knife isn't as good for skinning animals as a Bowie knife "

Actually, in most cases one would be better off with a Swiss Army type knife than a Bowie knife. Even on a large animal, parts of the skinning process are delicate and take a light touch, and the smaller blades are better at that. And then get to smaller animals - may as well just beat that rabbit with a rock as use a Ka-Bar. In addition, the saw blade of a SA knife is handy for cutting cartilage, etc.

A Bowie knife is a fighting knife, made more for killing and rough work. It will do other things, but not that well. That being said, when the meteor hits because DOD "blocked" the satellite information, and I have to choose 1 knife, it would be the Ka-Bar or similar. Why? Because I want to stay alive to have the opportunity to skin rabbits with my poor tool, and whipping out a Swiss Army knife will only make a zombie laugh.

Re:Don't benchmark it on Ubuntu

[rbanffy]

They also have an easier job. MSVC doesn't need to address as many architectures as GCC does. IIRC, there is no MSVC for s/390

Re:Don't benchmark it on Ubuntu

[Anonymous Coward]

I am shocked and amazed that the company which destroyed all competing commercial compilers by hiring their best developers away now has a really good compiler.

Re:Don't benchmark it on Ubuntu

[selven]

Just because you can justify Linux's inferiority in this one area doesn't mean that we don't need to improve it.

Re:Don't benchmark it on Ubuntu

[msclrhd]

It will depend on what compiler settings were used to build the glibc library, what architecture it was tuned for, whether or not the glibc library is targeted to the specific version of the kernel you are using. It also depends on what options were used to build the companion libraries, as well.

Also, the last time this was discussed on slashdot, I seem to remember that it was Profile Guided Optimisation (PGO) that helped FF on Windows. This would allow the compiler to better structure the machine code to keep as much of the program running in the CPU cache as possible (by pre-loading branches in conditional statements that are executed more frequently).

Re:Don't benchmark it on Ubuntu

[Skye16]

Wait, is it a zombie meteor?

I'm confused.

Re:Don't benchmark it on Ubuntu

[True Grit]

improving performance on its most popular platforms if they want to remain relevant...

You don't understand, GCC's portability is the reason for its very existence. Thats not optional. GCC has got more than 20 backends (architectures) and 8-10 frontends (languages) for goodness sake, no freakin' wonder its a little slower than MSVC/ICC, those latter guys only have to worry about 2 backends (x86/x86_64) and 2 frontends (C/C++).

GCC's wide portability is what currently guaranttees its relevancy. The minute GCC starts chasing clock cycles on the x86 platform (at the expense of the others), that will be its curtain-call...

But why?

[Anonymous Coward]

Is there any explanation as to why there is the difference?

Re:But why?

[Freetardo Jones]

The Windows version is compiled with PGO (profile guided optimization) while Linux versions aren't.

Re:But why?

[moon3]

The problem is that GCC is pretty much the only compiler on Linux used these days and while the code is very nice C++ compilers on Windows produce a bit better code still.

But when I mention Watcom C++ or other aspiring open source compiler here, a compiler that could possibly interest Linux community and spawn some competition for GCC then I get modded down often by people citing GCC is good enough for everybody and everything.

Re:But why?

[larry bagina]

also worth mentioning is llvm. gcc-llvm has an llvm backend doing code generation (which sometimes beats standard gcc, sometimes doesn't). There's also a non-gcc c/objective c/c++ compiler, clang, in development, though it may be a couple years before c++ support is complete.

Re:But why?

[guyniraxn]

What if we threw SwiftFox in to the comparison?

Re:But why?

[Rockoon]

Watcom isnt even top-3 anymore.

The king of C compilers is now ICC, with both MSVC and GCC way behind.

Re:But why?

[Prune]

One can always use the Intel compiler on Linux. The performance of compiled code is comparable to that the Intel compiler produces with its Windows version.

Re:But why?

[pz]

The problem is that GCC is pretty much the only compiler on Linux used these days and while the code is very nice C++ compilers on Windows produce a bit better code still.

In my experience, MS's VC++ produces not just a bit better code than gcc, but whole hocking meeses better code. VC++ is a damned good compiler, no matter what one might think about the company that generated it, while gcc is a merely decent one, no matter how much one might want to promote FOSS.

Re:But why?

[David Gerard]

The OpenWatcom license is OSI-approved, but I don't see why. It failed DFSG, and I asked the FSF about it and they can't make head nor tail of it either.

Re:But why?

[electrosoccertux]

It feels just as slow. It's not just Gnome, it's slow in KDE and XFCE, too.
It is currently faster to run Firefox.exe under Wine than it is to run it native in Linux. (Yes I have tried this, the difference is night and day; it's just as fast in Wine as it is in windows).

Re:But why?

[asa]

Many of the rendering engine parameters can be set in Firefox's preferences. Firefox prefs are stored as a js file. Swiftfox makes a different set of trade-offs in these parameters than the Mozilla engineers. You're free to take your pick, but I'm goin' with the Mozilla engineers.

Re:But why?

[amdpox]

Wine runs the code almost raw - the only difference to running on windows is that Win32 API calls are handled by Wine rather than Windows. All the computation that Firefox itself does is run exactly as it is on windows, ad thus the optimisations MSVC makes to the machine code are still there when it's run under Wine.

Re:So what shall one use now?

[mister_playboy]

This is nothing new. Running Windows FF in WINE is faster than using Linux native FF.

Re:There! You have it!

[Anonymous Coward]

This proves that, um, Windows,er, Linux is....um...what the fuck does this prove again?

And why the fuck should I care if there's a 15% difference in performance of Firefox between those two OSes? I use my particular OS for reason that have nothing to do with how well Firefox runs on it.

That 15% could very well be measured in hours when the Slashtard coders get through with their Web 2.0 abominization of Slashdot.

Re:My problem with Firefox is this

[pablomme]

But on Linux, it is inherently ugly. The beast looks ancient and the fonts and dialogs make matters worse.

Widgets and dialogs, ok, that's your aesthetic preference. But fonts? After a couple of years using Ubuntu I hate how Windows fonts look pixelated even with Cleartype on. Freetype is much better at its job than Cleartype. If only because of that, I prefer the looks of Firefox on linux than on Windows.

maybe linux carries some of this blame

[asa]

Putting the blame all on Firefox when there's no doubt a certain amount of performance penalty that comes with a Linux's less good compiler is just lame. How about telling the linux tool makers to build tools that output faster and smaller code instead of demanding that app developers solve those problems? Finally, what "linux" build was this? Did it use profile guided optimization and other performance features of Mozilla's official Windows build system? If not, you're comparing apples to oranges.

Re:maybe linux carries some of this blame

[loufoque]

This is a myth.
I have barely ever noticed a performance increase when comparing code compiled with equivalent options on GCC, ICC and MSVC.

Quite the contrary, GCC is faster more often than you'd think.

Re:maybe linux carries some of this blame

[RebelWebmaster]

See bug 418866 for some guidance.

Re:maybe linux carries some of this blame

[multipartmixed]

Why don't you compile Tracemonkey with MSVC, GCC, and ICC, then run the sunspider benchmark. Make sure you have PGO turned on for the MSVC build.

Then you can go eat your hat.

Re:maybe linux carries some of this blame

[asa]

Got links? I'd love to see some support for this. Until I get those, I'm going to take the large number of anecdotal accounts from the people who actually compile popular apps like Firefox on Windows and Linux as having a bit more weight than your lone voice without any supporting evidence or even specif anecdotes.

Re:maybe linux carries some of this blame

[bloodhawk]

It is not a Myth, GCC is a good compiler, MSVC and ICC are fantastic compilers. This doesn't make GCC bad by any means, but it is gonna be a very rare occurance that GCC produces faster code, more work and effort has been spent on the other 2 to make them that much better.

Re:maybe linux carries some of this blame

[bsmedberg]

Mozilla does comparative performance testing for the best GCC compiler flags constantly. There are several reasons why our Linux builds are slower than Windows:

1. The Windows ABI is cheaper: every relocated symbol in Linux is resolved at runtime by loading the PIC register and going a GOT lookup. Windows avoids PIC code by loading the code at a "known" address and relocating it at startup only if it conflicts with another DLL.
2. Mozilla code runs fastest when 99% of it is compiled for space savings, not "speed". Because of the sheer amount of code involved in a web browser, most of the code will be "cold". Tests have shown that at least on x86, processor caches perform much better if we compile 99% of our code optimizing for codesize and not raw execution time: this is very different than most compiler benchmarks. The MSVC profile-guided optimization system allows us to optimize that important 1% at very high optimization levels; the GCC profile-guided optimization system only really works within the confines of a particular optimization level such as -Os or -O3. In many cases using PGO with Linux produced much *worse* code!
3. The GCC register allocator sucks, at least on register-starved x86: we've examined many cases where GCC does loads and saves that are entirely unnecessary, thus causing slowdowns.

Believe me, we'd really love to make Linux perform as well as Windows! We spent a lot of time in Firefox 3 with libxul reducing startup time by making symbols hidden and reducing the number of runtime relocations...

Re:Proving yet again

[asa]

Actually, it probably does say something about the superiority of the Windows compiler and potentially other Windows tools.

Re:So what shall one use now?

[asa]

A very large chunk of Firefox's developers have Linux as their primary platform. Linux Firefox absolutely doesn't get crap treatment from Firefox developers. You're obviously not familiar with the comparative qualities of the compilers on different platforms or you would asking "why do the Linux compilers get beat so badly by the Windows compilers."

Re:My problem with Firefox is this

[WarwickRyan]

>Ask Apple or even Windows folks.

You have seen Safari, haven't you?

It puts the 'f' in fugly.

Re:My problem with Firefox is this

[jdgeorge]

Firefox on Windows looks great/awesome/beautiful....name it. But on Linux, it is inherently ugly. The beast looks ancient and the fonts and dialogs make matters worse.

Folks, I am not trolling so have a look for yourselves and compare....

I'm running Windows XP and Ubuntu 9/04 side by side on similar laptops. Just to test, I looked at the main pages for Slashdot, Wikipedia (English), and Amazon, side-by-side.

My eyeball result of looking for differences between pages rendered with Firefox on Ubuntu 9.04 vs Windows XP:

• Slashdot (slashdot.org): indistinguishable
• Wikipedia (en.wikipedia.org/wiki/Main_Page): indistinguishable
• Amazon (www.amazon.com): Bold fonts in the "Shop All Departments" navigation menu appear too big on Ubuntu; they don't quite seem to fit properly.

Other than the issue for Amazon, the pages rendered look identical to me. The fonts for the menus look identical. I still disagree with the choice the mozilla team made to have the preferences/options menus with different titles in different locations for Linux versus Windows, but other than that, the UI seems consistent to me. The default GNOME theme for Firefox isn't as pretty as the new Firefox theme on Windows, but that's a minor aesthetic thing, and it's not ugly, it just isn't pretty.

Re:My problem with Firefox is this

[TodLiebeck]

Firefox on Windows looks great/awesome/beautiful....name it. But on Linux, it is inherently ugly. The beast looks ancient and the fonts and dialogs make matters worse.

In Ubuntu 9.04 here, and I personally think the stock DejaVu fonts on Linux look quite nice. Actually prefer the traditional toolbar on Linux with Tango icons (tango.freedesktop.org) rather than the "enlarged back button" version found on Windows and OSX.

The only problem I see is the topic of this thread, i.e., performance. It's slow enough to feel slow, and the fact that most Linux distros run so well on old hardware makes the problem worse.

The bigger problem for the "Linux browsing experience" still seems to be Flash. Visiting a Flash-heavy site (like the horrible items produced by any given automaker) is a painful experience...it's bad enough that I'll typically crack open the MacBook. I find Flash sites consume an order of magnitude more CPU running natively in a Linux browser than they do running in a Windows XP VirtualBox instance hosted by the same Linux OS.

AdBlockPlus and FlashBlock are the only things that enable me to continue to use this computer for web browsing. It's somewhat of a sad state of affairs, given that it's more than quick enough to run multiple VirtualBox instances, Eclipse instances, and a GIMP instance with dozens of files open at the same time. But give it one web page with a few Flash advertisements, and you'll think you're on a Pentium 60.

Where's the proof that GCC is solely to blame?

[pembo13]

I keep hearing people saying that it's all GCC's fault, but I have seen no real proof of that. Nor why a profit making company such as Mozilla can't throw devs at GCC to fix the underlying problem.

Re:Where's the proof that GCC is solely to blame?

[characterZer0]

Why can't they just use Intel's compiler?

Re:Where's the proof that GCC is solely to blame?

[hattig]

Compilers aren't a simple problem that you can 'throw devs at'.

Maybe the Intel compiler could be used, but it breaks on anything non-trivial.

Or we can suck up the 15% performance degradation, especially if it is due to profiled optimisations on Windows, and just be happy that it is twice as fast in terms of Javascript, that might make Slashdot usable on a netbook again. It's certainly going to be faster than IE8 on Windows, and that is what most Windows users will be using in the end.

I only hope that Canonical make it available for 8.10, because we can't all upgrade to 9.04 because some companies (looks at VIA) take 6 months to get their drivers updated.

Re:Where's the proof that GCC is solely to blame?

[mR.bRiGhTsId3]

Its my understanding that a bunch of the firefox core is written in C++. ICC uses a different name mangling scheme that GCC, so you can't link C++ code compiled with ICC with code compiled by GCC.

Re:Where's the proof that GCC is solely to blame?

[Freetardo Jones]

That stopped being true back in 2007 when they released version 10.0.

Re:Where's the proof that GCC is solely to blame?

[mR.bRiGhTsId3]

Well, I guess the only reason to not compile with ICC is that "0h N0es! iNtel is t3h unfr33!" because every piece of literature I'v ever read suggests ICC produces slightly larger binaries which are greatly faster when compared to GCCs output. In fact, for this reason I seem to remember a whole bunch of guides from the olden days on Gentoo for how to install ICC.
And thus, the lesson we have learned is that when you have the engineers who designed the architecture on hand, you can write a kickass compiler.

Re:Where's the proof that GCC is solely to blame?

[asa]

Why can't they just use Intel's compiler?

Who is "they"?

Builds created by Mozilla get used but very few people compared to builds made by Ubuntu and other Linux distros. Perhaps you should contact them and as them why they're not using the Intel compiler.

Re:makes sense

[ianare]

... and anonymous users are dyslexic (tub onyl midly).

Re:There! You have it!

[Colonel Korn]

This proves that, um, Windows,er, Linux is....um...what the fuck does this prove again?

And why the fuck should I care if there's a 15% difference in performance of Firefox between those two OSes? I use my particular OS for reason that have nothing to do with how well Firefox runs on it.

That 15% could very well be measured in hours when the Slashtard coders get through with their Web 2.0 abominization of Slashdot.

People have been complaining or quite some time about poor performance on slashdot. What is it that shows this poor performance? I don't recall doing anything that isn't instantaneous here.

What about other common cross platform software ?

[godrik]

Are there around some tests about other open source software that could help us understand the problem ? We can find some on open office : http://www.oooninja.com/2009/03/multiplatform-benchmark-30.html Or Tomcat : http://mediakey.dk/~cc/tomcat-performance-linux-faster-than-windows/ But that does not seem to gie a clear understanding of what's happening.

Re:I'm not sure about this

[Teun]

Hmm, Adept says: Firefox Version 3.5~b4~hg20090330r24021+nobinonly-Oubuntu1 Package is maintained by Ubuntu Mozilla Team

I run it and it works.

Re:There! You have it!

[jalefkowit]

The lag when posting a comment between hitting the "Preview" button and actually seeing the preview is downright painful.

Re:Does gcc "do" Windows?

[Mad Merlin]

Can gcc compile Firefox for Windows, so that we can more confidently apportion blame?

Yes.

Re:My problem with Firefox is this

[FooBarWidget]

Unlikely to be X. I can play normal videos just fine at full speed. It's only Flash that eats CPU and is slow.

Re:A Good Test would be Slashdot

[rts008]

It's /. and it's sorry assed javascript, not firefox.
I went to preferences and chose 'classic view' to end this assclown behavior. That also ended the funny bars showing up in the middle of replies, the friend/foe icons plastered on top of comments, and other asshattery that was going on here also.

Re:My problem with Firefox is this

[AmigaHeretic]

>>I hate how Windows fonts look pixelated even with Cleartype on

There not pixelated, they have holes.

http://www.ecofont.eu/ecofont_en.html
It's save electricity. See, MS is justing trying to be a "green" company.

GCC, ICC, MSVC

[JambisJubilee]

It is not a myth. ICC kicks the crap out of GCC. I didn't believe it until I had access to a computing cluster (Intel processors) with ICC installed. My ANSI C code runs about twice as fast using ICC than with GCC. Would you really expect anything different?

As always, YMMV, but I suggest that anyone who doubts this to download Intel's compiler (it's free as in beer) and try it out.

It's not open source, which does suck. But it does consistently produce faster code.

Re:My problem with Firefox is this

[zuperduperman]

The horrible fonts were what drove me away from Ubuntu after I installed it recently, hoping to use it as my primary desktop. I'm sure you've managed to fix up your fonts somehow, but let me tell you, a default ubuntu install (from the 8.x series, haven't tried more recent) produced such an eyeball searing ugliness in FireFox that it almost single handedly convinced me that Ubuntu wasn't ready yet (for me). The fact that a few searches with Google reveal hundreds of various ways to improve the fonts actually makes it even worse.

Since you'll undoubtably deny this having not witnessed it yourself, just search on google and see the thousands of perplexed newbies being driven away from linux by the fonts you think are so beautiful:

http://www.google.com/search?q=ubuntu+firefox+ugly+font

Re:...but does Adobe Flash still hang up?

[asa]

The Javascript speed is not much of a factor. The one truly annoying thing with Firefox is the gawdawful Adobe Flash plug-in that hangs up at random, causing the whole browser to come to a screeching halt.

So why not get Flashblock or remove the Flash plugin?

Re:So what shall one use now?

[BZ]

> So why isn't the Moz Foundation investing in a better compiler?

This is a good question; I believe it's largely a matter of not having found anyone to pay to do the work, but I'm not privy to details on this stuff...

> what does this say about the performance of any app running under Linux compared to the
> Windows version?

That if performance is gated on computation and codesize, and the code identical, it'll tend to be faster on Windows....

Thing is, for most apps performance is gated on I/O or display (if gated at all) more than on computation and codesize.