Firefox Notably Improved In Tom's Hardware's Latest Browser Showdown

Billly Gates writes "Tom's Hardware did another benchmark showdown, since several releases of both Firefox and Chrome came out since their last one. Did Mozilla clean up its act and listen to its users? The test results are listed here. Firefox 13.01 uses the least amount of RAM with 40 tabs opened, while Chrome uses the highest (surprisingly). Overall, Firefox scored medium for memory efficiency, which measures RAM released after tabs are closed. Also surprising: IE 9 is still king of the lowest RAM usage for just one tab. Bear in mind that these tests were benchmarked in Windows 7. Windows XP and Linux users will have different results, due to differences in memory management. It is too bad IE 10, which is almost finished, wasn't available to benchmark." Safari and Opera are also along for the fight.

Comments

Why IE9 did well

[Stonent1]

Because the browser is part of the OS, the RAM is already in use as part of the windows explorer.

Re:Why IE9 did well

[Bill Dimm]

It is interesting that IE uses 1276 MB for 40 tabs, so 1276/40 = 31.9 MB/tab, while with only one tab open it takes 31 MB, i.e. there seems to be no memory overhead for the software itself. For comparison, Firefox uses 794 MB for 40 tabs, so 19.85 MB/tab, and 61 MB when there is only one tab open, so an overhead 41 MB for the software itself.

Re:Why IE9 did well

[icebike]

while with only one tab open it takes 31 MB, i.e. there seems to be no memory overhead for the software itself

Which is exactly what the GP was saying: that any browser written into a specific OS can use the facilities of the OS itself to mask its true size.
Its sort of like measuring the memory impact of the Chrome browser running on a Chrome-OS tablet.

But disregarding memory utilization by the browser itself, your numbers are pretty interesting all by themselves.

What the heck is IE doing with that extra 11meg per tab when there are multiple tabs open? The web page itself is only the size that it is, and presumably the tests all loaded the same pages in the same order. 11meg is a pretty good guess for the average size of pages. Its almost as if IE build a page in the background and handing the whole thing to the render engine while keeping a copy as backup.

I wonder if the tests were run in a memory constrained machine? The idea that unused ram is wasted ram might lead some of these developer to use what is available more or less freely depending on available ram.

Re:Why IE9 did well

[dgatwood]

IE is a better browser than it used to be, but it started out so far behind that they're going to be catching up for a while yet.

For example, their DOM selection range support is still way behind, as is their memory management. (It is absolutely unacceptable to tell JavaScript coders that they should not add methods to an element or they'll cause memory leaks. I mean, really!?!)

And IE still has fascinatingly severe bugs. For example, create a trivial HTML page that uses Javascript to set the src property of an existing iframe to the same URL as the loading page, and none of the JavaScript scripts on the second page ever run. (IE 9) The only robust workaround I've found is to replace the iframe with a new element. That workaround, in turn, when combined with IE's hack where they dispose of the DOM tree for an iframe's contents when the iframe is detached even if parts of it are still in use by JavaScript code (their hack "fix" for the aforementioned memory leaks) led to hours of extra debugging for me. (Wait, how can contentDocument.body legally be null?)

And it is fairly easy to wedge things using its development pane. And its contentEditable support is seriously subpar. (You can't easily select content that spans a div boundary, for example.) And it caches XHR requests when other browsers don't, which caused me lots of headaches (though admittedly I should have been sending appropriate headers to begin with).

Even in its current, much-improved state, IE is still a plague. If they keep up this level of improvement, it might be a viable browser for the website I'm developing in 5 years. As it is, I'm going to support Firefox, Safari, and Chrome, but I have no plans to support IE at this time. It just isn't feasible to work around all the bugs—even in IE 9. We'll see about IE 10, but I'm not holding my breath.

more information on firefox

[cyfer2000]

there is a website called are we slim yet tracking the memory usage of firefox.

Re:more information on firefox

[AmiMoJo]

This obsession with memory use is wrecking Firefox. Chrome is much, much faster and I prefer to trade memory for performance.

Firefox does stupid things like delaying image decoding until the image is on screen, making the whole browser stutter like crazy. I turn that off because the reason my computer has lots of ram is to avoid that. Can't they even detect when you have lots of free RAM and make use of it? Of course not, that would make FF look bad in pointless benchmarks.

Re:more information on firefox

[Anonymous Coward]

This. Mod up.

The stuttering behaviour became extra prevalent starting with Firefox 13 (for me anyway). Sites like Amazon would cause the browser to stutter while trying to scroll through content before images were loaded. A complete uninstall + clean + reinstall of FF13 didn't help. Rolling back to FF12 worked fine. Disk = Intel 510-series SSD, under AHCI on Windows XP SP3. Moving my FF cache to a MHDD did not improve things either, so it's not disk I/O which is was the cause.

I then tried Chrome. Holy *shit* that thing is fast, and I don't just mean loading pages -- I'm talking about interactivity/responsiveness within the browser (for keystrokes, mouse clicks, etc.) too. What I didn't like about Chrome: 1. tab-only (I do not like tabs, and have tried many times to use them but just cannot do it), 2. its lets-fork-a-new-process model (which is part of its original design from the very beginning) makes it very hard to tolerate when using Task Manager (yes I know Chrome has its own internal "manager" that can let you kill off a specific tab/page), 3. quite possibly the worst Configuration page/setup I've ever seen, with lots and lots of adjustments missing, and finally 4. a non-user-friendly bookmark interface (lots of Chrome users complain about this and state hands down that Firefox does this a lot better. The biggest ding against Chrome was when they removed the Bookmark button/icon so now you have to go through 3-4 clicks to expand your bookmarks). Back to Firefox I went...

The memory bloat problem in FF is real, but a lot of it has to do with how people use their browser. Most of my colleagues do insane shit like load up 20 tabs on launch and leave those open at all times. Who the fuck uses a computer like that? Oh, wait... well, I certainly don't. Many of those pages my colleagues load use Flash, lots of Javascript, etc.. One even loads 6-7 tabs filled with stock-ticker-esque pages; yeah, those are going to be real CPU and memory friendly. You can even see evidence of this here on Slashdot; "Who in the name of satan has 40 tabs open? *checks tabs* Guilty as charged m'lord". Stop abusing your computer!

As I see it, the biggest complaints about memory usage seem to be coming from a demographic of people who aren't using their system in a resource-friendly manner. People leave their browsers open for days, sometimes a full week. I'm a system administrator -- when I'm done with something, I close it. Same goes with memory management: when you're done with something, free() it. Folks over the years have tried hard to argue with me about this point ("no, a good garbage collector.........") -- all bullshit. Every GC on the planet is shit. Free memory when you're done with it, and use threads (preferably on an OS where userland threads map more or less 1:1 to a kernel thread, and use a programming language where this thread model exists (e.g. not Ruby!)) so that allocations can be freed when the thread ends. This isn't rocket science; KISS principle all the way.

It would greatly benefit everyone if they learned be a little more conscientious of how to use a computer in a resource-friendly way. Otherwise we're doing the exact thing that can never be achieved: trying to solve social problems (usage behaviour) with technology. It never works.

Re:more information on firefox

[hackertourist]

As I see it, the biggest complaints about memory usage seem to be coming from a demographic of people who aren't using their system in a resource-friendly manner.

People should not need to do that. Quit trying to force me to adapt to the machine instead of the other way round. This isn't a Sinclair Spectrum, it should be able to handle 40 tabs without crashing or filling up all my RAM.

Performance improvements

[jaak]

Interestingly enough, the Tom's Hardware pages-per-article benchmark shows that Firefox can now handle an article spread over twice as many pages as before!

40 tabs?

[trancemission]

Who in the name of satan has 40 tabs open?!?

*checks tabs*

Guilty as charged m'lord..........

Your opinion may differ

[trifish]

What I care about in a browser is (in order of importance) security, compatibility, reliability, speed.

I stopped examining RAM usage of any software since the time I bought 16GB of RAM for practically no money.

Even before, when I had "only" 4GB of RAM, I had swap file turned off for years and I haven't seen a single "Insufficient RAM" error.

What the summary did not include

[trifish]

The summary elegantly avoided the most important metric - Page Load Time. Ok, so let's see how we're doing there:

IE9 - fastest
Safari - 2nd
Chrome - 3rd
Firefox - 4th
Opera - 5th

The page load time tests are the same eight pages in our startup time tests: Google, YouTube, Yahoo!, Amazon, Wikipedia, craigslist, eBay, and Wikipedia.

http://www.tomshardware.com/reviews/windows-7-chrome-20-firefox-13-opera-12,3228-6.html

Re:What the summary did not include

[Anonymous Coward]

Is there anything of value to comment on when the page load times are so close to each other on all the browsers excepting the last? The first 4 finishers range from 880ms to 947ms, which is less than a 10% differential across 8 websites. I doubt you'd notice any difference in your day-to-day real life browsing with respect to page load times.

Re:Isn't IE embedded into the OS where possible

[icebraining]

Your knowledge is outdated. Firefox has separated plugins like Flash into their own processes using plugin-container multiple versions ago. You can kill the process without bringing Firefox down, and in fact I've done just that many times.

Re:Too bad for others

[icebike]

No, he's saying that a screw up long ago prevents him from ever looking at it again.

Its the old "cut off your nose to spite your face method" of software selection. There is no redemption in some people's eyes.

Re:who gives a fuck

[NotBorg]

Why "enterprises" can't customize open source software is a bit of a mystery to me. Interns really are cheap these days (esp. by $$$ enterprise $$$ standards). Seriously enterprise customers want a browser specifically tailored to their needs for absolutely nothing. It's funny.

Auto updates can be turned off both at compile time option and as an installed option. It's never been easier to bring in a custom patch set and build software and yet they're still bitch'n. They don't even have to pay a fucking license fee but act as if they're paying customers. They act like they dished out thousands of dollars for support like they do for their Oracle database software or Microsoft servers. They'll pay MS and Oracle per processor/core for less customization but when it comes to Mozilla they expect $0.00 to get them everything.

Enterprise babies need to grow up and L2.

Re:who gives a fuck

[Surt]

firefox tools/options/advanced/update
But you may not be qualified to install firefox if you couldn't figure that out.

Re:Too bad for others

[Korin43]

For anyone who's actually interested, the Memshrink Blog is a fascinating account of how a team of developers have been reducing Firefox's memory usage. Interestingly, Firefox's memory usage has never been particularly bad (it just seems to be because web pages are so much more complicated), but addons have had horrible memory problems for a long time (and unfortunately, that's pretty hard to detect).

Still a dog on Linux

[Pausanias]

I love Firefox, but it's still about 50% faster on Windows 7 than on Linux. Chrome wins clearly on Linux. I agree that on Windows they're comparable.

Firefox throwing out the baby with the bath water

[hackertourist]

The recent improvements in FF memory usage have some severe drawbacks. FF will throw away images in background tabs, and reload them when you switch to the tab. But the reason I load new pages in background tabs is that I don't want to wait for the page to load and be rendered.

It gets worse when you quit FF, and then reopen the app and have it reload your last session. It will create the tabs, but it won't load the page until you activate the tab. Now this is something I'd be willing to tolerate for tabs that have been open for a few weeks. But not for tabs that I've created recently and/or activate frequently.

I also notice that FF memory usage steadily increases over a couple of days, while the number of tabs remains roughly constant.

In other words, they have reduced the memory footprint not by tackling whatever process is hoarding memory like Scrat stacking acorns in his giant hollow tree, but by throwing away items that use memory but are otherwise static (images).

Memory is not the Firefox problem

[aussersterne]

Like others here, I have many gigabytes of RAM, so I'm not too concerned with the memory usage issue. Firefox is also good on speed; that doesn't bug me.

But I stopped using Firefox back when I was last a Linux user (ca. 2009) and have continued to use alternatives (Chrome and Safari, most notably) on OS X because Firefox suffers from too many WTF? moments. Whatever you call them—bugs, the results of Firefox's architecture, I don't care—they make Firefox a non-starter for me. For example:

(1) When using Firefox on OS X all window updates sometimes suddenly stop. Nothing is clickable, nothing is scrollable. The way around this is to drag the window—even just one pixel. After that, refreshes will return. That's bad if you have data auto-refreshing on a Firefox window you're monitoring. And it's not an OS X bug because only Mozilla applications (Firefox, Thunderbird) display this issue, and have done for the last umpteen versions. (For the record, this happend both on my older Mac desktop and on my new unibody MBP.)

(2) The UI still sucks more than any other browser. Widgets and graphical elements misaligned from their active (i.e. clickable) zones, tearing and refresh issues for stateful widgets, etc. The point of the UI is to metaphorically embody what's going on in the code. Once the UI no longer reflects program state, you basically can't talk to your program.

(3) Crashes. Firefox remains the most crash-happy of the browsers. It does this at random. My last crash-followed-by-bug-reporting window was yesterday, when I fired up the latest version of FF for OS X to survey the meta titles of a bunch of pages rapidly. (My biggest complaint about Chrome is the absence of the meta title in the title bar.) About 10 minutes in, FF crashed. My uptime is measured in months right now, and I've had instances of Chrome up for that long. With FF I'm lucky if I get two days.

(4) Graphical appearance. It's damned hard to find a nice, professional FF theme that looks minimal. I just want something that has the same ethos as Chrome or Safari: simple widgets, no cruft, all of a cloth, that integrate well with the OS appearance. There are dozens of FF themes and "personalities" but all of them have that same "I'm OSS!" appearance that KDE also suffers from.

If Firefox were to stop sucking on these points, I'd give it another look regardless of memory use. But it's been a lot of releases since 2009 and though I keep the latest version of FF installed for cross-browser testing, I haven't seen any improvement on these points that would make me want to switch for my general browsing needs.

Workset and memory usage

[cnettel]

The main problem with this article is that it seems like the authors are simply reporting the compound number for Memory - Working Set for all browser processes. This is problematic in so many ways it ain't even funny. Foremost, multiple processes requiring the same page mapped into RAM will count that page multiple times. With architectures involving a lot of cross-process communication and multiple instances, like IE and Chrome, all code pages that are mapped into multiple processes will be counted multiple times (including any Windows OS DLLs mapped by IE), as long as they are part of the active working set. In addition some of the inter-process communication is probably handled by mapping common regions of memory, and thus also counted multiple times.

So, in one sense, IE and Chrome are losing out big in the multi-tab test. On the other hand, when tabs are closed, the related processes are also completely closed. This means that the associated heap is returned in full. In Firefox, even if some regions of the heap are freed back to the OS, Windows will not actively reclaim that working set. Slowly, the untouched pages will be removed and replaced by caching, but if there is no memory load, the best guess by the OS is that a process that just used a lot of memory and freed it might start doing it again. It is possible that Firefox, due to the monolithic process structure, ends up with a more fragmented heap (or a heap implementation that is not returning pages to the OS) so the OS could not successfully reclaim the pages (and only page them out to disk as a last resort), but we do not know that from these tests. It is easy to try this yourself, write up some small C program allocating a big buffer, freeing it and then pausing for a scanf/gets or something.

It is relatively easy to measure CPU usage or amount of I/O. Total time usage is also easy to measure and it says something about the conditions even under load. Memory is quite different. Memory is allocated all the time and the total bandwidth usage is closely related to the actual computations of the CPU. Even in a fully virtualized environment, the hypervisor cannot keep too detailed stats on memory usage - the overhead would balloon. At the same time, memory is a constrained resource with complicated temporal dependencies. You cannot free some memory from a process now and just give it back later. And when you bring multiple related processes into the mix, what you measure is all depending on how you define your tests. In many situations, the best metric is probably to look at full system metrics anyway, i.e. delta on total available/free memory in Windows. Many of the same issues will still apply, though. The question really becomes why you are interested in memory usage. If you want to know how the OS will behave when another process is allocating a lot of memory, test for that and verify memory usage as well as load times when you bring back the browser.

Re:You know, I really don't care

[Wordplay]

I'm a little confused. I'm on latest Lion. Just killed Firefox. Restarted it. There are my tabs. As far as I know it should work for you.

I work for Mozilla QA. If you want to talk about this more, you can contact me at gmealer@mozilla.com (my name is Geo) and I'll either help you out or direct you to someone who can.