Real-World Firefox 3 Memory Usage Leads the Field

An anonymous reader writes "The author developed a program to snapshot memory usage per process every 3 seconds on Windows. Using this he recorded 3 hours of memory usage for five different browsers under real-world usage scenarios: Safari 3.1, Firefox 3, Flock 1.2 (a browser based on Firefox 2), Opera 9.5, and Internet Explorer 8. A million data points indicate that Firefox 3 has a surprising advantage over the other browsers tested. These are real-world tests and not contrived benchmarks."

Comments

Wonder what Firefox 2 looked like ...

[xmas2003]

Interesting test - pretty amazing how FF3 basically flatlines at around 120 MBytes for over 2 hours of usage ... would have been interesting if the same methodology could be used with FF2 to see how much of an improvement FF3 is over that and how well the leaks were fixed.

Re:Wonder what Firefox 2 looked like ...

[Bandman]

Cue Iraqi spokesman

There are no memory leaks. All memory usage is as we intend it to be. Any reports of leaks are lies by people who do not understand our page caching system. The infidels will never take Baghdad.

Re:Wonder what Firefox 2 looked like ...

[Mascot]

The article states FF3 is an improvement over FF2, without offering data points for FF2. However, it also mentions Flock is based on FF2, so I'm guessing they've assumed the Flock data is representative for FF2.

Re:Wonder what Firefox 2 looked like ...

[reg]

That won't do those without 120 MB of memory to spare much good. Setting a cap on memory usage isn't a good solution, IMHO -- using well-designed memory handling that proactively frees memory seems to me to be a far better solution than a cap and garbage collection model.

(-1, Uninformed)

Firefox has no global cap on memory. It will dynamically configure it's caches (to some extent) based on the available RAM. It would be a stupid design to leave lots of RAM free, and reload stuff over the net. It also proactively frees memory, in most cases, although it sometimes delays a little, because it knows that you might turn around and reuse all of that memory you just stopped using. The GC is just for JavaScript (required by design) and for DOM nodes which end up being circularly referenced (which is unavoidable).

Finally, 120MB is not a lot of RAM. Notice that the other browsers are using similar amounts of RAM.

Regards
-Jeremy

Re:Wonder what Firefox 2 looked like ...

[icegreentea]

Looking at the RAM usage from TFA, 120 megabytes is acceptable. Web browsers apparently use a lot of RAM (when you load a lot of sites).

Re:Wonder what Firefox 2 looked like ...

[neokushan]

Perhaps in days gone by it would have been shocking for browsers to use so much memory, but the memory being used is directly related to the content on the sites it browse.
We recently seen studies indicating that the average website has tripled in size in the last couple of years alone, imagine how much bigger websites have got in the last 10-15 years?
More images, higher resolution/dpi ones at that, flash, plus there's all the scripting engines and other plugins that have accumulated as well.
While I comlpetely agree that software bloat shouldn't be accepted (Nero, I'm looking at you) and if anything, programs should become MORE efficient with age (in an ideal world), I think browsers could be one exception since the content they're handling has got so much bigger.
So really, the best way to indicate progress here is to pit the fully featured browsers against each other and see who comes out on top. Probably still wont be firefox, but I bet the big three still use a lot of RAM compared to what was the norm a few years ago.

What time of day did he do his tests?

[Anonymous Coward]

I find that for certain hours of the day (namely in the evening) My memory usage skyrockets. It probably has to do with the increased number of images I am loading :D

But what memory metric was taken?

[Anonymous Coward]

IIRC the memory displayed in process manager isn't necessarily the memory requested/used by the program, but merely what Windows has allocated, partially based on the applications requirements and partially based on what Windows _thinks_ the program needs.

As such there's room for applications to look like they're using more memory than they are which can lead to misleading stats. If this test has only taken into account the memory windows has allocated it doesn't necessarily act as a measure of how efficient the program is at least, just how good it is at playing Window's memory management system.

Re:But what memory metric was taken?

[Evanisincontrol]

Ok, I see your point. Let's pretend Browser X is using dirty tricks with Windows's memory management system to shrink down how much memory is allocated to it. Browser Y is not doing that and appears to be less efficient.

Well and good, but it's irrelevant. The remains that Browser X is taking less memory from Windows's pool of resources. It doesn't matter how Browser X is doing it or how efficient Browser X being with the memory internally, it is a solid truth that Browser X is using occupying fewer system resources than Browser Y.

It's really a moot point, because it's unlikely that the developers of Browser X knew any "cheats" that would let them use substantially less memory than every other browser out there.

HTTP 503'd (aka /.ed)

[scatters]

Service Not Available.

At the time of posting this, there were like, 10 comments in the thread. Assuming that only 10% of all /.ers RTFA, that means that the site can support only 1 simultaneous user.

Service Unavailable

[MessageDrivenBean]

Already /.-ed? Or do I have to use any of the mentioned browsers... :-)

Bah.

[Anonymous Coward]

If they didn't compare with Dillo/lynx, it's meaningless. Also, already slashdotted.

Terrible reference

[toleraen]

Between:

These aren't stress tests, and I probably never went over 4 windows in each browser, with at most 3 tabs in each window.(Emphasis mine)
and
.the individual numbers should not be compared to each other...

...how is this supposed to be taken seriously? "Contrived benchmarks" at least provide consistent and reliable results. They may not provide a completely accurate picture of real world browsing, but it's a hell of a lot better than this anecdotal "test".

Re:Terrible reference

[CyberLife]

This was my first question too. Real-world testing is all well and good, but how controlled was it? What assurances do we have that his results really paint the picture he claims and not something else?

Note: The site is down so I haven't read the article yet. I'm guessing it fails to address this concern?

Wait

[Mensa Babe]

Don't get overexcited just yet. Let me quote some of the most important parts of the article that were completely overlooked in the summary for some reason:

"These results are from opening Memory Watcher and then using the browser between 9,000 and 11,000 seconds (close to 3 hours). Each browser is tested in a separate session, and there are brief periods of inactivity throughout the time period. [...] The above profiles are not a direct comparison in any way, but they offer a visualization of trending in the memory behavior of the layout engines and interfaces. [...]These aren't stress tests, and I probably never went over 4 windows in each browser, with at most 3 tabs in each window. [...] An automation script will never give the same insight into performance over time as will this sort of profile." [emphasis added]

In other words, it is evident that there was no guarantee whatsoever that every browser would display exactly the same sequence of web pages. It is easy to jump to conclusions that if Firefox has used the least memory then it must "[have] a surprising advantage over the other browsers." But is it a logical course of reasoning? Or only a post hoc ergo propter hoc fallacy combined with wishful thinking? The truth is that the amount of memory used during an hour of downloading web pages is strongly correlated with the speed of downloading and displaying said web pages. Is it the case that Firefox couldn't download, format and display pages as quickly as Internet Explorer because of the native Windows internal API hooks that help Explorer work faster than any independent browser could possible aspire to? That is quite possible. Unfortunately the results of that experiment are inconclusive and the methodology was unreliable.

If slashdotted

[Anonymous Coward]

Final memory usage in MB
Safari 636.9
Firefox 3 111.8
Flock (Firefox 2) 191.9
Opera 9.5 190.6
Internet Explorer 194.4

Re:If slashdotted

[Koiu Lpoi]

Ffffffuck, is that a typo on Safari? That's stupendously huge! Is that because they have to load up some kind of ridiculous layer to make this mac-alike app run on windows? I know that iTunes uses a ton of ram too, but not six hundred megs huge.

How did they measure memory consumption?

[Idaho]

As the server is (already!?) down, I didn't yet have a chance to RTFA. So perhaps it is in the article somewhere, but I couldn't help wondering: how did they actually measure memory usage?

I'm asking because, these days, that pretty much amounts to rocket science.

Different operating systems report memory usage differently, even between different versions of the same OS (yes, I'm looking at you, Vista vs. XP). If they used "top" or its equivalent, it matters a lot whether they looked at real usage, virtual memory size (can be huge but that doesn't say anything) or what-have-you. Some OS's cheat quite a bit in what memory is reported as being "free" or "available", as well. Then we get to questions like "does it include the size of shared libraries", if not, is that fair if the libraries are really only used by that one application? Etc. etc.

So I'm not saying memory using doesn't matter (it very much does), it's just hard to measure it exactly. And, any attempts at doing so, should be documented precisely.

Re:How did they measure memory consumption?

[jd]

I don't know about these guys, but since most of these will be dynamically linked files, I'd probably be looking for ways to trap the malloc library calls, in much the same way as most of the debug malloc implementations do for various Unixes. If you track the maximum usage and the unfreed total, you can determine the memory consumption and memory leakage without relying on any system-specific interpretation of what memory is.

Re:How did they measure memory consumption?

[Tweenk]

The OS was Vista, and the program was written in .Net to use the function PrivateMemorySize64. MSDN says it returns "the amount of memory which cannot be shared with other processes". It also says it's the same as the "Private Bytes" value in taskmon. Probably it means that it's the amount of memory the process received from mallocs (or rather GlobalAllocs/LocalAllocs/HeapAllocs), and which can't be assigned to some other process.

It's worth noticing that the guy bothered with a GUI and an interactive filtering option for such a simple program. I wonder whether he ever heard of CLI, because it looks like a perfect fit for this kind of program.

By the way, why not post CoralCDN links (append .nyud.net to hostname) instead of direct links when the site in question is small and likely to be Slashdotted?

Re:Lools IIS can't hold its own

[nmb3000]

Lools IIS can't hold its own

Haha! That's funny and insightful!

Oh, wait.
The term "slashdotted" has become ubiquitous with smashing a webserver due to high traffic.
Most webservers are *nix based (though admittedly IIS is gaining ground).
Hm. Nevermind.

Great news FF - though Opera is speedy

[SgtAaron]

I've been blessed with using a Duron 950Mhz with a gig of RAM, lately. Quite speedy. Heh. But I've used worse, as many can no doubt also say. Oh, and an GeForce4, and of course the X Window System. :-)

I've always used Firefox, and Netscape before that, on my linux desktops. I must say that I tried Opera lately, for the first time, and found its rendering to be very spry. The difference was most noticable for me when loading very large web pages, or very detailed with lots of tables and such. The latter was our nagios service detail page, which the rendering in Opera was quite noticeable in its quickness.

So I get to be torn now, maybe, speed vs lean...
I do like speed. Opera's memory use doesn't seem to be so excessively bad as to negate the optimizations they seem to have coded into the rendering.

Aaron

Memory?...what about speed?

[Last_Available_Usern]

While it's admirable that it's the leanest of the bunch, if I have 2GB of memory and over half of that is unused at the moment, do I really care if my browser uses 25MB instead of 40MB? I would think the speed with which the browser (and subsequent windows) opened, as well as how quickly it loaded plug-ins and other embedded media, would be of more importance.

Re:Memory?...what about speed?

[AbRASiON]

I couldn't agree more, I want an incredibly incredibly snappy browser, I don't care how much resources it requires.

I have a theory and I'm convinced of this theory that cache in browsers simply has been broken, since the beggining
I've used netscape, ie, ffox, netcaptor and god knows what else and no matter how big I set the cache or how regularly I hit sites, they still seem slow to load images and content.

Perhaps it's the complexity of the pages has scaled up, I spose that is possible.
I've gone from browsing on a Pentium 166mmx with 32mb over the years up to a quad core 3.2ghz machine with 4gb of ram and moved from 128k dsl / 512k dsl / 1.5mbit dsl / 8mbit dsl and 18mbit dsl.

I am impatient, make no mistake but surely if we're just throwing images and text around, these damned things should be snappier?
FFox 3 is definitely a move in the right direction, it's noticably faster than FFox 2 but I still find moving back / forward, clicking links and interaction in general should still be faster :/

Re:Careful...

[Paul Neubauer]

Not really. Many Opera users are finding 9.50 to not be as good as claimed or hoped and finding it to be a memory hog. I am not alone in looking at 9.50, finding the the 9.51 snapshot to be less buggy, and sticking with 9.27 for normal non-browser testing browsing.

Now, maybe when Opera 9.52 or so is out, there might be some valid concern.

This doesn't mesh with my experience

[Curien]

My wife and I share a computer. She uses mostly uses Firefox, I mostly use Opera. This is on a 64-bit Ubuntu Hardy.

I have noticed no difference in her memory usage since we upgraded to FF3. I used to regularly have to kill her browser every once in a while (maybe once or twice a week) because it was eating up all the RAM. Since we upgraded to FF3, I can see no difference in memory usage.

For example, right now FF is using 300MB resident, Opera is using 100MB. Flashblock is installed on both browsers. Granted, that's not a terribly good test considering we've been browsing to different sites, but I've found that those numbers are fairly stable. FF usually levels off in the 300-500MB range, and Opera in the 100-150 range.

YMMV.

Re:This doesn't mesh with my experience

[diegocgteleline.es]

64-bit firefox eats a LOT more of memory. The windows versions are 32-bit only.

Re:you do know what "contrived" means right?

[toleraen]

Tests need to be repeatable, and when they are repeated they need to provide consistent results. If you can't provide consistent results on each subsequent test, you need to provide solid rational on why the results were outside your margin of error. Saying "Oh, I might have had one or two youtube videos up at the time" is not solid rational.

Scripts that visit the exact same pages, for the exact same time, do the exact same things across all browsers provide consistent, quantifiable results. Since everyone's browsing behavior is different no script will ever provide "accurate" results for real world usage. But then again, those scripts could be closer to my real world usage than this guys anecdotal test. Get it?

Re:Yah, but how reliable?

[rrkap]

I haven't had Firefox 3 crash for me yet (although I've only been using it since Download Day). I have noticed that it no longer gets hung up processing javascript the way Firefox 2 often did.

Re:Yah, but how reliable?

[Tweenk]

This is most likely related to the Flash plugin. The second suspect would be the Java plugin. For me Firefox never crashed on a website without Flash and Java, but I had a few crashes due to Flash bugs.

Irrelevant...

[at_slashdot]

That's pretty much irrelevant to me, I switched back to Opera because of performance issues.

Memory != performance.

For example, when I open a new tab in opera the CPU doesn't register almost any change, when I open a tab in Firefox it goes almost to 100% (that's in Linux, with many extensions added, and BTW, I need those extensions to duplicate Opera's features)

Re:Irrelevant...

[treat]

On my Fedora Core 7 system, with a 2.5ghz CPU, running "vmstat 1" causes the X server to use 5% CPU. (It is an nvidia graphics card, but I don't use their binary-only driver).

Typing in this firefox-2 text window gives a CPU usage of about 30%.

Opening a new tab causes 60% CPU utilization for 2 seconds.

I think it's time to give up on Linux on the desktop.

Vigorous mouse movement causes about 35% CPU utilization.

Re:Irrelevant...

[Grey Ninja]

That sounds like all your rendering is being done on the CPU to me. Windows will do the same thing if you refuse to use Nvidia's binary only Windows driver.

That being said, I have similar problems on my laptop, where scrolling a window often will peg my CPU. In my case, I have an older Intel integrated chip, which does a lot of 3D rendering on the CPU. Since the advent of AIGLX, it makes some rendering on my laptop fairly slow. At least, I THINK that's what's going on there. It's never really bothered me enough to look into it in-depth.

Re:Lools IIS can't hold its own

[Zero__Kelvin]

"The term "slashdotted" has become ubiquitous with smashing a webserver due to high traffic.
Most webservers are *nix based (though admittedly IIS is gaining ground)."

So what you are trying to say is that Slashdotting is becoming more and more popular?

Definitely important

[XanC]

In the case of Firefox, memory usage has ranged from 25MB to $MEMORY_AVAILABLE. Which sucks no matter how much you have.

What cap?

[clarkn0va]

Setting a cap on memory usage isn't a good solution, IMHO -- using well-designed memory handling that proactively frees memory seems to me to be a far better solution than a cap and garbage collection model. I haven't seen any mention of a cap, even if that's a natural conclusion based on the flat line that one can observe in the graphs.

If you check this fairly lengthy explanation of how memory usage was improved in FF3 you'll see that it is mostly attributed to reduced fragmentation and leaks, and smarting caching, just as you are advocating.

db

Re:...contrived benchmarks.

[billcopc]

<keynote style="Steve">
Safari on OS XI is going to be 400% faster. It's going to look 700% rounder, and integrate seamlessly with your ego. It will make you 1500% more smug, no matter how smug you were before.

Firefox ? Not smug.
IE8 ? *chuckles* next slide.
Opera ? They still have square corners, what does that tell you about their priorities ?

It's so awesome we had to give it a new name: Snow Safari.
(*applause*)
</keynote>

Re:Careful...

[Ilgaz]

' They're trying to charge money ofr something everyone else gives away for free, and with the latest browsers they're running out of legitimate advantages to boast."

Please change that template, it is 2008 already.

Re:you do know what "contrived" means right?

[bunratty]

You mean like these consistently reproducible results?

Firefox 3 used less memory than Firefox 2, Internet Explorer, and Opera, and it also freed more memory than the other browsers when pages were closed... The results of this experiment, which others have been able to consistently reproduce using the same tools, represent a big victory for Firefox, which has previously faced widespread criticism for its high memory consumption.

The application doesn't allocate memory, though

[Gazzonyx]

Well, your OS allocates that memory. In the case of Windows, if you hit 40 MB once, it very well may leave that much dedicated to FF, even if it's only using 25 MB internally. So, you have dedicated, unused RAM.

I think on Linux, how fast it will put allocated and unused RAM back to the pool will depend on the vfs_cache_pressure, but I'm not sure about that, as that reclaims inodes from cache to make room for the buffers. VM management always confused me.

Re:you do know what "contrived" means right?

[toleraen]

Yup, just like those. I'm not disputing the claims that FF3 uses less memory (been using it since alpha 3). I just do a lot of software testing at work, and I thought I'd bring out my inner pedant in relation to this supposed "test".

Re:keep laughing

[just_another_sean]

You mean Debian. Or Debian Lenny. No need to use "GNU/Linux" at all.

And laugh all you want, but your characterization of this as an "uptime" issue is incorrect to say the least.

I am as disgusted by twitter as the next /. user but what's wrong with GNU/Linux? That's what Debian calls their distro.

I don't insist that every distro add GNU to their name; that's up to them. But Debian chooses to so at least in their case could we maybe lose the "RMS is teh zealot!!" jokes?

Debian is a great dsitro for promoting Free software and I am pretty sure they made the choice to keep GNU in the name on priniciple. I respect them for it and it's a major reason why Debian remains my distro of choice.

Re:Hi twitter

[Crazyswedishguy]

He does raise an interesting question (and I'm in no way bashing Vista):
how do browsers compare on Mac OSX? Does Safari compare similarly to Firefox 3 on Mac OS X?
I wouldn't be surprised if Safari performs better on OS X than on Vista because Apple has put more time into developing the Mac version...
Similarly, I wouldn't be surprised if Firefox is optimized for the larger Windows market. I know that back in the day, on the old PPC macs, Firefox was pretty slow, which was a good reason to use Safari.

Except on Linux

[DrYak]

It would be a stupid design to leave lots of RAM free, and reload stuff over the net. Well, it depends on your OS.
Linux automatically uses all free memory for disk cache, and is very efficient at it. Instead of hogging memory that could be used by other process, Firefox could exclusively rely on a disk cache and rely on Linux' native and performant disk-cache to handle the in-memory caching of those files.
The best part of this is, if some other process needs the memory, Linux will simply free some memory from the cache, but the files will still be ready on the disk and the over application will still be performing well. Whereas if a 120Meg space is enforced a in-RAM cache, when memory becomes scarce, the system is at risk of paging out piece of the software (pages of code itself instead of pages holding cache) and thus make the whole system less responsive.

The GC is just for JavaScript (required by design) and for DOM nodes which end up being circularly referenced (which is unavoidable). You would only need to keep the DOM nodes of the current page. Past pages are freed and don't (usually) keep DOM objects alive.

Finally, 120MB is not a lot of RAM. Well, it depends. Notice that Linux is also very often used on kiosk with limited features and on old hardware which may not have huge amounts of resources.

Being able to run within a small memory space is critical for linux. Otherways, there won't be any difference with Vista.

Also a lot of problems are comming from bad Add-ons or even half-assed Browser Plugins. Flash is such a pain in the ass that can momentanily freeze the whole browser session.

Disclaimer : I run Firefox on Linux with in-memory cache disabled and using Gnash plugins instead of Flash (runs in separate process and can have autostart disabled). Adblock+ and Noscript also help avoiding that my browser loads tons of useless shit. And until recently my main desktop was a Pentium-III with 440BX chipset (a machine on which 1GiB of RAM is a rare occurence), but I didn't get any major problem even on recent distros. (Vista on the other hand had to wait)

Re:Except on Linux

[packman]

I can't say that what you're saying is very 'informative', sorry. While linux indeed uses free memory to cache files, browsers do not cache 'files' in memory, they cache fully preprocessed DOM-trees and pre-rendered parts of webpages. Disabling in-memory caching only hurts performance there, forcing the browser to reprocess and revalidate all your pages. The only reason your browser uses the disk is because it's faster than your network connection and saves speed between browser-sessions or when visiting a page which has been cleaned up from your browser's own memory cache. Disk I/O - even with loads of OS-level caching will always be slower than letting the application just keep stuff in memory.

Most of your browser's in-memory cache is pre-processed data, including the last X entries accessible with your back-button - and that for each tab - are cached. This means images, DOM-tree, CSS-styles and sometimes even pre-rendered parts of the page.

I really hope you don't believe the disk io is the limit here. Parsing, validating and rendering takes up a good part of the responsiveness - even on modern PC's with fast quad-core cpu's. It's not only the CPU's that are advancing in speed, also webstandards require more & more processing power.

Also with todays machines with 1GB+ RAM - I don't really see the problem of memory useage. I have 4GB at home - and the moment my linux uses even only 25% of that for disk caching, it will have very old stuff in that disk-cache of which a lot would probably never be used again. I'd rather have my browser use 2Gb and feel extremely snappy, than have it use less memory just to preserve memory.

Re:How long has 256 MB been standard?

[flappinbooger]

Or, the other user is the botnet admin who 0wnz his box.

Re:-5 (Outdated)

[Ilgaz]

On fairly modern Nokia E65 with Apple Webkit based free "Nokia browser", I did total 1 hour browsing. That is heading to Opera.com for whatever they offer for my OS (it took that much to reach the right download page) and later bought it with Opera Mobile client itself.
Thank God Opera Mobile exists as a choice. That is their 2 years old 8.65 code I bought. It is AGES ahead of anything on that smart phone.
If they charge for their marvellous work, a browser which uses less memory than 8 MB on my S60 device that will , hell it is $20!
That 2 years old code picks the right css from my web pages (all w3c compliant) and shows them like I have setup a special WAP site for phones.
Now waiting for their 9.5 achilles heel.
What bothers me is, they get 5-7 million downloads without advertisement (9.5 Desktop), they help hundreds of millions of possible devices doesn't even have a good WAP browser to access the web, they strictly support standards, they never sold out to MS (imagine Opera quality IE only exists on Win Mobile), they even support Windows 98 and they get THAT comment as return.

Re:Does it still have the Awesomebar? It does?

[LargeMythicalReptile]

Parent is not informative.

Took me all of 10 seconds with google: "Firefox3 Awesomebar disable". And yet, that page is absolutely not what the GP was asking for.

To quote:

Some things are more important than resource conservation, such as not screwing the user by needlessly taking away functionality and telling them "you'll get over it".

I'd gladly have Firefox 3 with the same footprint as Firefox 2 if that's the price to pay for keeping the old address bar autocomplete functionality in the code.

gumpish--and I, and a lot of people as far as I can tell--want the "classic" (read: FF2) location bar behavior. There is currently no way to get that in FF3. There used to be an about:config setting for it in some of the betas, but they took it out and told those people who liked the old functionality to shut up and deal.

The about:config tweaks on the page you linked to will disable the location bar dropdown entirely (heck, it even says as much in the edit--did you even read the page you linked?).

A sibling post suggests the oldbar extension, but that only changes the appearance of the bar and not the behavior.

As I said, there is currently no way to get this functionality in FF3. The closest thing is this extension, but that's not perfect. Really, this bug needs to be fixed.

Can you imagine...

[Foerstner]

...what it would be like if Apple came up with a benchmark for web browsers? They'd do some kind of splashy announcement. Geeks would question its relevance in the real world. Soon they'd do some new tests and proclaim that the competition now performs better on those same tests anyway. Eventually, the rabid Apple phanbois will claim that the next release will bury the competition.

Apple. So predictable.

Re:Hi twitter

[gilgoomesh]

I've been running Safari 3.1.1 all day (about 6 hours non-stop with 462 pages in history) and it's currently using 230.9MB RAM (fairly steady).

Opening up FF3 to check, loading 52 pages then closing down one again... 129.18MB.

Not a totally fair comparison but it's some numbers, nonetheless.