Real-World Firefox 3 Memory Usage Leads the Field
An anonymous reader writes "The author developed a program to snapshot memory usage per process every 3 seconds on Windows. Using this he recorded 3 hours of memory usage for five different browsers under real-world usage scenarios: Safari 3.1, Firefox 3, Flock 1.2 (a browser based on Firefox 2), Opera 9.5, and Internet Explorer 8. A million data points indicate that Firefox 3 has a surprising advantage over the other browsers tested. These are real-world tests and not contrived benchmarks."
Interesting test - pretty amazing how FF3 basically flatlines at around 120 MBytes for over 2 hours of usage ... would have
been interesting if the same methodology could be used
with FF2 to see how much of an improvement FF3 is over that
and how well the
leaks were fixed.
Hulk SMASH Celiac Disease
I find that for certain hours of the day (namely in the evening) My memory usage skyrockets. It probably has to do with the increased number of images I am loading :D
IIRC the memory displayed in process manager isn't necessarily the memory requested/used by the program, but merely what Windows has allocated, partially based on the applications requirements and partially based on what Windows _thinks_ the program needs.
As such there's room for applications to look like they're using more memory than they are which can lead to misleading stats. If this test has only taken into account the memory windows has allocated it doesn't necessarily act as a measure of how efficient the program is at least, just how good it is at playing Window's memory management system.
Service Not Available.
At the time of posting this, there were like, 10 comments in the thread. Assuming that only 10% of all /.ers RTFA, that means that the site can support only 1 simultaneous user.
A One that isn't cold, is scarcely a One at all.
Already /.-ed? Or do I have to use any of the mentioned browsers... :-)
Quisque verborum suorum optimus interpres...
If they didn't compare with Dillo/lynx, it's meaningless. Also, already slashdotted.
Between:
.the individual numbers should not be compared to each other...
...how is this supposed to be taken seriously? "Contrived benchmarks" at least provide consistent and reliable results. They may not provide a completely accurate picture of real world browsing, but it's a hell of a lot better than this anecdotal "test".
These aren't stress tests, and I probably never went over 4 windows in each browser, with at most 3 tabs in each window.(Emphasis mine)
and
Don't get overexcited just yet. Let me quote some of the most important parts of the article that were completely overlooked in the summary for some reason:
"These results are from opening Memory Watcher and then using the browser between 9,000 and 11,000 seconds (close to 3 hours). Each browser is tested in a separate session, and there are brief periods of inactivity throughout the time period. [...] The above profiles are not a direct comparison in any way, but they offer a visualization of trending in the memory behavior of the layout engines and interfaces. [...]These aren't stress tests, and I probably never went over 4 windows in each browser, with at most 3 tabs in each window. [...] An automation script will never give the same insight into performance over time as will this sort of profile." [emphasis added]
In other words, it is evident that there was no guarantee whatsoever that every browser would display exactly the same sequence of web pages. It is easy to jump to conclusions that if Firefox has used the least memory then it must "[have] a surprising advantage over the other browsers." But is it a logical course of reasoning? Or only a post hoc ergo propter hoc fallacy combined with wishful thinking? The truth is that the amount of memory used during an hour of downloading web pages is strongly correlated with the speed of downloading and displaying said web pages. Is it the case that Firefox couldn't download, format and display pages as quickly as Internet Explorer because of the native Windows internal API hooks that help Explorer work faster than any independent browser could possible aspire to? That is quite possible. Unfortunately the results of that experiment are inconclusive and the methodology was unreliable.
Karma: Positive (probably because of superiour intellect)
Final memory usage in MB
Safari 636.9
Firefox 3 111.8
Flock (Firefox 2) 191.9
Opera 9.5 190.6
Internet Explorer 194.4
As the server is (already!?) down, I didn't yet have a chance to RTFA. So perhaps it is in the article somewhere, but I couldn't help wondering: how did they actually measure memory usage?
I'm asking because, these days, that pretty much amounts to rocket science.
Different operating systems report memory usage differently, even between different versions of the same OS (yes, I'm looking at you, Vista vs. XP). If they used "top" or its equivalent, it matters a lot whether they looked at real usage, virtual memory size (can be huge but that doesn't say anything) or what-have-you. Some OS's cheat quite a bit in what memory is reported as being "free" or "available", as well. Then we get to questions like "does it include the size of shared libraries", if not, is that fair if the libraries are really only used by that one application? Etc. etc.
So I'm not saying memory using doesn't matter (it very much does), it's just hard to measure it exactly. And, any attempts at doing so, should be documented precisely.
Every expression is true, for a given value of 'true'
Lools IIS can't hold its own
Haha! That's funny and insightful!
Oh, wait.
The term "slashdotted" has become ubiquitous with smashing a webserver due to high traffic.
Most webservers are *nix based (though admittedly IIS is gaining ground).
Hm. Nevermind.
"What do you despise? By this are you truly known." --Princess Irulan, Manual of Muad'Dib
/)
I can't be arsed to do this properly. There are a lot of graphs and stuff. But here are the text bits. Pure copy and paste. No formatting. Live with it.
Web browser performance is an often talked-about and flaunted thing, but many claims are not really backed up by solid evidence. I wrote software that collected millions of data points over 14 hours of actual browsing time, and this article reveals my findings.
Problem
Many people load hundreds of web pages, sometimes at the same time, often over periods of 3+ hours. Users complain about the memory usage of Firefox, Safari, or Internet Explorer, and we need a way to identify which browsers are better at managing memory than others. Traditional benchmarks do not look at all the things you might do with a program, and we need real-world numbers over a period of hours.
Solution
I developed a Windows Forms application in .NET called Memory Watcher that "watches" the system memory numbers. It uses a timer to poll the processes every 3 seconds. It then records every number and also prints them out in a grid on the screen. This allows us to keep track of each program's memory usage over time and with real-world usage.
Memory Profiles
These results are from opening Memory Watcher and then using the browser between 9,000 and 11,000 seconds (close to 3 hours). Each browser is tested in a separate session, and there are brief periods of inactivity throughout the time period. The vertical axis is the memory used in MB, and the horizontal axis contains the memory "checkpoints" my program took (one every 3 seconds).
(Graphs and more graphs)
Benchmark Details
The above profiles are not a direct comparison in any way, but they offer a visualization of trending in the memory behavior of the layout engines and interfaces. This is not a diagnosis or bug report. Let me show some important metrics of the above results.
Browser name Exact version Time active (s)
Hours Comments
Safari 3.1.2 10,470 s
2.91 hours Normal browsing
Firefox 3.0 9,681 s
2.69 hours Normal browsing
No extensions
Flock 1.2.2 10,146 s
2.82 hours Flock is based on Firefox 2.0
No extensions other than the default
Opera 9.5 9,855 s
2.74 hours No extensions
Only browser was used
IE 8.0 10,236 s
2.84 hours Used 7.0 rendering mode
No extensions
The system is Windows Vista SP1, and the computer has 3.0+ GB of RAM. No plugins are disabled, but the Acrobat Reader and Java plugins were (presumably) not used. Flock is based on Firefox 2.0 but its memory usage is probably worse because it uses built-in extensions.
* Just regular stuff
* These aren't stress tests, and I probably never went over 4 windows in each browser, with at most 3 tabs in each window. I didn't look at many pages that are extremely heavy on images, and no "browser benchmark" style pages. Gmail was used on each browser.
* Not just pages
It is hard for a regular benchmark to "simulate" a user actually clicking on things. Interactions with the user can greatly influence memory or performance. Having a responsive browser is probably more important than just having a "fast" one at showing pages.
* Plugins included
My profiles include Flash and possibly other plugins. A browser might have memory issues with a plugin and that could cause a significant problem with the user experience. (Most Windows Vista crashes have been due to graphics cards, not Vista itself, for example.)
* Real-life usage
An automation script will never give the same insight into performance over time as will this sort of profile. As developers, we want to make programs that work wel
Yeah, there are 2 of you. It's your fucking flash plugin d00ds.......
(why would anybody install that piece of crap is beyond me)
My Firefox never crashes, I repeat never. (Ubuntu 8.04) I use it for hours and hours every single day.
I've been blessed with using a Duron 950Mhz with a gig of RAM, lately. Quite speedy. Heh. But I've used worse, as many can no doubt also say. Oh, and an GeForce4, and of course the X Window System. :-)
I've always used Firefox, and Netscape before that, on my linux desktops. I must say that I tried Opera lately, for the first time, and found its rendering to be very spry. The difference was most noticable for me when loading very large web pages, or very detailed with lots of tables and such. The latter was our nagios service detail page, which the rendering in Opera was quite noticeable in its quickness.
So I get to be torn now, maybe, speed vs lean...
I do like speed. Opera's memory use doesn't seem to be so excessively bad as to negate the optimizations they seem to have coded into the rendering.
Aaron
While it's admirable that it's the leanest of the bunch, if I have 2GB of memory and over half of that is unused at the moment, do I really care if my browser uses 25MB instead of 40MB? I would think the speed with which the browser (and subsequent windows) opened, as well as how quickly it loaded plug-ins and other embedded media, would be of more importance.
Not really. Many Opera users are finding 9.50 to not be as good as claimed or hoped and finding it to be a memory hog. I am not alone in looking at 9.50, finding the the 9.51 snapshot to be less buggy, and sticking with 9.27 for normal non-browser testing browsing.
Now, maybe when Opera 9.52 or so is out, there might be some valid concern.
I don't subscribe to RMS's GNUtopian vision.
My wife and I share a computer. She uses mostly uses Firefox, I mostly use Opera. This is on a 64-bit Ubuntu Hardy.
I have noticed no difference in her memory usage since we upgraded to FF3. I used to regularly have to kill her browser every once in a while (maybe once or twice a week) because it was eating up all the RAM. Since we upgraded to FF3, I can see no difference in memory usage.
For example, right now FF is using 300MB resident, Opera is using 100MB. Flashblock is installed on both browsers. Granted, that's not a terribly good test considering we've been browsing to different sites, but I've found that those numbers are fairly stable. FF usually levels off in the 300-500MB range, and Opera in the 100-150 range.
YMMV.
It's always a long day... 86400 doesn't fit into a short.
Tests need to be repeatable, and when they are repeated they need to provide consistent results. If you can't provide consistent results on each subsequent test, you need to provide solid rational on why the results were outside your margin of error. Saying "Oh, I might have had one or two youtube videos up at the time" is not solid rational.
Scripts that visit the exact same pages, for the exact same time, do the exact same things across all browsers provide consistent, quantifiable results. Since everyone's browsing behavior is different no script will ever provide "accurate" results for real world usage. But then again, those scripts could be closer to my real world usage than this guys anecdotal test. Get it?
I haven't had Firefox 3 crash for me yet (although I've only been using it since Download Day). I have noticed that it no longer gets hung up processing javascript the way Firefox 2 often did.
I like my beverages with warning labels!
This is most likely related to the Flash plugin. The second suspect would be the Java plugin. For me Firefox never crashed on a website without Flash and Java, but I had a few crashes due to Flash bugs.
Those who would give up liberty to obtain working drivers, deserve neither liberty nor working drivers.
That's pretty much irrelevant to me, I switched back to Opera because of performance issues.
Memory != performance.
For example, when I open a new tab in opera the CPU doesn't register almost any change, when I open a tab in Firefox it goes almost to 100% (that's in Linux, with many extensions added, and BTW, I need those extensions to duplicate Opera's features)
"It is our choices, Harry, that show what we truly are, far more than our abilities." -- Prof. Dumbledore
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
In the case of Firefox, memory usage has ranged from 25MB to $MEMORY_AVAILABLE. Which sucks no matter how much you have.
If you check this fairly lengthy explanation of how memory usage was improved in FF3 you'll see that it is mostly attributed to reduced fragmentation and leaks, and smarting caching, just as you are advocating.
db
I am literally 3000 tokens away from the chaotic crossbow --Stephen
Some, but I'm not one of them, my Opera (9.51 on XP SP3) has been running for a few days now (almost 4), with a peak of 157MB, currently at 96MB, VM of 114MB, and an I/O of almost 12GB's... opening up every site on my SpeedDial (9 sites + this one) brought me up to 122MB, VM of 140MB...
But, I honestly don't care how much it uses, because so far it hasn't impacted (noticeably) on any other software, and always starts (launches, or maximizes) instantly, and I prefer Opera's interface. And how much memory it uses isn't enough to make me prefer one browser over another... CPU usage on the other hand, might, but most of them are pretty much the same in that regard.
And after closing all those tabs, its down to 92MB, VM 110MB, peak the same.
http://dotnetperls.com.nyud.net/Content/Browser-Memory.aspx
<keynote style="Steve">
Safari on OS XI is going to be 400% faster. It's going to look 700% rounder, and integrate seamlessly with your ego. It will make you 1500% more smug, no matter how smug you were before.
Firefox ? Not smug.
IE8 ? *chuckles* next slide.
Opera ? They still have square corners, what does that tell you about their priorities ?
It's so awesome we had to give it a new name: Snow Safari.
(*applause*)
</keynote>
-Billco, Fnarg.com
I'd like to see this same test measured over performance (mem over load time) - that would tell us something interesting. The working set numbers are somewhat arbitrary, that is the allocation scheme they may have been "optimized" by the dev team in some way. The fact Safari isn't capping it's own usable makes me want to see why that is.
/LabMonkey09
' They're trying to charge money ofr something everyone else gives away for free, and with the latest browsers they're running out of legitimate advantages to boast."
Please change that template, it is 2008 already.
On the off chance you're not just trolling...
Instructions are here:
http://www.kennycarlile.net/2008/06/17/disable-firefox-3-awesomebar/
Took me all of 10 seconds with google: "Firefox3 Awesomebar disable". So... I guess you'll enjoy firefox 3 with the smaller memory footprint (assuming TFM is correct).
Clones are people two.
What a fool believes, he sees, no wise man has the power to reason away.
Well, your OS allocates that memory. In the case of Windows, if you hit 40 MB once, it very well may leave that much dedicated to FF, even if it's only using 25 MB internally. So, you have dedicated, unused RAM.
I think on Linux, how fast it will put allocated and unused RAM back to the pool will depend on the vfs_cache_pressure, but I'm not sure about that, as that reclaims inodes from cache to make room for the buffers. VM management always confused me.
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
Yup, just like those. I'm not disputing the claims that FF3 uses less memory (been using it since alpha 3). I just do a lot of software testing at work, and I thought I'd bring out my inner pedant in relation to this supposed "test".
Firefox running on an AMD processor uses "2200+" Mb of memory, which actually means about 100 Mb in an Intel processor
NetworkMirror.
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
Look at this article talking about memory usage in Firefox 3 by mozilla dev Parlov. There is a graph there that suggests Firefox 2 uses around 80Mbytes more memory than Firefox 3 over time.
If an aftermarket upgrade part on your car causes the engine to blow up do you blame the original manufacturer of the car? It's not feasible to expect a framework to catch every possible problem that could arise from a poor implementation of that framework. You'd like to catch most of them but there will always be someone or something that can fubar it in ways you never imagined.
Play me online? Well you know that I'll beat you. If I ever meet you I'll "/sbin/shutdown -h now" you. -Weird Al, kinda.
Memory that is being shared between processes (e.g. through the use of libraries) is actually counted towards the RSS so something that is sharing a lot of pieces with another process may appear artificially high.
I, personally, wish flash development under Linux was remotely on par with the Windows offering. This has been one of the things holding the "old folks" in my family and down the street from running something nice on their PC like Ubuntu instead of Windows. An amazing number of them have Bongo premium accounts (and the like) and after setting one person up, then having to nuke Ubuntu for Windows because 90% of their flash-based games would no longer work, made me think something more needs to be done in this area instead of just ignoring it. Sadly, it's closed source and the open source options don't suit their needs. It's unfortunate, as if this one hitch were removed, I could think of five people I could get running Linux by next week. As it stands, I'm still waiting indefinitely to save them from Vista.
I'm not a coder, I'm an electronics/hardware/telco/admin/networking guy so I am pulling this out of my butt, but would it be worth investing in getting the Windows version of flash to run under Linux somehow?
You mean Debian. Or Debian Lenny. No need to use "GNU/Linux" at all.
And laugh all you want, but your characterization of this as an "uptime" issue is incorrect to say the least.
I am as disgusted by twitter as the nextI don't insist that every distro add GNU to their name; that's up to them. But Debian chooses to so at least in their case could we maybe lose the "RMS is teh zealot!!" jokes?
Debian is a great dsitro for promoting Free software and I am pretty sure they made the choice to keep GNU in the name on priniciple. I respect them for it and it's a major reason why Debian remains my distro of choice.
Creationist Textbook Stickers Declared Unconstitutional by CowboyNeal
Why the fuck should we care that you hate twitter?
They may have the Windows version memory under control, but the OSX (Leopard) version of FF 3.0 is still a PIG. Just starting up FF with 6 tabs (1 is Slashdot) and the memory usage is over 160Mb and virtual memory used is over 900Mb.
And did they have to make the OSX skin so darn ugly?
He does raise an interesting question (and I'm in no way bashing Vista):
how do browsers compare on Mac OSX? Does Safari compare similarly to Firefox 3 on Mac OS X?
I wouldn't be surprised if Safari performs better on OS X than on Vista because Apple has put more time into developing the Mac version...
Similarly, I wouldn't be surprised if Firefox is optimized for the larger Windows market. I know that back in the day, on the old PPC macs, Firefox was pretty slow, which was a good reason to use Safari.
This space up for sale.
Linux automatically uses all free memory for disk cache, and is very efficient at it. Instead of hogging memory that could be used by other process, Firefox could exclusively rely on a disk cache and rely on Linux' native and performant disk-cache to handle the in-memory caching of those files.
The best part of this is, if some other process needs the memory, Linux will simply free some memory from the cache, but the files will still be ready on the disk and the over application will still be performing well. Whereas if a 120Meg space is enforced a in-RAM cache, when memory becomes scarce, the system is at risk of paging out piece of the software (pages of code itself instead of pages holding cache) and thus make the whole system less responsive. The GC is just for JavaScript (required by design) and for DOM nodes which end up being circularly referenced (which is unavoidable). You would only need to keep the DOM nodes of the current page. Past pages are freed and don't (usually) keep DOM objects alive. Finally, 120MB is not a lot of RAM. Well, it depends. Notice that Linux is also very often used on kiosk with limited features and on old hardware which may not have huge amounts of resources.
Being able to run within a small memory space is critical for linux. Otherways, there won't be any difference with Vista.
Also a lot of problems are comming from bad Add-ons or even half-assed Browser Plugins. Flash is such a pain in the ass that can momentanily freeze the whole browser session.
Disclaimer : I run Firefox on Linux with in-memory cache disabled and using Gnash plugins instead of Flash (runs in separate process and can have autostart disabled). Adblock+ and Noscript also help avoiding that my browser loads tons of useless shit. And until recently my main desktop was a Pentium-III with 440BX chipset (a machine on which 1GiB of RAM is a rare occurence), but I didn't get any major problem even on recent distros. (Vista on the other hand had to wait)
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Or, the other user is the botnet admin who 0wnz his box.
Flappinbooger isn't my real name
On fairly modern Nokia E65 with Apple Webkit based free "Nokia browser", I did total 1 hour browsing. That is heading to Opera.com for whatever they offer for my OS (it took that much to reach the right download page) and later bought it with Opera Mobile client itself.
Thank God Opera Mobile exists as a choice. That is their 2 years old 8.65 code I bought. It is AGES ahead of anything on that smart phone.
If they charge for their marvellous work, a browser which uses less memory than 8 MB on my S60 device that will , hell it is $20!
That 2 years old code picks the right css from my web pages (all w3c compliant) and shows them like I have setup a special WAP site for phones.
Now waiting for their 9.5 achilles heel.
What bothers me is, they get 5-7 million downloads without advertisement (9.5 Desktop), they help hundreds of millions of possible devices doesn't even have a good WAP browser to access the web, they strictly support standards, they never sold out to MS (imagine Opera quality IE only exists on Win Mobile), they even support Windows 98 and they get THAT comment as return.
Parent is not informative.
Took me all of 10 seconds with google: "Firefox3 Awesomebar disable". And yet, that page is absolutely not what the GP was asking for.To quote:
Some things are more important than resource conservation, such as not screwing the user by needlessly taking away functionality and telling them "you'll get over it".I'd gladly have Firefox 3 with the same footprint as Firefox 2 if that's the price to pay for keeping the old address bar autocomplete functionality in the code.
gumpish--and I, and a lot of people as far as I can tell--want the "classic" (read: FF2) location bar behavior. There is currently no way to get that in FF3. There used to be an about:config setting for it in some of the betas, but they took it out and told those people who liked the old functionality to shut up and deal.The about:config tweaks on the page you linked to will disable the location bar dropdown entirely (heck, it even says as much in the edit--did you even read the page you linked?).
A sibling post suggests the oldbar extension, but that only changes the appearance of the bar and not the behavior.
As I said, there is currently no way to get this functionality in FF3. The closest thing is this extension, but that's not perfect. Really, this bug needs to be fixed.
...what it would be like if Apple came up with a benchmark for web browsers? They'd do some kind of splashy announcement. Geeks would question its relevance in the real world. Soon they'd do some new tests and proclaim that the competition now performs better on those same tests anyway. Eventually, the rabid Apple phanbois will claim that the next release will bury the competition.
Apple. So predictable.
The US free market: two halves of a government-granted duopoly are free to set the market price.
While it is slightly diffferent:
I was an academic researcher working for a large engineering firm where the top level work is classified and they don't want anyone connecting the dots with what you are publishing and the work they are actually trying to do. So while I was free to publish my research, I was not allowed to attached the name of the Company to any of the papers.
that unused resources are wasted resources? If Safari has enough RAM to cache everything, why shouldn't it? Now, run the same test with 512MB RAM and see if memory consumption does the same thing.
That's really what I want to know - not how much it uses, but how willing the browser is to give it back when other processes need it. That'd be useful.
I've been running Safari 3.1.1 all day (about 6 hours non-stop with 462 pages in history) and it's currently using 230.9MB RAM (fairly steady).
Opening up FF3 to check, loading 52 pages then closing down one again... 129.18MB.
Not a totally fair comparison but it's some numbers, nonetheless.
Damnit, can't use mod points in this thread now.
The way I've found to restore sane address-bar behaviour was to do the following:
Install the oldbar extension
about:config browser.urlbar.matchOnlyTyped boolean set to true.
Bingo! No more hyper-annoying address bar.
Moderation Total: -1 Troll, +3 Goat