Notes On Reducing Firefox's Memory Consumption
Skuto writes "At yesterdays linux.conf.au Browser miniconference in Ballarat, Australia, Mozilla engineer Nicholas Nethercote gave a detailed presentation about the history of Firefox's memory consumption. The 37 slides-with-notes explain in gritty detail what caused Firefox 4's memory usage to be higher than expected, how many leaks and accidental memory use bugs were tracked down with Valgrind plugins, as well as the pitfalls of common memory allocation strategies. Current work is now focused on reducing the memory usage of popular add-ons such as AdBlock, GreaseMonkey and Firebug. Required reading for people working on large software projects, or those who missed that Firefox is now one of the most memory-efficient browsers in heavy usage."
I use other browsers for development, but only Firefox to browse, it's the only browser that I feel is actively protecting my privacy.
Any other opinions on that?
The final link actually shows Firefox is one of the most memory inefficient browsers in heavy usage.
I have 8gb of memory on my main computer. I want firefox to use up as much of it as it can to improve my browsing experience. On my netbook, I have 1, I want it to sip as little as it can. There should be an easy way to tell the browser how much memory I want used for certain tasks such as caching and whatnot. Addons should have their own seperate allocation, and each individual addon should be configurable for how much memory it can use.
Free / Open source software is about choices to run things how you want to. With that said, I don't have much room to complain because I've never contributed code to FF.
My main point though, is that screaming "THE RAM USAGE IS TOO HIGH" is not effective. I have a lot of ram, and I want it to be used, just not wasted; and I want more control over it.
In my experience Firefox (minus plugins etc) has leaked/used less memory than Google Chrome.
BUT, when a plugin/page starts leaking lots of memory the difference is with Google Chrome you can close the offending tab/window and the memory is freed up. You don't lose your sessions, don't lose your place on other tabs/windows. You can even reopen the page (esp if the page just slowly uses more and more memory).
In contrast with Firefox you often have to close the entire browser to free up the memory. And IMO that's Firefox's biggest problem when it comes to memory.
It doesn't matter how much better Firefox is on its own at memory management, in practice many people using Firefox are using it because of the plugins (otherwise they'd be using some other browser), and the plugin developers may not be so good at memory management.
i won't be taking TomsHardware's word for it since I run Linux and they didn't even bother testing with it. but it does motivate me to perform my own checks. memory usage and sluggishness were the reasons i ditched firefox for chromium last year.
I'm already up to Firefox 9, so this must be from the ancient past, right?
Or you could, you know, uncheck the "allow acceptable ads" and you will be back to normal...
At yesterdays linux.conf.au Browser miniconference in Ballarat, Australia...
Wait what? Something actually happened in this god forsaken place? Something I.T. related that wasn't IBM related, IBM sponsored, a thought bubble from IBM, an idea from someone who use to work for IBM or taught for IBM or was a student at the IBM approved University courses? linux.conf.au Sponsors.... knew it was too good to be true.
I use FF because it is much more configurable (about:config has entries for more than any other browser), it also has more plug ins, and those plug ins it does have tend to be more effective for just about everything.
I also don't like Google Chrome calling home all the time (uninstalled). I also have IE9 and Opera installed.
Speed issues are moot outside of benchmarks these days (unless you are running IE7 on a netbook). IMO it is pure placebo effect to say one browser is faster than another in regular browsing on a modern computer.
Windows users should investigate the Pale Moon browser over at www.palemoon.org. Firefox optimized and without all the gunk.
Kriston
Firefox is the app that uses the most RAM in my system, has always been, even more that Skyrim under WINE. I tried Chrome for a while, and while I didn't dislike it, I simply didn't want to forfeit my customized environment.
However, I never saw Chrome as using any less RAM. I usually got Firefox with 20+ tabs open (and around 100 in "not loaded in RAM" mode with the new features, think old BarTab) grouped in Panorama groups. In Chrome, because tab space is small, I usually had around 20, and both browsers were consuming 400mb of RAM each. I'd say Firefox uses LESS memory overall.
Thing is, firefox FEELS slow. Try to open Youtube's subscriptions page and you'll lose control of the browser for at least one second.
I can easily see people unable to close their porn when their significant other enters the room. Porn moves the world, thus people would prefer to use Chrome for porn. Thus Chrome's usage rises while Firefox's decreases. If Mozilla makes it more convenient to use Firefox for porn, the browser usage will crush Chrome.
thanks for that! just unchecked it - stupid to have it on by default, isn't it?
Applications that are said to leak memory are really failing to release allocated memory that they requested. The application doesn't have memory to leak at all.
I suggest a better term for this is "memory hoarding."
Citation Needed.
I am sure that many people here would like to know how you equate Ad-blocking with supporting SOPA?
Come on please tell us.
I'd rather be riding my '63 Triumph T120.
Of course it's not stupid. It's the whole reason for the feature in the first place. So that 'acceptable' ads can be shown and used to support the websites you visit. They need the money, you know. The reason it's on by default is that the ads are specially screened for their acceptability, and only ads that are deemed not annoying to users are presented.
Dump the firefox address space to a file and check the proportion that's simply text urls. For a daily user with history set to a typical value like 30 days it's shockingly large - tens to hundreds of megabytes. That's mostly the sqlite url/places db. Simple lz77 on *.sqlite gives a 50% reduction. Dictionary-based schemes would give a 95% reduction; academic work shows all urls can be coded in 4-5 bits. The few extra cycles per lookup are nearly free, and much cheaper than RAM.
There are hundreds of equally dumb design decisions holding back necessary features. Have fun waiting for a competent sandbox.
See page 27 of the PDF. He explains that the allocator, jemalloc, rounds up some allocation requests to easier to handle size. He calls this wastage "slop". Then look at the final point on that page - in order to reduce slop, always allocate a power of two sized block, as those never have slop.
So in order to stop the allocator wasting memory by using up more memory than requested, we're supposed to ask for more memory than we need? That seems to be a facepalm moment. Let's move wastage to where we can't measure it, so that we can't see any wastage any more. The bind moggles.
Also FatPhil on SoylentNews, id 863
"Current work is now focused on reducing the memory usage of popular add-ons..." -- Folks, we need the 64-bit version, and presto. Memory is not an issue these days so get over it. If they meant "memory leaks" rather than "memory usage" then I would understand the concern.
... its just accidental memory usage.
Oh good, I was starting to get worried there.
I'm a good cook. I'm a fantastic eater. - Steven Brust
Each ad is NOT screened before it is whitelisted. The ad companies on the whitelist simply have an agreement with the Adblock Plus author to follow certain guidelines.
I'm sure the New York Times had a "do not infect our users" agreement in place, too...
I get warnings from AVG about every other day warning about FF memory usage looking suspicious. I'm pretty sure I'm not infected with anything. But having say 3-4 tabs open on Win 7 (might be the problem ;-)) uses ~560MB of RAM. Not overly complex sites either, /., coding site, youtube say (not playing a video). That said I rarely hit 80% of RAM used overall so I don't really care how FF uses my RAM other than if more RAM implies slower because more stuff has to go back and forth to the CPU. It could just mean that FF preloads a bunch of stuff so it runs faster though (don't hear a lot of people say it is faster :-)) so tricky.
And one of the big reasons I gave up Desktop Linux.
This signature has Super Cow Powers
Remember when Firefox was the little browser that stood up against the big bad IE6? That was over 7 years ago now, which is sad because there are a lot people online too young to remember the IE/Netscape wars and how a "Phoenix" rose from the ashes of Mozilla/Netscape. Firefox became fat despite removing features such as the status bar, and his "quick fix" diet didn't work well enough. Now Firefox is getting his ass kicked by Chrome daily and IE10 will be coming up soon which will smack him when hes down.
It's time to for the Phoenix to rise again, chop of all the fat from Firefox and become the little browser that could once more.
Except that option is bugged and does not actually work properly.
Besides, it's a matter of trust. ABP is an ad-blocking product that now has a whitelist of ads it allows to be shown. This would not be a problem if it was a user-configurable list that was disabled by default, but that is not the case. The authors have chosen to a) prevent users from editing the list, b) they have enabled the option by default, and c) they have been dishonest about the entire issue from the beginning and continuing to present. That's three strikes, each one ethically unforgivable given the circumstances.
Please read what Eric Bishop (author of Trueblock Plus) has to say on the matter before you dismiss this as a non-issue. Frankly, the PR coming from ABP is disgusting, but what offends me more is the fanatical support they seem to have stirred up in their userbase. Read the description and FAQ on the project homepage (it's short, I promise): https://trueblockplus.org/
is reportedly code named "Honey Badger" because it takes all the memory it wants.
Or did they fix Firefox 4? Just wondering if I should upgrade to the latest FF9, since I'm still running FF 3.6...
Dump the firefox address space to a file and check the proportion that's simply text urls.
You are aware that not all of Firefox's address space is in memory, right? I'm pretty sure that Firefox does not use hundreds of MB of RAM just for storing URLs.
I love Firefox. It still has a few annoyances though. Having to sometimes manually kill the Firefox task to start a new browser window because it's "already open" is my biggest one. But, I still love its customizability over the sheer rigidness of Chrome.
The reason it's on by default is that the ads are specially screened for their acceptability
Blatantly FALSE. This is the list: https://easylist-downloads.adblockplus.org/exceptionrules.txt
Anything matching these simple filters is displayed. (I had to insert a couple of spaces on the last line to get the comment through the spam filter.)
[Adblock Plus 2.0]
! Checksum: apz2+jXNPQ5CLFYHWghn6A
! Expires: 10 days
! Text-based search ads on suche.netzwelt.de
@@||google.com/uds/*$script,subdocument,document,domain=suche.netzwelt.de
@@||suche.netzwelt.de^$elemhide
@@||google.com/aclk^$domain=suche.netzwelt.de,subdocument
@@||googleadservices.com/pagead/aclk^$domain=suche.netzwelt.de,subdocument
||images.netzwelt.de/partner/*
! Static image ads on t3n.de
@@||guruads.de/api/view/*$script,domain=t3n.de
@@||guruads.de/u/b/*$image,domain=t3n.de
! Text ads on Sedo parking domains
@@$sitekey=MFwwDQYJKoZIhvcNAQEBBQADS wAwSAJBANnylWw2vLY4hUn9w06zQKbhKBfvjFU CsdFlb6TdQhxb9RXWXuI4t31c+o8fYOv/s8q1LGPga3DE1L/tHU4LENMCAwEAAQ,image,~image
Here's how Eric Bishop (the guy behind Trueblock Plus) responds:
Q: It's possible to turn the ads off in AdBlock plus, isn't that good enough?
A: No, it's not. Defaults are very powerful things, most people never touch them. People react very differently depending whether a choice is presented as “opt-in” vs “opt-out.” The creator of Adblock Plus knows this, saying that enabling ads by default is “the only way to reach the goals.”
Q: Adblock Plus claims their “goal” is to prevent the loss of advertising revenue to meaningful and important sites that rely on it for survival. Isn't that a good thing?
A: Maybe, but that explanation neatly side-steps the real problem. The problem is that I want to be able to decide for myself which sites are meaningful and important. When Adblock’s “acceptable” ads are enabled, the user is not the one deciding which sites get a pass and which do not, the owners of Adblock Plus are. That is a huge problem. The person that controls this “acceptable” list can sell a spot on the list to the highest bidder, making a profit by violating your trust. While impossible to prove, it seems likely the real motivation behind this feature is greed and has nothing whatsoever to do with any sort of altruistic desire to preserve websites that depend on advertising revenue.
This is the list of current “acceptable” ads, as defined by Adblock Plus. This list includes text advertisements on Sedo parking domains, a giant company that sells domain names, and publishes advertisements on unused domains. This is hardly a poor little website where meaningful content is in danger of disappearing because no one will view their advertisements.
academic work shows all urls can be coded in 4-5 bits
5 bits gives a total of 32 possible URLs, so this is only true for very small Internets...
Or do you mean 4-5 bits per character?
I am TheRaven on Soylent News
I have never understood why memory leaks are so problematic to find. When I was programming in C long ago and had memory leaks, I simply created my own wrappers for malloc and free that stored where in the source code it was being called from, then at the end of the program I could simply check this list and it would display all the memory that had not been released and the corresponding line in the source where malloc was called.
Ever since they copied chrome with the tabs in the titlebar idea it has been slow as balls (referring to program start time not necessarily page loading time). Chrome for the win.
You are aware that not all of Firefox's address space is in memory, right? I'm pretty sure that Firefox does not use hundreds of MB of RAM just for storing URLs.
You may be pretty sure, but on any system with swap disabled (many SSD users) you're also wrong.
At work I have a quad core Q6700 with 4GB of RAM. At home I use an older single core Athlon 64 3500+ with 2GB of RAM. Both machines run Windows XP.
Both machines run FF 3.6 because I keep reading that later versions have worse memory use and UI performance characteristics.
On both machines, I experience two problems with FF memory usage (all figures were reported as "private bytes" by Sysinternals process explorer):
1) Memory usage keeps growing until it reaches a threshold (1.5GB on my home machine) after which FF locks hard with close to 100% CPU use and never recovers. Closing tabs did not bring the memory usage down in a perceptible way.
This used to require daily restarts of FF but lately the problem does not seem to happen that often. A restart can still cut the memory use by half (same tabs courtesy of session restore), which helps with problem #2 below, but it takes much longer to go over 1GB.
2) Periodic "stuttering" where FF will pause for a short period every once in a while with CPU usage spikes approaching 100%. The duration of the pauses seem directly related to the amount of memory that FF uses. That, and the periodic nature of the "hiccups", lead me to believe it is related to garbage collection.
Unfortunately, it makes viewing videos when FF is running (even using an external viewer) impractical, so I have to close FF and start IE8 each time I go to youtube,
There's a bug report that was opened almost 3 years ago (and still unassigned).
The responses that I get are:
* You're using an old version.
True, but according to the comments, bug 490122 is still present in the newer versions (up to v10) and people say that 3.6 is more responsive (especially on older HW) than the newer versions.
* It's the plugins/addons/extensions.
Perhaps, but the reason I use FF at all is because of the extensions and I would expect such an "extension-centric" product to help me figure out which one is misbehaving (for example, by reporting the memory usage of each tab).
Now, reading the article and the slides, I am getting hopeful that these issues are being addressed.
State of the art for a corpus of 1M urls c. 2005 was 5 bits, cf. IBM patent - don't have notes handy. PAQ/PPM + splitting + hacks (unicode to 8-bit etc) gets you most of the way there.
If you don't like Google Chrome calling home all the time- there is a binary distribution of open-source Google Chromium for windows with all that callback crap removed. It's called "SRWare Iron". Well, supposedly removed- since it's binary I cannot verify that. But you can compile chromium from source yourself if you really want to.
http://en.wikipedia.org/wiki/SRWare_Iron
--Coder
Since so many in this thread talked about ancient history, I'll mention one. Does anyone remember the option to browse without images, from back in the day? Sometime around Netscape Navigator 2? Used to be a rather prominent option, but somewhere along the line it was quietly dropped off the menus.
Intellectual Property is a monopolistic, selfish, and defective concept. It is "tyranny over the mind of man"
Virtual memory can be backed by the original file it was loaded from (if the data hasn't been modified) -- this is in fact extremely common with executables.
No need for swap in that case.
Citation Needed.
I am sure that many people here would like to know how you equate Ad-blocking with supporting SOPA?
Come on please tell us.
Perhaps he's Jamie Kellner, in which case the only reason I wouldn't tell him, and everybody who defends him, to go fuck themselves is that they might manage to have an orgasm in the process, and they don't deserve to enjoy themselves.
I don't remember right now the setting, but what finally 'fixed' FF's big memory usage for me was reducing the number of pages from the history that it keeps in memory, in case you hit the 'back' button.
There's a setting that controls this. I think it's browser.sessionhistory.max_entries Search for that.
If you browse image-intensive sites, I saw memory usage get up to 4-6GB before. Now, it never goes over 1GB, with 50 misc tabs open still.
Don't steal. The government hates competition.
Nice list of plugins. Together with the status bar plugin, those are exactly the only plugins I use.
Actually adblock is theft. By using adblock, you support grand theft, burglars and, er, arsonist.
I disabled every data gathering option I could find in Chrome and it still was trying to get at the internet on it's own.
I never saw Iron do any such thing. Also your "analysis" is ancient and out of date, based on Chrome 5 source, Iron now uses Chrome 16 mainline source.
I can't tell if you are trolling or really, truly use an antivirus which warns you about applications that are using your computer's RAM, which is what it's there for. I'd be willing to bet that AVG isn't really the problem, though, it's just that you have it configured to do this.
Pretty much the only thing you should need to manage RAM use is Task Manager.
Then perhaps the proper solution is exposing a memory manager API to let applications mark some blocks of RAM as "purgeable" so that they can be discarded (and rebuilt later) if the operating system experiences memory pressure. Mac OS has been doing this since 1984. For example, most cached data (DOM of recently closed tabs and windows, as well as decompressed forms of off-screen images) could probably be marked purgeable.
I wasn't saying AVG was the problem. I'm aware of FF RAM usage because AVG tells me about it periodically. I'm not that much of a nerd that I check my RAM usage all the time even when my computer isn't behaving poorly. AVG IS suggesting that something might be wrong with the app because it is using so much RAM though. Not sure if it is a generic thing like anything using > X MB of RAM gets flagged, or if it is RAM usage growth over time that triggers the warning or what. Don't really care because I ignore it since I'm no where near my RAM limit.
How do you figure that?
You cherry picked your data. Here are all of the Firefox rankings:
FYI, the second "f" is lowercase: the name is Firefox, not "FireFox". While we are on the topic the official abbreviation is FX, despite most people using FF.
There's a hidden treasure in Python 3.x: __prepare__()
The user experience on Mac is incredibly bad. Firefox is borderline useless. Cut and paste operations, scrolling after you open a new window, and many other actions are frustratingly laggy. None of the other browsers, Safari, Chromium, or Opera have this problem. It is painful to use Firefox on a Mac.
...plus one Slashdot tab wouldn't take 2 GB of ram here and bog one core at 100%...
I couldn't live without all the add-ons. (Especially as a web developer.)
And I know that when uncompressed, all that stuff has to take memory. Plus I actually find it useful to keep some closed tabs in memory, since I often use that feature...
But still, WTF?
Have fun waiting for a competent sandbox.
Bullshit. I'm calling you out on this.
Its called letting the OS cache disk access
We appear to disagree on the meaning of "cache". It's not just the browser's "cache" of downloaded objects that would be marked purgeable. Are you claiming that cached DOMs, used for back and forward buttons, recently closed tabs, and recently closed windows, should all be written to the HDD? Or that cached decompressed images should be written to the HDD as BMP files?
You cherry picked whether to rank Firefox relative to the best or the worst in those datum. Without knowing how many browsers were tested, that data is almost completely useless.
Faster, & better = Waterfox -> http://waterfoxproj.sourceforge.net/
* Enjoy, for those of you using Windows in 64-bit builds that is... & that want a 64-bit version of this program that's highly compiler optimized!
APK
P.S.=> I've been using it since version 7.x, & it's excellent (surprisingly so) & with AdBlock + NoScript as addons loaded (that's it though)...
... apk
Actually, if you're on Windows, just install RAMDisk free edition (http://memory.dataram.com/products-and-services/software/ramdisk) and make an, say "e:" drive with it (make it, say 150mb large), then proceed into Firefox "about:config" zone and make/edit a STRING key called "browser.cache.disk.parent_directory" inside this put: "e:\FirefoxCache". Last step is new INTEGER key called "browser.cache.disk.capacity" inside this put "1048576" and there you go; clean cache on reboot and fast - fast - fast cache with no disk usage. The power of configuration :)
My point was that if you can't tell your computer is slow, and/or you need AVG to tell you that it might be slow due to excessive RAM use by Firefox, something's wrong.
What's the point of a warning that "something might be wrong" when it should be plainly evident by the time something is wrong?
Most likely AVG is just being annoying and needs to be configured not to do that.
Yep I agree. It should be ram usage + low ram not just RAM usage. All I saw for settings was to completely disable that type of warning not exclude certain apps. So I'll live with it it is just a popup balloon every few days so not a big deal.
He admits it here -> http://slashdot.org/comments.pl?sid=2610194&cid=38641004 perhaps law enforcement ought to be advised? Last I knew, stalking/harassing others online, especially via surreptitious means is against the law.
He admits it here -> http://slashdot.org/comments.pl?sid=2610194&cid=38641004 perhaps law enforcement ought to be advised? Last I knew, stalking/harassing others online, especially via surreptitious means is against the law.
He admits it here -> http://slashdot.org/comments.pl?sid=2610194&cid=38641004 perhaps law enforcement ought to be advised? Last I knew, stalking/harassing others online, especially via surreptitious means is against the law.
http://slashdot.org/comments.pl?sid=2610052&cid=38754490
APK
http://slashdot.org/comments.pl?sid=2610052&cid=38754490
APK
http://slashdot.org/comments.pl?sid=2610052&cid=38754490
APK
Well, I'd disable them entirely. I don't need an antivirus telling me my computer might be slow. I can tell.
The first thing I do if my computer is being noticeably slow is hit Ctrl-Shift-Esc to open Task Manager. Either some runaway process is pinning one core at 100%, or something grabbed way more memory than it should have.
Either way, it's a matter of either waiting for the process to finish what it's doing, or killing it.