Slashdot Mirror
Firefox Memory Leak is a Feature
SenseOfHumor writes "The Firefox memory leak is not a bug. It's a feature! The 'feature' is how the pages are cached in a tabbed environment." From the article: "To improve performance when navigating (studies show that 39% of all page navigations are renavigations to pages visited less than 10 pages ago, usually using the back button), Firefox 1.5 implements a Back-Forward cache that retains the rendered document for the last five session history entries for each tab. This is a lot of data. If you have a lot of tabs, Firefox's memory usage can climb dramatically. It's a trade-off. What you get out of it is faster performance as you navigate the web."
about:config then search for browser.sessionhistory.max_total_viewers and set it to 0. This will be 0 pages in the cache per tab. You will get a reload slow down since FF will be going out to the web. You can manually set this to 2 or whatever you want. By default FF will cache upto 8 pages per tab with 1 gig of memory or more.
It's not the most ideal solution, but I can drag and drop the favicon (the icon in between the Home button and address in the address bar, with default toolbar settings) to my tabbar to effectively get a duplicate of the current page. (Tab Mix Plus might be the cause of this feature). I don't have a Firefox that isn't loaded with Tab Mix Plus around, but I don't think you need the extension to do this.
Tab Mix Plus also has an option to always open the current page in a new tab.
"Why can't I hit ctrl-T and get a new tab with the same page I'm currently on, then hit reply and anything I want to quote I can just switch tabs instead of screwing around with back/forward and scrolling."
/. equivalent of the Farkit extension.
I just replied to your post using Firefox. I middle clicked on "Reply to This," which brought up your post appearing by itself in its own tab. I just copied and pasted some of your post into my reply, hit "Submit" and went on my merry way. Isn't that simple enough? Although I would like to see a
Bill Clinton: Pimp we can believe in. - The Shirt!!!
At least in the OS X version, commandclicking (and probably middleclicking as well, but I haven't got a mouse connected) the back button solves this problem nicely. I would guess that middleclicking the back button works under other operating systems as well. So just click "reply" first, then middleclick the back button. :)
I have a really elegant proof for Fermat's last theorem. If this sig was only a bit longer...
FTFA: "...For those who remain concerned, here's how the feature works. Firefox has a preference browser.sessionhistory.max_total_viewers which by default is set to -1."......If you set this preference to another value, e.g. 25, 25 pages will be cached for every tab. You can set it to 0 to disable the feature, but your page load performance will suffer.
--
From the Tips & Tricks page:
// Specify the amount of memory cache: // -1 = determine dynamically (default), 0 = none, n = memory capacity in kilobytes
// Disable memory cache:
Specify the memory cache usage
Normally, Firefox determines the memory cache usage dynamically based on the amount of available memory. To specify a specific amount of memory cache, add the following code to your user.js file:
user_pref("browser.cache.memory.capacity", 4096);
To disable the memory cache completely, add the following code:
user_pref("browser.cache.memory.enable", false);
It's just Crap.
Ben was mistaken, it's cached globally.
See this comment by Boriz Zbarsky:
and this comment by David Baron:
(Boris and David are back-end developers; they have much more working knowledge of this than Ben does.)
Also, there are actual memory leaks in Firefox. See this weblog post about progress on that. However, as that weblog post says as well, most excessive memory usage that people are seeing is entirely due to faulty extensions.
Looks like you need the duplicate tab extension...p ?id=28&application=firefox
https://addons.mozilla.org/extensions/moreinfo.ph
Unexpect the expected!
I found that pressing Ctrl+Z after Ctrl+T brings up the URL from the last tab you were on. Now you just need to press Enter.
This extension, tab mix plus, has that functionality and a lot more built in (Duplicate Tab). Not sure if you can make it a keyboard shortcut though. Very handy tool for me.
I'd be curious to know if I'm the only person that doesn't have that preference. I've got a browser.sessionhistory.max_entries, but that's it.
See Firefox is the most unstable program in common use.
The Firefox CPU hogging bug makes a computer unusable until all Firefox windows and tabs are closed. Basically, Firefox uses first maybe 10%, then maybe 20% of the CPU, and, as Firefox windows and tabs are opened and closed, continues taking more of the CPU time until Firefox is closed. This CPU usage is with NO Firefox activity, or any activity of any program.
This bug is more than 3 years old. It is extremely difficult to characterize; no one has succeeded yet. Here are some clues:
Somehow Thunderbird and Mozilla share this bug. Sometimes when Firefox is taking say, 94% of the CPU, and Firefox is closed completely, Thunderbird or Mozilla will begin using a lot of CPU time. Very weird, but it often happens.
Firefox 1.5.0.1 is much worse than 1.5, which is worse than earlier versions. This suggests that there is some resource in Firefox that is being more overused as features are added.
The CPU hogging bug continues unchanged when Firefox 1.5.0.1 is installed with a clean profile and no extensions.
Too many mouse clicks too closely spaced will often increase Firefox's CPU usage, or sometimes cause it to crash.
--
Before, Saddam got Iraq oil profits & paid part to kill Iraqis. Now a few Americans share Iraq oil profits, & U.S. citizens pay to kill Iraqis. Improvement?
firefox's memory usage has always been a thorn in my side. I tend to average around 20 to 25 tabs open, usually while I'm running other ram hungry applications. Firefox generally was eating up about 200-250 megs of ram on my machine (and I've seen it go as high as 600 megs). After changing the browser.sessionhistory.max_total_viewers to 0 and running "top" firefox seems to be using about 46 megs of ram right now. It also doesn't feel particularly slower than it did before. I have a feeling that the benefit of caching so much was actually having a negative return after a certain point because the machine was so starved for ram.
On a side note, if anyone is like me and looks in about:config for browser.sessionhistory.max_total_viewers and doesn't see it, you have to actually add the line. Right click and choose "new" then type in "browser.sessionhistory.max_total_viewers" and then 0 (or whatever you like).
Famous Last Words: "hmm...wikipedia says it's edible"
I regularly use middle-click to open a link in a new tab in the first place. However, the Mac OS X version of Mozilla lacks this option, expecting me to configure my mouse to do a command-click on middle-click instead to get the same functionality I enjoy on Linux.
Usually the only time I use a browser under Windows is for Windows Update.
And just testing right now, middle-clicking on the Back button does nothing for me under Linux. It has a visual reaction but otherwise does nothing else. Maybe it is another one of those Firefox features not found in Mozilla?
Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
For the sake of knowledge, you *don't* need the extension.
- Opera has none of these problems. So, the quote from the Mozillazine blog
shown below, although it is typical, is not supported by the
facts.
- Whatever causes the CPU hogging bug is definitely associated with extreme
memory use. No doubt there are leaks, but this is not a leak, since it is not
necessarily associated with greater use of Firefox.
- Users often report that just leaving Firefox open overnight causes CPU
hogging and extreme memory use.
- The problems are the same in Mozilla browser.
- It's good to test Firefox with a laptop in a quiet environment. When you
hear the laptop fan begin to run while there is no activity, you know Firefox
has begun to suck CPU cycles.
- Putting a computer into standby or hibernation often makes the CPU hogging
bug much worse. That's why Firefox users sometimes just leave their computers
on.
- When a computer takes a long, long time to start from standby, you know Firefox
is taking CPU cycles. What about coming out of standby makes Firefox unstable? No
other program has that problem.
Quote from the blog linked in this Slashdot story About the Firefox "memory leak": "A lot of people complain about the Firefox "memory leak(s)". All versions of Firefox no doubt leak memory - it is a common problem with software this complicated."No other program in common use is so buggy. The problems in Firefox are not "common".
Another quote from the linked Mozillazine blog: "What I think many people are talking about however with Firefox 1.5 is not really a memory leak at all. It is in fact a feature."
That's not what the technical magazines, newsletters, web sites entirely devoted to Firefox problems, and even the mainstream media say. They say it is a serious problem.
Mozilla developers have been denying that there is a serious problem for more than 3 years. It seems that it would be less work to fix the problem than to undertake a cottage industry of trying to convince people they aren't having problems. Mozilla developers have been impeding characterization by marking Bugzilla bug reports of these problems invalid.
However, it is clear that it would take a serious scientific investigation; this is not an easy bug to characterize.
The Clone Window extension.
Look it up, dingus. There's no reason that every web browser should behave exactly like IE out of the box. That's what the extension feature is for. =)
Firefox crashes when two browser windows are making synchronous XMLHttpRequests. I have experienced this under Linux - I have no idea whether it is the same under Windows. Basically under Lunux all Firefox windows are running in the same thread utilizing a scheme of cooperative multitasking.
So far so good. The bug appears when two separate Firefox windows are making periodic synchronous XMLHttpRequest-s. When such a potentially lengthy task has to be executed synchronously, Firefox creates a new "nested" event queue. If two (or more) browser windows are doing it at the same time, new event queues are created all the time and eventually (within 5 minutes) the application core-dumps.
I found this by recompiliging Firefox with debug information and debugging it. Even if my interpretation of what happens is not completely correct, the fact remains - a simple JavaScript can crash Firefox causing all open browse windows to be closed.
The solution is to always use asynchronous XMLHttpRequest (which is a better practice anyway) and to hope that the same problem doesn't appear in other places. Still, it is troublesome.
That's one way, but you'd have to somehow instruct the compiler to use that for every pointer dereference. The easier method is to go in and change the values during compaction. Compaction is also known as stop-and-copy; it starts with a live set, everything you can reference from the stack, then copies over only the live objects while modifiying every pointer that uses it. It's messy but it works. Allocation is dead simple and fast. There's no fragmentation. And the runtime is limited by the live set rather than the heap size. There is a huge downside, however.
I wouldn't recommend mixing anything resembling C pointer maths with compaction, since its incredibly difficult to tell what's a pointer and what isn't (in fact, without modifying the compiler, it can't be done in C or C++). For this reason, the Boehm collector (a collector that replaces new and delete) goes for the Mark-and-Sweep method instead of compaction. Because you dont move objects, you don't have to worry about figuring pointers. Boehm's collector is also called conservative, not only because it doesn't modify live objects, but also in that it treats any data on the stack or in the heap as a potential pointer. If the data points inside the heap, the object containing that address is marked. This can lead to false positives on occasion, but there's no helping that without any support from the compiler (again contradicting the grandparent). The good news is that a false positive isn't going to cause direct harm in mark and sweep. All that happens is that space that could be used isn't; Boehm claims this is irrelevant in today's operating systems with virtual memory, although I doubt you'd see an entire page's worth of false positives. Certainly, I can't do any better than him.
In language R&D labs where people are paid quite well to think hard and long about things, they tend to use both approaches in what's called a "generational" collector. Young objects can be copied or collected as needed, while older objects are mark/swept away as needed. This works because old objects much more likely to stay than new ones. Last I knew, both Java and C# use generational techniques, because it makes sense in most nearly every case. However, as I described above, C++ doesn't have that, and even those libraries that replace new and delete have conventions and costs associated with it. I certainly wouldn't try to take Boehm and pidgeonhole it into Mozilla. And even if you did, it still wouldn't solve the compaction problem. All you can do is hope the virtual memory manager is doing it's job well. Even though the application and garbage collector is more likely to know what's useful than the VM manager.
I Browse at +4 Flamebait
Open Source Sysadmin
Firefox 1.5.0.1 under windoze, middle clicking the back button brings up the page in a new tab, and I dont have any tab related extensions installed.
Come as you are, do what you must, be who you will.
Simply put, Opera is a better browser ;-)
.. paranoid crackpot leftover from the days of Amiga.
1. set browser.sessionhistory.max_total_viewers to "0"
2. It does (try opening a huge Fark photohop thread, huge as in multiple hundreds of pictures, see Firefox ramp up to 600 or 700Mb ram consumption, close the fark tab, see firefox' ram usage drop dramatically to regular ram usage levels)
"The way we can tell it's C# instead of Haskell is because it's nine lines instead of two." -- wadler
It doesn't. (or Process Explorer lies when it tells me that Opera is using 160Mb of ram after some surfing, which is roughtly what Firefox also maxes at)
"The way we can tell it's C# instead of Haskell is because it's nine lines instead of two." -- wadler
I wouldn't know about C, but this statement is utterly false as applied to C++.
No, its not false, its true.
Replacing the default new and delete routines is perhaps not for the inexperienced C++ programmer, but to say that there's an complete lack of compiler support is simply wrong.
And? What do you mean with compiler support for heap compacting (or GC)?
Q: What has replacing new and delete with your own implementations to do with garbage collection?
A: Nothing
Q: How would new and delete of class A be able to compact a heap by moving allocated instances of class B down?
A: Difficult!
Q: So if you now add a class C you like to rewrite A::new and B::delete to also cope with class C instances?
A: I assume you understand that EVERY delete of EVERY class needs to know EVERY other class to be able to compact the heap, yes?
It is true that out-of-the-box C++ does not have a compacting garbage collecter, but one can certainly be written (and used, of course) with any conformant compiler.
Indeed, but not by merly only by replacing operator new and delete.
Existing C++ garbage collectors are very limited to more or less conservative garbage collecting. See e.g. Boehms c++ / C garbage collector.
And, the mere point of garbage collecting if you want to start nitpicking is: you don't ever call delete.
angel'o'sphere
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
An operating systems class might help you understand why memory usage meters are completely unresponsive in the down direction.
;).
;).
See here's what happens:
Firefox allocates memory for a rendered page. You've got 20MB allocated already, all in 3 chunks. None have enough room for the single large allocation it needs so the OS sets aside a new chunk of memory for the firefox process.
Now it's using say 28MB of memory. And only 22MB of that is used. Well, it does a couple more allocations, some fairly permanent ones, and these get put in the newest block of memory.
Then you close the tab. Firefox frees the associated memory. The OS changes it tables around for that block to indicate so. But it still has some stuff in that block. So guess what? Firefox' memory usage remains exactly the same.
The solution? Use a GC system. Some Garbage Collectors (most) actually move objects to condense them in memory. This is one of the things that makes garbage collections noticeable if a lot has happened since the last one (it's gotta move a lot of RAM and change a bunch of links to said RAM). It becomes especially bad when you move into swap space
The downside? While GC advocates will often amaze you with the fact that malloc is not an atomic operation (it has a lot of work to allocate, more or less depending on the current situation of your memory chunks and the free memory on the system), malloc is still not nearly as costly as a garbage collection cycle. And, free is atomic (at least, TMK all a good implementation does is remove something from a data structure, unless it's the last part in which case it also needs to mark that memory as free).
So, you see, no matter how few memory leaks firefox has, it still won't drop in RAM usage every time you click close.
If you want to prove memory leaks in firefox you can. Get yourself a memory debugger (such as valgrind) and run firefox under it. Now, I'll warn you that this is harder than it sounds:
1.) Memory debuggers are about 100x to 1000x slower than your machine natively.
2.) Firefox is a script, not a binary, it sets up a bunch of stuff for the binary to run.
3.) Everything you see on the memory debugger is not necessarily a leak. Some of the leaks aren't even really leaks (it's generally no big deal to leak when you're exitting because the kernel cleans that up for you).
4.) To get any useful information on the leaks (other than size) you'll need to have compiled with debug symbols and you'll need to have the source code.
Go ahead, post your list of firefox memory leaks. Then post your list of IE memory leaks. I bet both have some, but neither has anything major. And I bet it takes you a week to find them
Yep, type about:config in the address bar, then change browser.chrome.image_icons.max_size to 0.
Type "about:config" (no quotes) into your address bar, then scroll down to the browser.sessionhistory.max_total_viewers setting, double-click on it, then change the number to 0 and hit OK. Any sort of Firefox setting like this is found in about:config.
Want Slashdot headlines on your site? Try SlashHead
The article has been corrected. Note that the maximum number of cached pages, regardless of the number of tabs, defaults to 8, and that's only if you have at least 1 GB of RAM. RTFSC:
s history/src/nsSHistory.cpp#161
http://lxr.mozilla.org/seamonkey/source/docshell/
If you're unhappy with the memory usage with 50 tabs open, I advise the following workaround:
DON'T DO THAT.
There's no failure quite as dissatisfying as a complete and total solution to the wrong problem.
Firefox 1.5 implements a Back-Forward cache that retains the rendered document for the last five session history entries for each tab. This is a lot of data. If you have a lot of tabs, Firefox's memory usage can climb dramatically. It's a trade-off. What you get out of it is faster performance as you navigate the web.
The only problem is there were bugs filed for memory leaks long before Firefox 1.5 and the Back-Forward cache were implemented. Maybe this feature does contribute to Firefox's large memory footprint, but to say that this feature is the only reason and that there are no leaks is simply false.
And a detailed explanation of the feature and it's values can be found here.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Using a lot of memory is not a memory *leak*. A memory leak is where memory is allocated but the pointer to it is lost so the memory doesn't get freed.
Pointer arithmetic is irrelevant here, because it's undefined behaviour to move a pointer to outside the object through pointer arithmetic (exception: The one-past-end pointer, but that can be easily resolved by just allocating an extra byte at the end).
Now pointers per se are relevant, because they are usually implemented as direct address of the object it points to. There is nothing in the C++ standard which forces this implementation, but I guess you would upset most low-level developers (e.g. OS or embedded) if you used another implementation. Now the implementation as direct address is problematic because there's no way the compiler could know what is a pointer and what isn't (you could have a normal integer which just happens to match the address of an allocated block; changing that integer on moving the block would of course be fatal).
However, a standard conforming pointer implementation could also consist of an index into an address table and an offset relative to that address (after all, the x86 protected mode segmented address is exactly that, just at processor level). Since now each access would go through that address table, moving blocks of memory around would be quite easy: You only would have to change that one entry (Ok, in multithreaded programs it would be more complicated because another thread could currently use the address; probably one would have to scan the stored register values of all threads for values matching the address range of the block to move, and don't move the block if it is found). The entries of the table would probably have to be GCed (without compacting, of course).
Of course that would slow down all memory use, even for programs where this wouldn't have a benefit, so it's unlikely to ever get implemented.
[Manual implementation through handle]
Not every part of a program needs the same performance. Indeed, in the high-level parts of your code it is often viewed as error to use raw pointers at all. Also note that if the handle is implemented the same way as the hypothetical pointer impementation above, but without the offset part (because in high-level code you usually won't use pointer arithmetics), and implement the index into the table just as normal pointer to the table entry, the overhead is just an extra indirection (basically you use **p instead of *p), which for high-level code isn't much of a performance hit. Of course the compaction pass itself costs performance, but you wouldn't do that if you wouldn't consider it worth that performance hit (after all, since you manually implement it, you have complete control over when this compaction pass is done). For freeing the table entries, you have the choice of using any memory management method you want, from explicitly freeing by function call through reference counting up to garbage collection. Of course for all objects managed this way, you should make sure that you can cheaply relocate them.
The Tao of math: The numbers you can count are not the real numbers.
And guess what, Opera can be extended in many different ways:
http://virtuelvis.com/archives/2005/01/opera-and-f irefox-extensions f irefox-extensions-ii
http://virtuelvis.com/archives/2005/09/opera-and-
Clever signature text goes here.
I was not necessarily talking about Firefox, just addressing (hah) a misconception in the parent post: fragmentation is not related to garbage collection. I didn't want to get into a lengthy discussion of virtual addressing, page faults, etc. as well ;)
But you're correct. Thanks to the rise of "smart" binned allocators like dlmalloc, fragmentation is no longer the huge concern that it used to be with (for example) the basic Win32 heap API. Modern allocators are now reasonably smart about reusing best-fit blocks and not leaving tiny holes scattered in your VM pages.
Anyway, fragmentation is certainly not the primary cause of Firefox's high memory use. I am merely pointing out that my original post's parent had a poor conception of how fragmentation could be dealt with in.
A handle can be defined as a double pointer, and the locking operation could just be dereferencing it (assuming a single-threaded model of operation). I was speaking metaphorically about notification, not describing an actual event model ;)
The technique is still very useful if you have a program that follows one of a few very specific allocation patterns. For example, consider a program that allocates many small blocks at once and then frees a large (but non-consecutive) percentage. The ability to compact the heap lets you regain all the space you lose when your free holes are smaller than your VM page size.
Will most applications have this problem? Probably not. But the ones that do have it, tend to have it in spades. A handle-based allocation sytem can be a real win in those cases.
At least, it has been according to the unofficial 1.5 changelog. The list of Mac-specific bugs fixed includes: