Firefox Memory Leak is a Feature

SenseOfHumor writes "The Firefox memory leak is not a bug. It's a feature! The 'feature' is how the pages are cached in a tabbed environment." From the article: "To improve performance when navigating (studies show that 39% of all page navigations are renavigations to pages visited less than 10 pages ago, usually using the back button), Firefox 1.5 implements a Back-Forward cache that retains the rendered document for the last five session history entries for each tab. This is a lot of data. If you have a lot of tabs, Firefox's memory usage can climb dramatically. It's a trade-off. What you get out of it is faster performance as you navigate the web."

Comments

Total cached page limit.

[Short Circuit]

So there's a way to limit the number of cached pages per tab, but no way to limit the total number of cached pages, for those of us who have fifteen tabs open?

Whoops!

Re:Total cached page limit.

[bpd1069]

FTFA: "...For those who remain concerned, here's how the feature works. Firefox has a preference browser.sessionhistory.max_total_viewers which by default is set to -1."......If you set this preference to another value, e.g. 25, 25 pages will be cached for every tab. You can set it to 0 to disable the feature, but your page load performance will suffer.

Re:Total cached page limit.

[SmartSsa]

From the Tips & Tricks page:

Specify the memory cache usage

        Normally, Firefox determines the memory cache usage dynamically based on the amount of available memory. To specify a specific amount of memory cache, add the following code to your user.js file: // Specify the amount of memory cache: // -1 = determine dynamically (default), 0 = none, n = memory capacity in kilobytes
        user_pref("browser.cache.memory.capacity", 4096);

        To disable the memory cache completely, add the following code: // Disable memory cache:
        user_pref("browser.cache.memory.enable", false);

Re:Total cached page limit.

[masklinn]

1. set browser.sessionhistory.max_total_viewers to "0"
2. It does (try opening a huge Fark photohop thread, huge as in multiple hundreds of pictures, see Firefox ramp up to 600 or 700Mb ram consumption, close the fark tab, see firefox' ram usage drop dramatically to regular ram usage levels)

Re:Total cached page limit.

[Anonymous Coward]

An operating systems class might help you understand why memory usage meters are completely unresponsive in the down direction.

See here's what happens:
Firefox allocates memory for a rendered page. You've got 20MB allocated already, all in 3 chunks. None have enough room for the single large allocation it needs so the OS sets aside a new chunk of memory for the firefox process.
Now it's using say 28MB of memory. And only 22MB of that is used. Well, it does a couple more allocations, some fairly permanent ones, and these get put in the newest block of memory.
Then you close the tab. Firefox frees the associated memory. The OS changes it tables around for that block to indicate so. But it still has some stuff in that block. So guess what? Firefox' memory usage remains exactly the same.

The solution? Use a GC system. Some Garbage Collectors (most) actually move objects to condense them in memory. This is one of the things that makes garbage collections noticeable if a lot has happened since the last one (it's gotta move a lot of RAM and change a bunch of links to said RAM). It becomes especially bad when you move into swap space ;).
The downside? While GC advocates will often amaze you with the fact that malloc is not an atomic operation (it has a lot of work to allocate, more or less depending on the current situation of your memory chunks and the free memory on the system), malloc is still not nearly as costly as a garbage collection cycle. And, free is atomic (at least, TMK all a good implementation does is remove something from a data structure, unless it's the last part in which case it also needs to mark that memory as free).

So, you see, no matter how few memory leaks firefox has, it still won't drop in RAM usage every time you click close.
If you want to prove memory leaks in firefox you can. Get yourself a memory debugger (such as valgrind) and run firefox under it. Now, I'll warn you that this is harder than it sounds:
1.) Memory debuggers are about 100x to 1000x slower than your machine natively.
2.) Firefox is a script, not a binary, it sets up a bunch of stuff for the binary to run.
3.) Everything you see on the memory debugger is not necessarily a leak. Some of the leaks aren't even really leaks (it's generally no big deal to leak when you're exitting because the kernel cleans that up for you).
4.) To get any useful information on the leaks (other than size) you'll need to have compiled with debug symbols and you'll need to have the source code.

Go ahead, post your list of firefox memory leaks. Then post your list of IE memory leaks. I bet both have some, but neither has anything major. And I bet it takes you a week to find them ;).

Re:Total cached page limit.

[afidel]

And a detailed explanation of the feature and it's values can be found here.

Re:Total cached page limit.

[Edgewize]

You're confusing garbage collection with heap compaction.

People have written garbage collectors for C++, and they work just fine. But they do not help with fragmentation, which is the problem you're describing. That requires a heap-compacting allocator (aka a "handle" allocator). Many languages with garbage collection also use a heap-compacting allocator. C++ does not, because of a low-level language "feature": pointers, and specifically, pointer arithmetic.

If an object moves in memory, then people have to be notified that it has moved, or they won't know where to access it. Languages like Java handle this behind-the-scenes; the system library tracks objects for you, and your program never knows (or cares) whre an individual object is.

C++ allows direct access to system memory, and it tells you precisely where your objects are located. Programmers are then free to do all kinds of things like compute distance to other objects, or convert the location to a number and do arbitrary math operations on it.

When an object moves, anything that refers to it needs to be updated. Well, good luck figuring that out in a language with pointer arithmetic! The system would need to magically determine whether or not a numeric value was actually a memory locations. And what if a program computed the distance between two objects, and later on used that distance to get from one object to the other? The system has no idea of what can be safely moved, and what has to stay put. So nothing can ever be moved.

There are workarounds of course -- if you write a program with heap-compaction in mind, then you can use a "handle" system, where every object has an ID. You remember the ID, and to access the object, you ask the system for a temporary memory location. And as soon as you're done, you "forget" the memory location and let the system shuffle things around in memory. The next time you give that ID to the system, you might get back a different memory location, but you were already expecting that so your program doesn't mind.

But handle allocation is slower, less efficient, and more annoying to use than a traditional fixed-location allocator. You have to start your project with it in mind; retrofitting existing code to use a handle allocator is a giant timesink and prone to conversion errors. And if you don't mind the loss of performance due to using a handle allocator, why are you using C++ in the first place?

What a small world

[Rude Turnip]

Did the Mozilla Foundation hire the same PR firm that Microsoft uses?

In other news...

[GillBates0]

...scientists have determined that the human appendix is not an evolutionary anomaly as previously thought, but an intelligent design feature aimed at keeping the humans guessing as to it's actual function.

And in totally unrelated news, the Mozilla foundation recently announced that their flagship browser Firefox shall soon be renamed to Bigfoot, to reflect the software's large memory footprint.

More breaking news on these topics at 11.

NOT per tab

[savala]

Ben was mistaken, it's cached globally.

See this comment by Boriz Zbarsky:

Ben, those numbers are NOT per tab. The bfcache is global; there are never more than 8 pages total in bfcache (and you need to have 1GB of RAM for this to happen). Most users have 3 or 5 pages in bfcache at any given time.

and this comment by David Baron:

The point of bug 292965 was that the pref should be global, not per-tab. Is that not working correctly?

(Boris and David are back-end developers; they have much more working knowledge of this than Ben does.)

Also, there are actual memory leaks in Firefox. See this weblog post about progress on that. However, as that weblog post says as well, most excessive memory usage that people are seeing is entirely due to faulty extensions.

Re:My pet peeve!

[Alystair]

I found that pressing Ctrl+Z after Ctrl+T brings up the URL from the last tab you were on. Now you just need to press Enter.

What I'd like Mozilla devs to do

[A beautiful mind]

Let them release 2.0, but then try to focus on:

• fixing the practically fixable bugs (not the design decisions)
• making code performance improvements (faster, with less memory!)
• security auditing

...for a half year or a year. I don't need new features, I'm currently happy with the ones I have and I'd prefer the current features working securely, in a speedy fashion and mostly without bugs. This time period would also give enough time for extensions to mature more.

Before someone jumps at my throat, it's just a description what I'd like to see, but of course its all up to the developers, they decide what to code and do with their time. It is just simple user feedback.

Firefox is the most unstable program in common use

[Futurepower(R)]

See Firefox is the most unstable program in common use.

The Firefox CPU hogging bug makes a computer unusable until all Firefox windows and tabs are closed. Basically, Firefox uses first maybe 10%, then maybe 20% of the CPU, and, as Firefox windows and tabs are opened and closed, continues taking more of the CPU time until Firefox is closed. This CPU usage is with NO Firefox activity, or any activity of any program.

This bug is more than 3 years old. It is extremely difficult to characterize; no one has succeeded yet. Here are some clues:

Somehow Thunderbird and Mozilla share this bug. Sometimes when Firefox is taking say, 94% of the CPU, and Firefox is closed completely, Thunderbird or Mozilla will begin using a lot of CPU time. Very weird, but it often happens.

Firefox 1.5.0.1 is much worse than 1.5, which is worse than earlier versions. This suggests that there is some resource in Firefox that is being more overused as features are added.

The CPU hogging bug continues unchanged when Firefox 1.5.0.1 is installed with a clean profile and no extensions.

Too many mouse clicks too closely spaced will often increase Firefox's CPU usage, or sometimes cause it to crash.

--
Before, Saddam got Iraq oil profits & paid part to kill Iraqis. Now a few Americans share Iraq oil profits, & U.S. citizens pay to kill Iraqis. Improvement?

releasing memory

[DreadSpoon]

The answer to that is pretty simple:

The heap, where dynamic allocations occur, is only allowed to grow or to be truncated. An application cannot release memory in the middle of the heap without also releasing the memory at the end of the heap.

So let's say Firefox makes 10 one-page allocations, and frees the first 9. The memory layout might look something like:
XXXXXXXXXU (X- unused, U- used)

Those 9 pages worth of memory aren't being used, but it's impossible to release them back to the OS.

Thankfully, there is some good news: when Firefox needs to allocate more memory, it can and will just reuse those 9 unused pages instead of allocating more memory from the OS and growing the heap.

The best solution to this problem is to use a compacting garbage collector. Which is something that Java and C# and other higher-level langauges can easily make use of (and many do use them), but which C and C++ can't really make use of given the complete lack of compiler support. That's one reason why a Java or C# app can actually out-perform a similar C/C++ app, especially with a good native-code compiler and an library implementation with a modern GC.

Re:releasing memory

[Profound]

>> That's one reason why a Java or C# app can actually out-perform a similar C/C++ app

Maybe in theory, but in practice 99.9% of the world uses C++ browsers because the Java ones suck.

Don't bug me

[joeytsai]

I think this submission is confusing two points. First of all, is this really a memory leak? A program that uses a lot of memory is not necessarily a leaking program. A memory leak is a programmatic error where memory is allocated but never freed, even when there's no way to use that object again. As the program continues to allocate memory, the heap size of the process increases until eventually the OS terminates the process (eg., the OOMKiller). Actually, many applications you normally use leak memory - but as long as they don't waste a ridiculous amount of memory most people don't care, especially since most process lifetimes are relatively short (compared to a daemon process like apache), and after termination the OS reclaims all the program's memory, leaked or not.

What is being described here sounds much more like a cache of recent pages, which in my opinion is perfectly sane for a browser. Sure, maybe the cache is a bit overzealous, but even if that's the case, just disable it - worse case scenario, you edit the source. But otherwise, this is definitely a feature - I can promise you it's much more programming effort to save old pages for a quick redraw than to free the old page and replace it with the new.

So I guess the discussion here is, "is it right for firefox to use so much memory?" My answer is yes. It is not a memory leak, it seems like a very valid design decision. But if you disagree, old versions of firefox still work great (I still haven't upgraded myself).

Re:spyware with IE or memory leak with FF..

[crabpeople]

How can you trust the CEO after he didnt swim across the ocean?

i could never stand behind a company like that and refuse to use opera products untill he makes good on his word. You cant just throw statements like that around. Browsers designed by liars are dishonorable browsers.

Firefox developers don't "get it"

[Vellmont]

Memory isn't an unlimited resource you just hoard whenever you think you need it. Right now my instance of firefox is taking up 128 megs! I've seen it up to 256 megs before. This is just simply insane. I've seen people who's computer performance has gone down the tubes because firefox is taking up all the memory (and these are machines with 512 megs of memory, not exactly tiny). What I'd like to convey to the firefox devs is this: Your application isn't the only one running on the system. Play nice and don't be a hog.

With the number of people complaining about this (and the number of people that don't even KNOW to complain) isn't it a safe bet that you've made a mistake in the amount of cached pages?

Rewritting history.

[SeaFox]

Firefox 1.5 implements a Back-Forward cache that retains the rendered document for the last five session history entries for each tab. This is a lot of data. If you have a lot of tabs, Firefox's memory usage can climb dramatically. It's a trade-off. What you get out of it is faster performance as you navigate the web.

The only problem is there were bugs filed for memory leaks long before Firefox 1.5 and the Back-Forward cache were implemented. Maybe this feature does contribute to Firefox's large memory footprint, but to say that this feature is the only reason and that there are no leaks is simply false.