Slashdot Mirror
Mozilla MemShrink Set To Fix Firefox Memory
darthcamaro writes "If you're like a lot of Firefox 4 users out there, you've probably noticed that Firefox has a serious memory problem — it uses more than it really should. At long last, Mozilla developers are finally set to take this issue seriously with a dedicated team called MemShrink that are focused on the problem. 'It's pretty clear by now that this is a much bigger problem than any one person can likely tackle,' Mozilla Developer Johnny Stenback said."
My experience has been that both leaks and overall memory use have gone down between 3.0 and 4.0.
At the moment, Firefox is at about 375 megabytes, with 16 tabs open. It has been open for 3 weeks. I do have browser.sessionhistory.max_total_viewers set to 3 and the anti-malware databases disabled though.
Nerd rage is the funniest rage.
A useful new feature in latest Nightly versions from Mozilla is about:memory. It gives you a full tree view of where the memory is being used. Of my 360MB which the browser is currently using, 101MB is JS, 46MB is storage (you back button and memory cache), 70MB on "heap-unclassified" whatever that is. JS seems to be the biggest consumer of memory.
OS X here -- I usually have 8 to 12 tabs open. I almost never see memory usage below 500M, and it usually grows to about 1G after 2 or 3 hours. Firefox 4 for the Mac is seriously broken w.r.t. memory usage IMHO, and if they can't fix it fast, I'll probably be switching. I clobbers the performance of the whole system when it hogs that much memory. I'm tired of having to restart FF all the time.
as this amount decreases Firefox can begin to proactively free memory used for the oldest cached data.
So your suggestion is that each application running should be fine allocating a huge cache and changing its memory footprint according to how much memory it sees available on the entire system, instead of the OS making a decision?
I am not sure I am convinced that the outcome of that methodology is optimal. If it were; I think i'd favor "number of pages swapped in/out per second" over amount of memory free, though.
I am trying to imagine the interactions of 4 or 5 different applications all running with a huge cache, and the same behavior.... when memory usage is low, all 5 applications prepare a huge cache -- their huge cache causes the total memory free to drop, eventually to 1MB... now, suddenly, all 5 applications will use a bit of CPU time proactively freeing up large swaths of cache -- CPU will be 100% running for a couple seconds, as processes adjust their memory.
After all 5 apps reduce their huge caches, suddenly there will now be a lot of memory free --- so much memory free, that one or more of the applications might immediately see an opportunity for increased caching.
So what mechanism will protect fairness? Each application will believe its cache is important, but who's to say one of the applications isn't more important to the user, or having more requests more frequently made of it (so that the user's performance will best if application X's cache is bigger versus application Y).
There seems a fundamental weakness here, involving each application trying to make their own memory management decisions about cache --- the application making the decision to expand its cache may be the one the user cares about the least, and the one whose cache is the least useful.
The OS is in a position to make decisions and mediate in regards to the working set needs of processes, and the user's actual usage patterns. The OS knows which running process makes the most demand of its cache -- the other processes don't know much about each other.
The system you describe is called malloc()!
In a system with a unified buffer cache (essentially, every OS in wide use except OpenBSD), it makes little difference whether a page of memory comes from a private memory allocation (e.g., a heap allocation), a memory-mapped file, or the OS's disk cache. When a process needs a page not already present in memory, the kernel's memory manager tries to find an unused page. If one is available, it hands it to the program that requested memory.
Otherwise, it looks for an in-use page, saves its contents, and hands the just-freed page to the program requesting memory. If that page is "dirty" --- i.e., it's backed by a file and somebody's written to that part of the file, or it's a private allocation backed by the page file --- the memory manager writes the page out to disk first. If the page isn't dirty, the memory manager can just discard its contents because it knows it can reconstruct it by reading back the original file.
When the memory manager has to go to disk to satisfy a request for a new page, it's called a hard fault. The mission of the memory manager is to reduce the number of hard faults, because hard faults are slow. The fewer hard faults you have, the less time will be spent waiting for the disk, and the faster your system will run.
The most important part of the memory manager is page replacement: i.e., how the memory chooses what page to evict in order to satisfy a memory allocation request. Most systems use an approximation of LRU (i.e., least recently used), throwing out pages that haven't been accessed in a while. It doesn't usually matter where a page came from. The only important factor is how recently it was accessed.
So, you can see that there's no difference between a program mapping a file into memory and modifying it, reading and writing it using file APIs, and just manipulating an equal amount of information in buffers created with malloc. To the kernel, all memory is made up of pages.
The "go away for a while" problem isn't caused by any particular memory strategy. It's an artifact of the memory manager's LRU approach. How does it know that the pages corresponding to Firefox are going to be used again? If some other program needs those pages, the older ones will be discarded. There is nothing applications can do.
Instead, the OS itself has to be tweaked to preserve interactivity. Sometimes the memory manager will prefer disk cache pages to malloc-backed ones. Sometimes (e.g., for Windows SuperFetch) the OS will try to identify pages belonging to activate applications and try harder to keep those in memory. Some systems favor keeping executable pages over private allocations. You can tweak the page replacement algorithm, but the basic idea, that all memory is made up of pages subject to the same management scheme, applies.
Ultimately, it's ridiculous to hear people talk about programs "keeping things in memory" like we were still dealing with DOS 6 and OS 9. The actual situation is a lot more subtle, and silly memory counters don't even come close to giving you a good picture of what's actually going on.
In short, don't worry about fine-tuning what's "in memory". Don't change behavior based on total amount of memory in the system. Operating systems (OpenBSD aside) ALREADY DO THAT. Just let the memory manager do its job, and give it enough information (via interactivity information, memory priority, etc.) to do its job properly. Don't try to hack around problems at the wrong layers.
I use this little restart firefox extension, kind of a cool trick.. Also config trim on minimize (google it).
-taosk8r
There are several things going on here:
1) JS JITs. These optimize for speed of compilation and speed of generated code; small size of generated code is not really something being optimized for except insofar as it helps one of the other two metrics. In the case of Firefox, just deciding to JIT a little less aggressively late in the 4.0 cycle saved a good bit of memory when JS-heavy pages are open.
2) Images. Sites are using more and more bigger images, in addition to larger and larger scripts. With images you have the options of decompressing on draw (slow, typically) or storing decompressed images in memory (uses lots of memory). Guess which one browsers are typically doing?
3) Leaks in webpages. By this I mean web pages that allocate more and more JS objects and have them all reachable (e.g. by sticking things in an array that's hanging off the Window) so the web page uses more and more memory. gmail did this until recently; they were working on fixing it last I checked. This means that if one of your "few tabs" is gmail and you've had it open for a while a lot of that memory could be actually being used by gmail.
about:memory in Firefox is being improved to make it easier to answer the "what's using the memory" question, at least....