Slashdot Mirror
Reducing Firefox's Memory Use
An anonymous reader writes "Many people have complained about Firefox's memory use. Federico Mena-Quintero has a proposal for reducing the amount of memory used to store images, which, in his proof of concept code, 'reduced the cumulative memory usage... by a factor of 5.5.'."
Coming from me, this is sarcasm, but it's a depressingly prevalent real attitude in the industry.
John
He gives three possibilities: storing it uncompressed in the server, storing it uncompressed in the client, and storing it compressed in the client (and uncompressing it on the fly).
I wonder if it might be interesting and worthwile to have an X extension to store it compressed on the server? That way there's a lot less X traffic, and potentially a lot more applications could make use of it.
The only condition is that you don't need to decompress in Moz, and recompress it to send to the X server, but just pass along the compressed data (there's some security implications with that though, but I guess they could be dealt with).
Windows has something called GDI objects (graphics device interface objects), and firefox uses too damn many of them.
Somewhere after 5000 of them are in use windows slows down to a crawl and dies no matter how much memory you have, and with enough tabs and windows open firefox will be using 4000+ of them all by itself.
I'm testing out browsers for use on some old machines as web kiosks. Basically, my choices are:
These machines (P1), and lots of machines like them, pretty much max out at about 64 megs of RAM. I could probably find more RAM, but it'd be costly, and there are usually hardware compatibility problems.
Although I'm leaning towards Opera at the moment, I was using Konqueror for a while. Linux does a great job of swapping, and Konqueror is quite snappy, so even with low memory it's a viable option. But, with all the libraries that Konqueror requires, 64 megs is kind of pushing it.
And there is a decided trend in hardware towards less memory and faster processors. It's not uncommon to find Pentium III's with only 128 megs of RAM. Unfortunately, many open source programs are written without limited memory requirements in mind.
It's kind of humbling to think that, as few as five years ago, a Pentium I with 64 megs of RAM would run an entire OS and web browser without so much as touching swap space. Today, you have to use apps designed for embedded machines to run in 64 megs of RAM, and you're lucky if you can run more than one app at a time.
From my testing, Firefox is barely outside the range of viable options for a machine with 64 megs of RAM. But as with any performance tuning, there are probably trade-offs. And having lots of options is usually the best strategy. But I think these improvements suggested for Firefox would be beneficial in almost any scenario. Avoiding I/O seems to be the best strategy on any system newer than, say, a Pentium I, when web browsing. So uncompressing images on the fly in exchange for less memory usage would doubtlessly be a good trade-off.
"I assumed blithely that there were no elves out there in the darkness"
This was a nifty piece of investigation, but doesnt address the largest cause of firefox memory usage. Namely, memory is not freed when tabs are closed.
5 6
See:
https://bugzilla.mozilla.org/show_bug.cgi?id=1314
Try a test. Fire up a clean FF and note memory usage. Go to somewhere like fark.com and open 50 links in tabs and note mem usage. Close every tab and see if mem usage goes down. It doesnt. Most people visit dozens of pages a day. Hundreds per week. After a while, the memory footprint of FF can grow to epic proportions (ie hundreds of megs) even with only a few tabs open because FF cannot release memory of closed tabs. I have to restart FF every week or so because I'm tired of it using 200MB for no good reason.
It doesnt bother me so much that FF stores uncompressed images for tabs which are active (ie. open, even if not visible). The article itself mentions a performance hit when storing compressed images. But why the f*** cant it free the memory when I close that tab? The fact that I explicity closed it should indicate that I dont want it anymore. FF developers have acknowledged the problem but have said that there is no easy fix. Probably a poor design in the underlying architecture, though no one associated with the project would state it that bluntly.
BTW, this article reminds me of one of the best reasons to use some sort of adblocking software. You save quite a bit of memory when you arent caching a dozen useless images with every new web page you visit. Especially in light of the above bug, you can significantly slow down the expanding memory footprint with adblocking.
Consider: since my box has 1G of memory, I do want the X server to hang on the all those pixmaps, because that makes firefax run fast. The hack would make it waste CPU time re-uncompressing images, whether it's needed or not.
With the way Firefox works now, if memory does start to run short, well, that's when the kernel will start paging things out based on its clever working set algorithms. If a given pixmap area in the X server hasn't been accessed in quite a while, it'll get swapped out to disk and the memory reclaimed. If the pixmap is accessed later, it'll automatically page back in.
I don't know about your box, but mine (Athlon XP2000+) can decompress JPEGs at a rate of around only 3MB per second. My disk drives, OTOH, are a hell of a lot faster than that.
In other words, letting the OS do its job by tossing the images onto swap when necessary strikes me as a much better strategy than constantly sucking up CPU decompressing every image every time it's used just in case the memory might be needed.
People worry too much about VMEM, IMHO. If I write a program that allocates 1G of memory, but then spins around using only 10k for the next hour, it'll have basically zero impact on the OS. Only ~10k if real RAM is actually getting used.
I've switched to opening links in new windows a lot now. In part this is because I want to group a set of tabs together. And it's easier to just close the whole mess by closing that whole window (generally all on one site or about one topic). But it seems FF is not free-ing up memory in these cases, either.
I don't see why a tab or a new window should be different internally, though. It should only be a matter of associating the state of a loaded page with a given display context. The real issue, though, is the memory management issue. Apparently something fundamentally wrong in the browser architecture is preventing that. I highly suspect it is due to over-abstraction and/or the inability of some tools they are using to properly destruct objects that are no longer needed. It does seem that large complex software projects such as this do tend to suffer a lot of complexity issues that result in basic things like free-ing memory becoming impossible to do. I don't encounter these problems in my programs, but then, I don't do anything nearly as large as Firefox, nor do I use a team of developers, nor do I use all these abstract tools by ignoring their internal operation implementations. I'll be curious as to the actual, real cause.
now we need to go OSS in diesel cars
One possible contributor to the memory issue here, and in some other programs, could be the way the memory is allocated. Memory is obtained from the kernel in chunks a multiple of a page size. These pages cannot be returned back to the kernel unless all usage of the entire page is gone. Memory usage for a typical object tends to be small pieces. If the pieces allocated for one page (in one tab) are interleaved with pieces allocated for another page (in another tab), then closing one of those tabs, even if the mainline code destructs all objects which correctly free all their underlying memory allocations, does not necessarily result in pages being released back to the kernel.
So how can memory allocations get scattered around like that? Consider that many objects need to persist as long as the page exists, but many others can be destructed because they are only needed when the page is being loaded or rendered. During loading and rendering, both sets of objects can be created in a mixture. Then the non-persistent ones would be destructed. Because of the order of allocation of underlying memory, the persistent objects tend to be interleaved with the non-persistent ones. That then means most pages may have some persistent object lying around, preventing it from being returned to the kernel.
Solutions to this problem would be difficult. But I also think this effort would be valuable for any and all large projects that can face this kind of memory issue. Some means is needed to control the memory alloction, and in particular to allow grouping of memory into contexts. The first kind of context would be a context for each tab or window being opened by the browser user. That way, if a tab is closed, it should substantially destruct objects grouped together. But this can also be wasteful because the non-persistent objects that do get destructed after rendering is done cannot have their memory recycled by other contexts. So another dimenstion of context needs to be on the basis of what is persistent vs. non-persistent, so that all non-persistent memory gets grouped together so it can be returned to the kernel as whole pages, which can then be recycled to other contexts (getting the pages from the kernel again).
This would require a much more involved memory allocation system. Further, it would also require major changes in many of the abstract programming classes used by such large programs ... in ways that tend to be counter to what the abstraction is all about in the first place. Abstraction is supposed to hide details about the underlying implementation so the programmer can/should concentrate on application logic. But this is not really an optimal way to program when dealing with limited resource issues that need to be managed, such as the memory issues seen here. In particular, the various classes themselves won't know whether they are persistent (in the context of what the browser application needs) or not. Many instances of the very same class may be created for both persistent and non-persistent intents, so the class itself could never be designed to make any such assumptions (e.g. think of hiding the details in reverse ... the class does not see the details of the application, e.g. which instances are to be long lived and which are not).
A concept that may help with this is one that would have to be applied to the whole of such object oriented programming, or even non-OO functions that also could allocate memory for such variant uses (this isn't fundamentally an OO problem ... OO merely exposes it due to the larger application scale that OO enables to be implemented). This concept is to create instance groups that can span laterally across all classes. It would require that each time an instance is created, that it be associated with a particular instance group. Then instead of destructing each instance individually, the group is destructed, which destructs all instances in the group. The implementation of all thi
now we need to go OSS in diesel cars
This is exactly the sort of thing you handle with a slider in the options; "How would you like Firefox to handle images" with High CPU/low RAM on one end and Low CPU/High RAM on the other. The slider would determine how aggressively Firefox caches uncompressed images.