I don't know why we bother caching parts of the sqlite files at all. That's what mmap and OS filesystem caches are for.
The good news is the sqlite memory use won't keep growing. At worst it will grow to about the size of the database, and then remain steady. It's only a significant fraction of Firefox's memory use when Firefox isn't using much memory.
On the other hand, fake-scan scams rely on Windows users' fear of Windows viruses in order to trick users into installing malware. I guess evil psychology tricks hurt users of both platforms.
Because each browser's JS engine is closely tied with its DOM, which is in turn closely tied with its layout engine. Otherwise it would be difficult to have efficient DOM calls and complete garbage collection.
"Developing a competitor to LLVM" would imply creating something generic enough to be a backend for many programming languages. Maybe something geared toward dynamic rather than static languages, and tuned better fast compilation, but still comparable in scope to LLVM.
If you're on Mac, you can use an app called "Spin Control" to figure out what's causing the pauses / CPU spikes.
My guess is that you're hitting garbage collection (GC) pauses. Firefox 3.6's GCs are global, so the more tabs you have open, the slower each GC is. Firefox 4 has GC compartments, so most GCs are per-domain or per-tab. Firefox 4 has also had its GC-timing heuristics tuned to allow smooth animations in many cases.
Firefox 5 has additional changes to limit the damage caused by having hundreds of tabs open, such as a drastic setTimeout clamp for background tabs. Firefox 5 is currently in the Aurora channel and will be in the Beta channel soon.
Consider setting browser.sessionstore.max_concurrent_tabs to 0 so that after you restart Firefox, it only loads the tabs you switch to.
What do you find inefficient about Firefox 4? Did you find Firefox 4 inefficient, or only Firefox 4 betas? I hope we can figure it out and fix it before your Firefox 3.6 stops getting security updates:)
It sounds like you're asking for an option like "Firefox, please pretend my computer only has 512MB of RAM" that affects all types of caches in Firefox. Or a slider that's like "Make Firefox as fast as possible [------|--] Leave as much RAM for other apps as possible". I don't think such a thing exists.
You can install RAMBack for a "clear all in-memory caches" button. In combination with about:memory, it can help you tell the difference between healthy caching behavior and memory leak bugs. Unfortunately, it skips sqlite-based caches, which are some of the largest.
You can disable Firefox's "Block reported attack sites" and "Block reported web forgeries" to save maybe 15MB of RAM. On the other hand, it's nice to have that extra defense against zero-day attacks.
You can disable Firefox's URL history to save maybe 35MB of RAM, but then you'll lose purple links and the awesomebar (unless you use bookmarks extensively). There used to be a way to tell Firefox to only keep a few weeks of URL history, but I can't find it now.
You can lower browser.sessionhistory.max_total_viewers from "8 if there's enough RAM" to 1 or 2. This controls how many navigated-away pages Firefox will keep in memory with their complete state (rather than just scroll position and form data). I'd advise against lowering it all the way to 0, because if you accidentally click a link or close a tab, restoring the page from bfcache is not only faster but also significantly less likely to lose page state (especially on AJAXy pages).
If you use session restore, you can instruct Firefox to only load restored tabs once you switch to them, by setting browser.sessionstore.max_concurrent_tabs to 0.
In most browsers, including Firefox, popups (including popunders) are blocked except when they appear in response to clicks.
In Firefox 4, we think we've solved the problem that allows popups to turn into popunders. Now that you see them right away, it should be clearer that they're appearing only in response to clicks, and you should be able to tell which sites they're coming from.
Don't read too much into the number of bugs found so far in each browser.
* Michal Zalewski (who created crossfuzz) works for Google, so of course he focused his own efforts on Webkit.
* Of the "60 bugs" found in Mozilla, 50 were found by me, with a significantly more powerful and yet unreleased fuzzer. If I pointed my fuzzer at other browsers, I'd find more bugs in the other browsers too.
* The low numbers for IE are from very brief testing.
Both 32-bit and 64-bit versions are being produced on all supported desktop platforms (Windows/Mac/Linux), with the exceptions that Windows 64-bit builds are only produced once a day rather than upon every checkin, and Mac builds are packaged as universal binaries where the 64-bit part requires Snow Leopard.
Running plugins out-of-process allows us to avoid the "plugin is for the wrong architecture" problem, at least in theory.
Firefox's JS engine now has three execution modes: interpreter (all platforms), tracing jit (several platforms), method jit aka "jaegermonkey" (x86, x86-64, ARM). On x86 you're probably using all three. On less popular platforms, it uses only the interpreter, which is slower but still works. And even the interpreter has gotten faster as part of the overall performance effort.
Why would an implementation that doesn't give away sensitive information be slow?
It would have to run the pointer through a one-way hash. Or store extra information for each object. Either way, there's a cost to generating and storing the hash code, which seems silly when you're going to stick it right into a normal hash table (which isn't exactly the fastest data structure in existence).
I don't understand why buckets make the code slow, leaky, or difficult to test.
Slow because the JavaScript code needs to do at least one comparison and branch on every lookup. (This is in addition to the native comparison and branch done by the native hash table!)
Leaky because the bucket has to hold onto the object (which is ok in some situations but not in others).
Difficult to test because usually the first item in the bucket will be the one you want, but sometimes it won't.
I also don't understand why a GC would have to keep extra information for an object whose hash code method was called.
A moving garbage collector would need to keep extra information around, because a hash-of-a-pointer changes when the object's location changes.
Are you ctrl+clicking a bunch of bookmarks? Try putting them in a folder and ctrl+clicking the folder to open them all in one click. Less clicking, more waiting for the browser to catch up!
If you run Kraken in Firefox and Safari, and then compare the results of the subtests (by copying and pasting a URL), you'll see that Firefox does better at some of the subtests and Safari does better at others. So in one sense, Mozilla has done exactly that.
If you have a loop consisting of creating divs and measuring their heights, you're doing it wrong. Of course forcing the browser to run its layout algorithm repeatedly is going to be slow.
If you're only doing it once, why do you care if it takes a few milliseconds?
How are you displaying your tree and grid? Creating a large DOM tree consisting of HTML table elements? Programmatically drawing to a canvas?
However you're doing it, you should ask Boris Zbarsky (bz) to profile it. He'll probably tell you exactly what you're doing wrong (forcing/causing the browser to do more work than really needed), and then immediately post a patch to Firefox to make it 30% faster anyway.
What you're really asking for is a way to keep tables of object-to-value.
One way a language can accomplish that, as you suggested, is for the language to supply a "hash code" function and then using the language's string-to-value hashes. But a "hash code" function has many problems. A moving garbage collector has to keep extra information around, at least for objects whose hash code has been retrieved. Any implementation that doesn't give away sensitive information (including pointers) is likely to be slow. More subtly, any implementation that prevents hash-key collisions will violate GC confidentiality. So the "hash code" function has to allow collisions, which means your object-to-value map (build on top of hashkey) has to use buckets, which make your code slow, leaky, and difficult to test.
A more straightforward solution is for the language to provide an object-to-value table type. I think this will be part of ES6 as WeakMap. Which, by the way, has excellent GC semantics.
The JaegerMonkey team understands why it's currently slower on 64-bit than on 32-bit. One reason is that the larger pointers on 64-bit systems don't play well with the value representation. If that can be fixed, perhaps by using a different value representation in 64-bit versions, it might end up faster on 64-bit than on 32-bit.
They're working on speeding up the 64-bit version. They have to, because of the plan to ship Firefox 4 as a 64-bit application for Mac OS X 10.6;)
(Btw, arewefastyet.com shows speeds of naked JavaScript engines, which are usually slightly faster than JavaScript engines inside web browsers.)
I think the markup-parsing part of the HTML5 spec is reasonably stable. Firefox is the first browser to implement the algorithm, so Firefox may have useful feedback for the spec.
Don't worry, we're planning to make many of the changes customizable. For example, you'll be able to put the tab bar above the address bar or below it. And we're not copying chrome pixel for pixel.
Slashdot Mirror
Perhaps browsers should let you modify the volume per hostname or per tab, since they are also application platforms.
There are some bugs related to sqlite memory use, bug 411894 and bug 650649.
I don't know why we bother caching parts of the sqlite files at all. That's what mmap and OS filesystem caches are for.
The good news is the sqlite memory use won't keep growing. At worst it will grow to about the size of the database, and then remain steady. It's only a significant fraction of Firefox's memory use when Firefox isn't using much memory.
That's not so great. If you upgrade to [Firefox 6] Aurora, and look in about:memory after doing that, where does Firefox think all the memory going?
On the other hand, fake-scan scams rely on Windows users' fear of Windows viruses in order to trick users into installing malware. I guess evil psychology tricks hurt users of both platforms.
Because each browser's JS engine is closely tied with its DOM, which is in turn closely tied with its layout engine. Otherwise it would be difficult to have efficient DOM calls and complete garbage collection.
"Developing a competitor to LLVM" would imply creating something generic enough to be a backend for many programming languages. Maybe something geared toward dynamic rather than static languages, and tuned better fast compilation, but still comparable in scope to LLVM.
If you're on Mac, you can use an app called "Spin Control" to figure out what's causing the pauses / CPU spikes.
My guess is that you're hitting garbage collection (GC) pauses. Firefox 3.6's GCs are global, so the more tabs you have open, the slower each GC is. Firefox 4 has GC compartments, so most GCs are per-domain or per-tab. Firefox 4 has also had its GC-timing heuristics tuned to allow smooth animations in many cases.
Firefox 5 has additional changes to limit the damage caused by having hundreds of tabs open, such as a drastic setTimeout clamp for background tabs. Firefox 5 is currently in the Aurora channel and will be in the Beta channel soon.
Consider setting browser.sessionstore.max_concurrent_tabs to 0 so that after you restart Firefox, it only loads the tabs you switch to.
What do you find inefficient about Firefox 4? Did you find Firefox 4 inefficient, or only Firefox 4 betas? I hope we can figure it out and fix it before your Firefox 3.6 stops getting security updates :)
I'm pretty sure you want browser.sessionhistory.max_total_viewers, not browser.sessionstore.max_tabs_undo.
It sounds like you're asking for an option like "Firefox, please pretend my computer only has 512MB of RAM" that affects all types of caches in Firefox. Or a slider that's like "Make Firefox as fast as possible [------|--] Leave as much RAM for other apps as possible". I don't think such a thing exists.
You can install RAMBack for a "clear all in-memory caches" button. In combination with about:memory, it can help you tell the difference between healthy caching behavior and memory leak bugs. Unfortunately, it skips sqlite-based caches, which are some of the largest.
You can disable Firefox's "Block reported attack sites" and "Block reported web forgeries" to save maybe 15MB of RAM. On the other hand, it's nice to have that extra defense against zero-day attacks.
You can disable Firefox's URL history to save maybe 35MB of RAM, but then you'll lose purple links and the awesomebar (unless you use bookmarks extensively). There used to be a way to tell Firefox to only keep a few weeks of URL history, but I can't find it now.
You can lower browser.sessionhistory.max_total_viewers from "8 if there's enough RAM" to 1 or 2. This controls how many navigated-away pages Firefox will keep in memory with their complete state (rather than just scroll position and form data). I'd advise against lowering it all the way to 0, because if you accidentally click a link or close a tab, restoring the page from bfcache is not only faster but also significantly less likely to lose page state (especially on AJAXy pages).
If you use session restore, you can instruct Firefox to only load restored tabs once you switch to them, by setting browser.sessionstore.max_concurrent_tabs to 0.
In most browsers, including Firefox, popups (including popunders) are blocked except when they appear in response to clicks.
In Firefox 4, we think we've solved the problem that allows popups to turn into popunders. Now that you see them right away, it should be clearer that they're appearing only in response to clicks, and you should be able to tell which sites they're coming from.
I blame C++. Hard to parse, hard to analyze, full of surprises.
So do a few other people at Mozilla, who are working on a new systems language called Rust.
Don't read too much into the number of bugs found so far in each browser.
* Michal Zalewski (who created crossfuzz) works for Google, so of course he focused his own efforts on Webkit.
* Of the "60 bugs" found in Mozilla, 50 were found by me, with a significantly more powerful and yet unreleased fuzzer. If I pointed my fuzzer at other browsers, I'd find more bugs in the other browsers too.
* The low numbers for IE are from very brief testing.
Both 32-bit and 64-bit versions are being produced on all supported desktop platforms (Windows/Mac/Linux), with the exceptions that Windows 64-bit builds are only produced once a day rather than upon every checkin, and Mac builds are packaged as universal binaries where the 64-bit part requires Snow Leopard.
Running plugins out-of-process allows us to avoid the "plugin is for the wrong architecture" problem, at least in theory.
Firefox's JS engine now has three execution modes: interpreter (all platforms), tracing jit (several platforms), method jit aka "jaegermonkey" (x86, x86-64, ARM). On x86 you're probably using all three. On less popular platforms, it uses only the interpreter, which is slower but still works. And even the interpreter has gotten faster as part of the overall performance effort.
It would have to run the pointer through a one-way hash. Or store extra information for each object. Either way, there's a cost to generating and storing the hash code, which seems silly when you're going to stick it right into a normal hash table (which isn't exactly the fastest data structure in existence).
Slow because the JavaScript code needs to do at least one comparison and branch on every lookup. (This is in addition to the native comparison and branch done by the native hash table!)
Leaky because the bucket has to hold onto the object (which is ok in some situations but not in others).
Difficult to test because usually the first item in the bucket will be the one you want, but sometimes it won't.
A moving garbage collector would need to keep extra information around, because a hash-of-a-pointer changes when the object's location changes.
Are you ctrl+clicking a bunch of bookmarks? Try putting them in a folder and ctrl+clicking the folder to open them all in one click. Less clicking, more waiting for the browser to catch up!
If you run Kraken in Firefox and Safari, and then compare the results of the subtests (by copying and pasting a URL), you'll see that Firefox does better at some of the subtests and Safari does better at others. So in one sense, Mozilla has done exactly that.
If you have a loop consisting of creating divs and measuring their heights, you're doing it wrong. Of course forcing the browser to run its layout algorithm repeatedly is going to be slow.
If you're only doing it once, why do you care if it takes a few milliseconds?
How are you displaying your tree and grid? Creating a large DOM tree consisting of HTML table elements? Programmatically drawing to a canvas?
However you're doing it, you should ask Boris Zbarsky (bz) to profile it. He'll probably tell you exactly what you're doing wrong (forcing/causing the browser to do more work than really needed), and then immediately post a patch to Firefox to make it 30% faster anyway.
What you're really asking for is a way to keep tables of object-to-value.
One way a language can accomplish that, as you suggested, is for the language to supply a "hash code" function and then using the language's string-to-value hashes. But a "hash code" function has many problems. A moving garbage collector has to keep extra information around, at least for objects whose hash code has been retrieved. Any implementation that doesn't give away sensitive information (including pointers) is likely to be slow. More subtly, any implementation that prevents hash-key collisions will violate GC confidentiality. So the "hash code" function has to allow collisions, which means your object-to-value map (build on top of hashkey) has to use buckets, which make your code slow, leaky, and difficult to test.
A more straightforward solution is for the language to provide an object-to-value table type. I think this will be part of ES6 as WeakMap. Which, by the way, has excellent GC semantics.
The option is available. Right-click a toolbar and uncheck "Tabs on top".
The JaegerMonkey team understands why it's currently slower on 64-bit than on 32-bit. One reason is that the larger pointers on 64-bit systems don't play well with the value representation. If that can be fixed, perhaps by using a different value representation in 64-bit versions, it might end up faster on 64-bit than on 32-bit.
They're working on speeding up the 64-bit version. They have to, because of the plan to ship Firefox 4 as a 64-bit application for Mac OS X 10.6 ;)
(Btw, arewefastyet.com shows speeds of naked JavaScript engines, which are usually slightly faster than JavaScript engines inside web browsers.)
The Bugzilla entry is now public.
I think the markup-parsing part of the HTML5 spec is reasonably stable. Firefox is the first browser to implement the algorithm, so Firefox may have useful feedback for the spec.
Don't worry, we're planning to make many of the changes customizable. For example, you'll be able to put the tab bar above the address bar or below it. And we're not copying chrome pixel for pixel.