Slashdot Mirror
Mozilla Unleashes the Kraken
An anonymous reader writes "Mozilla has released the first version a new browser benchmark called Kraken. Mozilla's Robert Sayre writes on his blog, 'More than Sunspider, V8, and Dromaeo, Kraken focuses on realistic workloads and forward-looking applications. We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads for pushing the edge of browser performance forward. These are the things that people are saying are too slow to do with open web technologies today, and we want to have benchmarks that reflect progress against making these near-future apps universally available.' On my somewhat elderly x86_64 Linux system Google Chrome 6.0.472.55 beta completes the Kraken benchmark in 28638.1 milliseconds, Opera 10.62 completes it in 23612.4 milliseconds, and the current Firefox 4 nightly build completes it in 19897.5 milliseconds."
One of Mozilla's longstanding issues with some of the other benchmarks is that they test toy problems that take longer to set up than to run. Yes, that favors browsers with JS engines that set up for execution quickly, and that portion of the engine is important. It doesn't show the real speedups for intensive applications in the browser, though. Optimizing the slow parts is the priority of most people right now, and getting the application set up a little faster at the beginning isn't as big a deal unless you have a lot of small scripts in one page.
An earlier blog post by Sayre and some of the comments to it display some of the issues.
For this particular Slashdot page right now, with both browsers opened fresh for it, Firefox 4.0 beta 6 uses 23 megabytes less resident memory than Chrome 5.0.375.125 does. It also uses about 1800 megabytes less virtual mapped memory, not that that matters nearly as much, but it's a big number in difference.
Epiphany 2.30.2 uses 11 megabytes less residential still, but about as much virtual as Chrome.
Galeon 2.0.7 uses about the same residential memory as Firefox and about twice as much virtual.
Midori 0.2.6 uses 5 megabytes less residential than Firefox, and about 1850 megabytes more virtual.
Arora 0.10.2 uses about twice as much residential memory as Firefox, and about twice as much virtual.
Dillo only needs 11 megabytes to render the page, but that doesn't have JavaScript and only shows a handful of comments without being able to get more.
Fennec 1.0 uses about the same memory footprint as Firefox 4.0 Beta 6, despite being the small-device Mozilla browser.
What is your exact complaint about Firefox's memory use? Are you still experiencing the huge memory leakage and growth from the 2.0 series?
You forgot the part about Apple winning in their test (sunspider), and the curious cache of the values of the sin() function in their JS engine that just happens to be the right size for that test.
Fundamentally, browser makers optimize their engine for what they consider important. They also put the things they consider important into benchmarks. The result is somewhat predictable.
Now Peacekeeper is an interesting mention, except I've actually looked at its code. This is a benchmark that measures things like 10,000 calls each of which removes 20 elements from an array that starts with 100,000 elements. It has (failed, interestingly) attempts to browser-sniff and run different code in different browsers. I wouldn't take its numbers to mean much of anything, in general, without some careful study of the exact tests you're looking at. Of course it's also measuring a lot more than just JavaScript; in that sense it's better than most of the benchmarks out there, if you think it manages to correctly measure the things it claims it's measuring.
One other thing, by the way: I fully expect that on Kraken shipping Chrome and Opera are faster than Firefox 3.6.
Actually FF4 nightlies beat IE9 in many of the IE9 testdrive tests. FF4 has the same Direct2D, Direct Write, and DirectX 9 hardware acceleration that IE9 does but FF4's javascript engine is better which gives it better FPS in those tests. FF4's javascript engine is a lot faster and there's still lots of room for improvement. FF4 Beta 7 will have the new javascript engine but it's already been merged to the nightly trunk branch so you can try it now if you want. Firefox had hardware acceleration first (in nightlies) and it's on track to be the first to have it in a major release (FF4).