Slashdot Mirror
Mozilla Unleashes the Kraken
An anonymous reader writes "Mozilla has released the first version a new browser benchmark called Kraken. Mozilla's Robert Sayre writes on his blog, 'More than Sunspider, V8, and Dromaeo, Kraken focuses on realistic workloads and forward-looking applications. We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads for pushing the edge of browser performance forward. These are the things that people are saying are too slow to do with open web technologies today, and we want to have benchmarks that reflect progress against making these near-future apps universally available.' On my somewhat elderly x86_64 Linux system Google Chrome 6.0.472.55 beta completes the Kraken benchmark in 28638.1 milliseconds, Opera 10.62 completes it in 23612.4 milliseconds, and the current Firefox 4 nightly build completes it in 19897.5 milliseconds."
How about IE performance? Too bad to even mention?
How are sites slashdotted when nobody reads TFAs?
Shame it only benchmarks one small part of the browser - Javascript.
ways suspicious when someone releases a benchmark that shows that their software is better than others, especially when other benchmarks have shown FF as slower than Chrome or Opera. I hope this isn't one of those M$ style tests that find the bits that their own software does well and others badly and test that.
Am I the only one who has had enough of these benchmark tests? I don't care about your milliseconds! What I want is low RAM usage (how about concentrating on THIS Mozzila?) and more features/plugins.
"we've got trenchcoats and bad attitudes" - John Constantine, HellBlazer
If you hadn't noticed, every synthetic benchmark released from a browser vendor favoured their engine, at time of release. At least Google had balls to call it v8bench.
While I believe all benchmarks (and non-comprehensive ACID tests) to be 3dmark-style pissing contests where they encourage developers to fast-path specific used functions, I have more confidence in Mozilla producing another (Dromaeo also tried to have a more realistic workflow).
Its not like only MS and Mozilla as browser vendor released their own benchmark in with their product is doing good.
Besides, whats so bad about it? Ain't it obvious they are gonna include in their benchmarks stuff that they feel is important and as a consequence - made it good during browser development?
It just shows that other browsers than FF lack in some areas, with might - or might not - be important.
One of Mozilla's longstanding issues with some of the other benchmarks is that they test toy problems that take longer to set up than to run. Yes, that favors browsers with JS engines that set up for execution quickly, and that portion of the engine is important. It doesn't show the real speedups for intensive applications in the browser, though. Optimizing the slow parts is the priority of most people right now, and getting the application set up a little faster at the beginning isn't as big a deal unless you have a lot of small scripts in one page.
An earlier blog post by Sayre and some of the comments to it display some of the issues.
Why don't they just grab the (say) 200 most visited sites on the internet, copy the JavaScript and use that to benchmark instead?
Simples.
Because the amount of time it takes to run the javascript on the top sites is pretty small (which is what the IE team was talking about around IE8's release). Performance on those sites mostly doesn't depend on whether your JS engine is the one in Chrome dev or the one in IE7. I only say "mostly" because I wouldn't be surprised if gmail is in the top 200. ;)
If you're going to worry specifically about JS performance (which is an assumption; the IE team is still saying that this focus is a mistake and to some extent they're right), you want to be benchmarking things that are gated on JS performance. That means identifying t the things that are slow with current JS engines and that people would like to be doing but can't because of said slowness, whatever those things are, and benchmarking those.
Google wins in their test! (that curiously heavily exploit recursion and other good parts of the V8 engine)
Microsoft wins in their tests! (that curiously heavily test only DirectX acceleration)
Beware: In C++, your friends can see your privates!
Comparing the FF4 nightly builds against the latest released versions of the other browsers is quite unfair. So I tried this thing on my Intel Core2 Duo T9400 @ 2.53GHz laptop, and Opera 10.70.9046 (the most recent alpha available from Opera) and that gives me 12841.5 ms +/- 2.5%. OTOH I don't have FF4 nightly builds here ... can somebody actually run a comparison on the _same_ hardware to check all the most recent available builds of all browsers?
"I'm never quite so stupid as when I'm being smart" (Linus van Pelt)
Have you considered that it may well be the other way around?
If Mozilla, Google, MS, Apple or whoever truly believe that those particular aspects of a browser are the most important, doesn't it make sense that they would optimize their browsers for those aspects? I think it makes sense that they would write tests for the exact same aspects that they have been optimizing their browsers for, -because- they believe these are the key aspects.
Lacking an objective measure, all you can do right now is decide with whom you agree the most and probably use their browser or another browser that ranks well on their test - if these benchmarks are a critical decision factor for you.
We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads
Or just better in terms of reflecting where their product is strongest?
Isn't it just a little bit suspicious when the browser people release a benchmark that scores their own browser as the fastest? Intel's benchmark in 2002 was known to have emphasized performance traits specific to Intel chips.
How did mozilla leash the kraken in the first place? or maybe it was godzilla that did that?
My browser's performance has always been "good enough". Can we talk about ergonomy, reliability, compatibility, please ?
The Cloud - because you don't care if your apps and data are up in the air.
Are you seriously trying to compare a single test on your system to a test run on completely different hardware? You can't say anything about Opera 10.70 vs FF nightly with your single result.
Mozilla Unleashes the Kraken https://secure.wikimedia.org/wikipedia/en/wiki/Kraken_botnet. I just wonder if this is somehow a 'Freudian slip', just to remind the world of how vulnerable the competition is?
with linux (kubuntu 10.04 64 bit). Completely froze Firefox 3.6.10pre. While it might be faster on Firefox, you can still use the browser while benchmarking with Chromium. Mozilla seriously needs to give each tab in Firefox it's own process. Try opening an article on slashdot with 300+ comments, while browsing at -1 and using the option that displays all comments. On my Quad Q6600 with 6GB Ram Firefox freezes for almost 20 seconds and is completely useless, unlike Chromium that let's me read other tabs while loading the article.
"Die endgueltige Teilung Deutschlands - das ist unser Auftrag." - Chlodwig Poth
Well, not to be too snarky, but my browsing profile currently matches what you say, with the added bonus of my constantly browsing the comments on Slashdot with Full/All-Comments on all the time. I've been browsing for about 30 minutes, I have 4 tabs open, and the following Addons: Adobe Contribute, ComEd's Real-Time Prices Toolbar, LastPass, FlashGot, Vuze Remote, and AdBlock.
I also have a Gmail session open, with Voice/Video Chat Enabled. My firefox is currently using 220 MB of Real Memory.
If I close Gmail, that drops another 20 MB.
I consider this machine a little long in the tooth, and I'm still running with 4 GB of ram. I can spare 220 MB for my browsing pleasure.
I'm with the following AC. You may be experiencing these problems. I don't. I would love to help figure out why you experience these problems, but I cannot reproduce them. How can I help fix them, then?
WhiteWolf666 an exBush supporter. All you new-school,compassionate,save the children Republicans can rot in hell
The major browser Javascript engines (with the apparent exception of IE) are all now within the ballpark of each other. And they all make slightly different tradeoffs and are optimized for slightly different conditions, and have all released benchmarks that illustrate the strong points of their browsers.
If you look at v8bench (Google's Javascript benchmark), sunspider (the Webkit Javascript benchmark), and now Kraken (Mozilla's Javascript benchmark), you'll see that the latest browser versions are basically within 5-30% of each other on identical hardware. Which one comes out ahead depends on whose set of optimization parameters you think is most important.
Attacking Mozilla for doing the same thing every other major browser maker does (not that your post was, but other posters have) is silly.