Mozilla Unleashes the Kraken

An anonymous reader writes "Mozilla has released the first version a new browser benchmark called Kraken. Mozilla's Robert Sayre writes on his blog, 'More than Sunspider, V8, and Dromaeo, Kraken focuses on realistic workloads and forward-looking applications. We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads for pushing the edge of browser performance forward. These are the things that people are saying are too slow to do with open web technologies today, and we want to have benchmarks that reflect progress against making these near-future apps universally available.' On my somewhat elderly x86_64 Linux system Google Chrome 6.0.472.55 beta completes the Kraken benchmark in 28638.1 milliseconds, Opera 10.62 completes it in 23612.4 milliseconds, and the current Firefox 4 nightly build completes it in 19897.5 milliseconds."

Comments

Javascript

[SimonTheSoundMan]

Shame it only benchmarks one small part of the browser - Javascript.

Re:Javascript

[Anonymous Coward]

The scary bit is that the world is quickly moving in a direction where serious desktop applications will be written in... Javascript.

So much for Java, .NET ; as soon as its possible to earn money through the Google App store for your Web app there will be a torrent of these applications being release to the world.

The web browser is the new platform.

It feels like going back 20 years in time.

Re:Javascript

[koiransuklaa]

Pity its horrible as a language

Many people have told me that... so far I'm unimpressed by their arguments. Yes, there are a lot of people who abuse js -- but how would that change if we gave them another language? Yes, working with the DOM in a cross-browser way is a pain in the ass -- but how would that change if you did that via another language? Yes, some js engines have bugs and performance issues -- but how do we know the engines for the other languages would be better (remember that we need an engine for every browser)?

Now, there are some valid complaints on javascript as a language: it was designed in a hurry and then left to rot. With all its faults, I still think it's a pretty damn beautiful and expressive language. The awful quality of books, tutorials and example code on javascript is a major reason for the reputation it has, but check out "JavaScript: The Good Parts" by Douglas Crockford if you want to see the genuine elegance in javascript -- this book should be a requirement for anyone who wants to either code in js or express an opinion about it in public.

Re:Javascript

[olau]

Sure, but the whole point of many Javascript-enabled applications is manipulating the DOM. Which is often really slow. Something as simple as putting a couple of invisible divs on the page and measuring their height is measured in milliseconds, not microseconds.

So while some applications obviously aren't possible without a fast Javascript engine, I think if you really want to make the web faster for people, you need to include a DOM benchmark. Something like inserting text, inserting elements, moving elements, fading elements.

Re:Javascript

[koiransuklaa]

Sorry, but this goes to the unimpressing-arguments pile... Debugging javascript is not more difficult than debugging anything else that runs in an environment like the browser and the tools that are available are fairly good. "Debugging by popups" was a choice you made, not something related to javascript the language.

Re:Javascript

[noidentity]

The web browser is the new platform.

It feels like going back 20 years in time.

Except that you now need what was then a supercomputer just to run Hello, world.

I hope that Firefox isn't playing Microsoft's game

[Chrisq]

ways suspicious when someone releases a benchmark that shows that their software is better than others, especially when other benchmarks have shown FF as slower than Chrome or Opera. I hope this isn't one of those M$ style tests that find the bits that their own software does well and others badly and test that.

Am I the only one?

[Mystery00]

Am I the only one who has had enough of these benchmark tests? I don't care about your milliseconds! What I want is low RAM usage (how about concentrating on THIS Mozzila?) and more features/plugins.

Re:Am I the only one?

[BZ]

Well... Mozilla _has_ concentrated on low RAM usage in the past. The actual memory usage of Gecko is significantly lower than its competitors if you load some pages and measure it.

At this point, they're actually trading off space for performance (e.g. making some core objects slightly bigger to improve certain performance characteristics).

Re:Am I the only one?

[Anonymous Coward]

What I want is low RAM usage

Ok, so you want a lighter, leaner browser...

and more features/plugins.

And at the same time more feature-rich.

Wait, what?

Re:Am I the only one?

[Mystery00]

I meant that Firefox still has RAM leaks.

Re:I hope that Firefox isn't playing Microsoft's g

[AberBeta]

If you hadn't noticed, every synthetic benchmark released from a browser vendor favoured their engine, at time of release. At least Google had balls to call it v8bench.
While I believe all benchmarks (and non-comprehensive ACID tests) to be 3dmark-style pissing contests where they encourage developers to fast-path specific used functions, I have more confidence in Mozilla producing another (Dromaeo also tried to have a more realistic workflow).

Re:I hope that Firefox isn't playing Microsoft's g

[paziek]

Its not like only MS and Mozilla as browser vendor released their own benchmark in with their product is doing good.
Besides, whats so bad about it? Ain't it obvious they are gonna include in their benchmarks stuff that they feel is important and as a consequence - made it good during browser development?
It just shows that other browsers than FF lack in some areas, with might - or might not - be important.

The circle is now complete!

[Jugalator]

Google wins in their test! (that curiously heavily exploit recursion and other good parts of the V8 engine)

Microsoft wins in their tests! (that curiously heavily test only DirectX acceleration)

... and now, Firefox wins in their test! (which has yet to be disassembled to reveal how they dodge Opera and Chrome from winning, when they use to in all others, including independent tests like Peacekeeper)

Unfair method?

[TeXMaster]

Comparing the FF4 nightly builds against the latest released versions of the other browsers is quite unfair. So I tried this thing on my Intel Core2 Duo T9400 @ 2.53GHz laptop, and Opera 10.70.9046 (the most recent alpha available from Opera) and that gives me 12841.5 ms +/- 2.5%. OTOH I don't have FF4 nightly builds here ... can somebody actually run a comparison on the _same_ hardware to check all the most recent available builds of all browsers?

Re:I hope that Firefox isn't playing Microsoft's g

[Grismar]

Have you considered that it may well be the other way around?

If Mozilla, Google, MS, Apple or whoever truly believe that those particular aspects of a browser are the most important, doesn't it make sense that they would optimize their browsers for those aspects? I think it makes sense that they would write tests for the exact same aspects that they have been optimizing their browsers for, -because- they believe these are the key aspects.

Lacking an objective measure, all you can do right now is decide with whom you agree the most and probably use their browser or another browser that ranks well on their test - if these benchmarks are a critical decision factor for you.

Biased?

[danwiz]

We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads

Or just better in terms of reflecting where their product is strongest?

Isn't it just a little bit suspicious when the browser people release a benchmark that scores their own browser as the fastest? Intel's benchmark in 2002 was known to have emphasized performance traits specific to Intel chips.

One thing I don't understand.

[M8e]

How did mozilla leash the kraken in the first place? or maybe it was godzilla that did that?

Who cares

[obarthelemy]

My browser's performance has always been "good enough". Can we talk about ergonomy, reliability, compatibility, please ?

Re:I hope that Firefox isn't playing Microsoft's g

[Fnkmaster]

The major browser Javascript engines (with the apparent exception of IE) are all now within the ballpark of each other. And they all make slightly different tradeoffs and are optimized for slightly different conditions, and have all released benchmarks that illustrate the strong points of their browsers.

If you look at v8bench (Google's Javascript benchmark), sunspider (the Webkit Javascript benchmark), and now Kraken (Mozilla's Javascript benchmark), you'll see that the latest browser versions are basically within 5-30% of each other on identical hardware. Which one comes out ahead depends on whose set of optimization parameters you think is most important.

Attacking Mozilla for doing the same thing every other major browser maker does (not that your post was, but other posters have) is silly.