Slashdot Mirror
IE9, FF4 Beta In Real-World Use Face-Off
An anonymous reader writes "Most browser benchmarks are isolated, artificial tests that can be gamed by browser vendors optimizing those specific cases. With only those benchmarks to go on, the folks at LucidChart were skeptical that the IE9 beta would actually outperform other modern browsers in real-world applications. To separate hype from reality, they built their first browser benchmarking tool, based in LucidChart itself. This benchmark is to SunSpider what a Left4Dead 2 benchmark is to 3Dmark Vantage. Product specs don't matter, only real-world performance on a real-world application. The results were surprising. IE9 held its own pretty well (with a few caveats), and the latest Firefox 4 beta came in dead last."
The way this benchmark measures "intra-frame time" is broken. In particular, it uses a setInterval with a 1ms delay. No browser actually respects that 1ms. Chrome clamps it to 5ms; others clamp it to 10ms, all to avoid the website thrashing the CPU pointlessly.
The upshot is that Chrome's interframe delay in the graph is about 5ms and Firefox 3.6's interframe delay is aboug 10ms. Which this particular benchmark can't tell apart from "no delay at all", given its methodology.
Firefox 4 beta, IE9 beta, Safari, and Opera seem to have delays greater than 10ms, so they're clearly doing some work they can't finish in 10ms.... or have slightly buggy timer implementations. Or both.
Of course in practice frame rates above 60fps or so are pointless since the screen doesn't redraw that often. ;)
On the other hand, on Mac, on modern hardware, I get 4.5fps in Chrome 7 dev on a random trial document I just tried, with JS render tiems on the order of 7ms (with a 7ms standard deviation) and "intra-frame time" of 224ms with a 900ms standard deviation (yes, those numbers are nuts). Firefox 4 beta comes in at about 11s for the JS (with 3ms stddev) and 125ms for the "intra-frame time" (with a claimed stddev of 0, which looks really suspicious).
It'd be nice if there were non-obfuscated source for this benchmark so its number-crunching could be evaluated; that 0 stddev is ... highly improbable.
This benchmark can be run by anyone in LucidChart. First, sign up for a free account here.
Nuff said
How have Apple and Microsoft handled your bug reports for Mail.app and Outlook? Did the handle them like Mozilla, where you enter the bug directly in their internal bug databases, monitor the progress, participate in discussions with the developers, and even contribute development yourself? Or do you have no idea what the status is, no influence on the outcome, and no ability to contribute at all? Were the bugs even submitted to development? Were you able to find a way to submit them to Apple and Microsoft at all -- could you communicate with anyone beyond level 1 end user support technicians?
Every application has bugs as old as its first release-- have you seen the age of some Windows security vulnerabilities, going back over a decade? -- and your particular concern won't necessarily get fixed. But if you compare the experience of handling end user bugs at Mozilla with the same thing at Apple or Microsoft, well, there really is no comparison.
Meanwhile, in the real world, people actually want the source of all their information to be fast. Go figure.