Slashdot Mirror
IE9, FF4 Beta In Real-World Use Face-Off
An anonymous reader writes "Most browser benchmarks are isolated, artificial tests that can be gamed by browser vendors optimizing those specific cases. With only those benchmarks to go on, the folks at LucidChart were skeptical that the IE9 beta would actually outperform other modern browsers in real-world applications. To separate hype from reality, they built their first browser benchmarking tool, based in LucidChart itself. This benchmark is to SunSpider what a Left4Dead 2 benchmark is to 3Dmark Vantage. Product specs don't matter, only real-world performance on a real-world application. The results were surprising. IE9 held its own pretty well (with a few caveats), and the latest Firefox 4 beta came in dead last."
Browser makers design for tests, real world data shows exact opposite results to what you expect. We've got a crew working on the story overnight and will have a full update for you on the weekend edition of Wicked Early News, we start before normal people wake up.
I'd like to see Chrome 7 results in there...
Unless the test contains porn and all the accompanying popups, it's not a real world test.
t
I must say that I have pretty much totally switched to Chrome. FF 3 and 4 are really dragging my system down, and often fail to load sites I depend on (gmail passwords are now randomly rejected in FF, but work in any other browser).
No native Gopher support?
From my cold, dead hands!
THL phish sticks
Try one of the Chrome forks, such as ChromePlus, SRWare Iron, Comodo Dragon, or a pure Chromium build. They're just like Chrome, but without the questionable Client-ID and RLZ modules that Google put in Chrome. I typically use ChromePlus since it has several features that I like builtin, but I've been trying the IE 9 beta and I like that as well. It's faster than Firefox in my opinion and I absolutely love the UI layout.
Hydraulic pizza oven!! Guided missile! Herring sandwich! Styrofoam! Jayne Mansfield! Aluminum siding! Borax!
from Linux this month after using Linux since 1993, I think this applies to all of FOSS.
Somehwere around 2000-2006 FOSS was basically head-and-head with commercial software in practical usability and maintainability, with its own distinct advantages and a relatively small learning curve.
Then there was this veer into "if you ever want all the Windows users to switch..." thinking, and in an effort to eliminate the learning curve FOSS threw away pretty much all of its advantages as well. If FOSS is just Windows/Mac OS/IE by another name, why choose FOSS?
Particularly when Windows/Mac OS/IE win on the polish, compatibility, and accessibility fronts by virtue of their being cathedral-built software?
With Firefox slow and cumbersome, Thunderbird choking on Gmail IMAP continuously while Apple's Mail.app sails along happily, and KDE4/GNOME3 being emblematic of the many ways in which FOSS has lost its way, I just decided I'd had enough of the nonsense. I'm ready to be able to walk into Best Buy, purchase any device, and expect that it will work seamlessly with the current generation of computing devices, without options, without Bugzilla (and condescendingly dismissive developer retorts), and without lots of consulting Google to find out how the gconf infrastructure has changed in the last two years or how HAL has been replaced by DeviceKit or policies moved from /etc tree A to uneditable dynamic filesystem B (but just use this easy command line management tool to set options...)
It just plain saves me a boatload of time and headache to use something else, like OS X plus Google apps plus Chrome. The pending desktopization of FOSS has fizzled thanks to the politics of the bazaar.
STOP . AMERICA . NOW
I'm not sure what is "real world" about spinning a UML box around another UML box in a giant (presumably) canvas-based javascript app.
For me, "real-world" means: is gmail fast enough? is opening a new tab fast? is image rendering fast enough? is html video fast enough? is the occasional embellished html5 animation fast enough? is typing into the address bar fast enough?
I'm sure their diagramming app is cool and everything, but I don't think I've ever seen anyone use anything like it, so I'm not sure what is "real world" about using it for a benchmark.
They even said that they altered the test in the middle to fix IE's performance problem. Come on.
There aint no pancake so thin it doesn't have two sides.
The way this benchmark measures "intra-frame time" is broken. In particular, it uses a setInterval with a 1ms delay. No browser actually respects that 1ms. Chrome clamps it to 5ms; others clamp it to 10ms, all to avoid the website thrashing the CPU pointlessly.
The upshot is that Chrome's interframe delay in the graph is about 5ms and Firefox 3.6's interframe delay is aboug 10ms. Which this particular benchmark can't tell apart from "no delay at all", given its methodology.
Firefox 4 beta, IE9 beta, Safari, and Opera seem to have delays greater than 10ms, so they're clearly doing some work they can't finish in 10ms.... or have slightly buggy timer implementations. Or both.
Of course in practice frame rates above 60fps or so are pointless since the screen doesn't redraw that often. ;)
On the other hand, on Mac, on modern hardware, I get 4.5fps in Chrome 7 dev on a random trial document I just tried, with JS render tiems on the order of 7ms (with a 7ms standard deviation) and "intra-frame time" of 224ms with a 900ms standard deviation (yes, those numbers are nuts). Firefox 4 beta comes in at about 11s for the JS (with 3ms stddev) and 125ms for the "intra-frame time" (with a claimed stddev of 0, which looks really suspicious).
It'd be nice if there were non-obfuscated source for this benchmark so its number-crunching could be evaluated; that 0 stddev is ... highly improbable.
This benchmark can be run by anyone in LucidChart. First, sign up for a free account here.
Nuff said
TFA (yes, I actually read it) says: "Firefox 4.0 Beta 6 came in behind all other browsers except for IE8". That's quite different from "dead last".
The real "Libtards" are the Libertarians!
I use Ubuntu 10.04. I have mostly switched to chrome (not completely; there are still sites that don't work properly with it). My problem with firefox was memory usage. I tend to have *lots* of tabs open and I often don't reboot for weeks. Firefox memory usage creeps up over time and my laptop slows. I keep reading that this is no longer supposed to happen, but it happens to me. Chrome with a comparable number of open tabs does not slow everything else down.
If Firefox were better behaved I would stick with Firefox.
(gmail passwords are now randomly rejected in FF, but work in any other browser).
Yeah, that would be the phishing malware screwing with you.
I use Firefox and IE regularly, have played with Chrome, and occasionally use Safari on the Macs at work.
I honestly can't notice any difference between any of them in rendering speed.
99.99% of the time, web browsing performance is network-limited anyway.
Surely standards support and browser stability are more important features, at least on platforms with more grunt than an iphone?
Any sufficiently advanced technology is indistinguishable from a rigged demo
--Andy Finkel (J. Klass?)
I'm not sure how they get off calling this a "real world benchmark", as it seems to bear almost no resemblance to what people normally use web browsers for: "The benchmark works by simply dragging a part of the diagram around the page for five seconds." WTF?
It certainly doesn't seem to be any more useful than the other browser benchmark being touted these days, and arguably it's much less useful, because it measures a single very narrow aspect of browser operation, one which has little connection with typical browser usage.
Moreover, the slashdot summary seems to go to great lengths to emphasize how "badly" FF4 did on this (useless, remember) benchmark, and to pump up IE9: "The results were surprising. IE9 held its own pretty well (with a few caveats), and the latest Firefox 4 beta came in dead last" -- but if you actually look at the results that emphasis is misplaced: almost all the browsers were quite close to each other, with a few outliers, but in no cases was FF4 an outlier, and indeed was pretty much identical to IE9 (on this test).
The only clear result I can see is: When doing a certain very specific type of javascript rendering, most modern browsers have pretty much identical performance, though chrome's particularly fast, and IE8 particularly slow.
Of course, that isn't very interesting to anybody except LucidChart users, of course, nor very likely to generate any controversy...
We live, as we dream -- alone....
SRWare Iron was created for the sole purpose of earning the "creator" some money on ad revenue. To borrow from my previous post on the subject:
Everyone mentioning SRWare Iron should know about this little tidbit: The story of Iron. The article and the linked IRC log tell a very interesting story about a guy less concerned with having a good reason to fork and more concerned with making money off of adsense and publicity for creating a "privacy-respecting" Chrome which is basically a perpetually outdated Chromium with a few checkboxes in "Under the Hood" defaulting to off.
The guy who runs that blog does not try to hide the fact that he's a Chrome developer, and he admits that there is the highly unlikely possibility that the person who was asking these questions was not the person who went on to release Iron. I was skeptical as well until I checked out the log file itself and quite honestly it would have to be an incredible coincidence for this guy to be asking such questions and providing the information that he does in his attempts to glean information on the right way to advertise his product as well as how to go about renaming the executable. There's more that makes it very reasonable to believe this is the guy who went on to release Iron, so please don't dismiss it until you've checked out the log file in detail. If this was a supremely unnecessary and elaborate hoax it sure is pulled off convincingly.
Using Iron after reading this information made me feel like I was supporting the wrong guy here and I couldn't do it anymore, it was just too uncomfortable seeing that this guy was looking for adsense revenue and to make a name for himself. The attitude of this developer is not one I would encourage at all.
"We invented personal computing." - Bill Gates
Really, fuck it. I've had it with corporate-sponsored dick-fighting contest about which browser is the fastest. I really, really couldn't care any less. Features, openness, security, standards compliance, yeah. But If I want a fast app, I'll go native, thank you. Maybe I'm too old, but I've always thought HTML sucked as a programming paradigm. As an information distribution mechanism, sure. But for interactivity? Please. It's about time somebody called bullshit on this. Hell, a goddamn Visual Basic app from fifteen years ago kicked the butt of most modern web sites in usability, performance and ease of maintenance. The only thing that makes the web so attractive is the barrier to entry : free, nothing to install, immediate access to the average brains of millions. Just like TV. No thanks for dumbing it down to this. And now you wanna make it faster? Piss off. Go write real code that does something, not just another abstraction layer.
-- Home is where you eat your heart out.
Alternatively just go to the Options page in google's Chrome version, and uncheck "Use a suggestion service to help complete searches and URLs typed in the address bar". Crome stops the 'snooping' then.
I guess the actual selection of versions shows how the point of the article was more about bashing FF4 compared to IE9 (in which it also failed, given the very small difference between them) rather than doing a honest comparison of all of the browsers.
"I'm never quite so stupid as when I'm being smart" (Linus van Pelt)