It's curious, given your theory, that there are all sorts of tests in that benchmark on which Firefox does worse than other browsers.
But in general, you do expect that if a vendor thinks something is important enough to benchmark they also think that it's important to optimize for. Every single other browser vendor that has released a benchmark has created it, optimized for it, _then_ released it (much more so than is the case for Kraken).
I'm not smoking anything. I'm just describing my personal experience, as well as that of various people who've sat down and meaured this. See for example:
Ah.. Firebug definitely has memory leaks where it grabs onto things and "caches" them for the lifetime of the process, especially if you actually use it. But even just having it installed will leak some.
The overall sluggishness is a separate issue; I agree that it's there, and it's being worked on.
Are you comparing your 11180.7 number to the numbers in the article submission to conclude that "Chrome is still king"? Or did you run other browsers on your hardware and OS? The hardware and OS _do_ make a difference, you know... especially that hardware bit.
Interesting. One question. Is this a Firefox profile with extensions installed? If so, which ones? Whenever I've looked over here (Linux and Mac), recent Firefox has used less memory than recent Opera....
You forgot the part about Apple winning in their test (sunspider), and the curious cache of the values of the sin() function in their JS engine that just happens to be the right size for that test.
Fundamentally, browser makers optimize their engine for what they consider important. They also put the things they consider important into benchmarks. The result is somewhat predictable.
Now Peacekeeper is an interesting mention, except I've actually looked at its code. This is a benchmark that measures things like 10,000 calls each of which removes 20 elements from an array that starts with 100,000 elements. It has (failed, interestingly) attempts to browser-sniff and run different code in different browsers. I wouldn't take its numbers to mean much of anything, in general, without some careful study of the exact tests you're looking at. Of course it's also measuring a lot more than just JavaScript; in that sense it's better than most of the benchmarks out there, if you think it manages to correctly measure the things it claims it's measuring.
One other thing, by the way: I fully expect that on Kraken shipping Chrome and Opera are faster than Firefox 3.6.
Because the amount of time it takes to run the javascript on the top sites is pretty small (which is what the IE team was talking about around IE8's release). Performance on those sites mostly doesn't depend on whether your JS engine is the one in Chrome dev or the one in IE7. I only say "mostly" because I wouldn't be surprised if gmail is in the top 200.;)
If you're going to worry specifically about JS performance (which is an assumption; the IE team is still saying that this focus is a mistake and to some extent they're right), you want to be benchmarking things that are gated on JS performance. That means identifying t the things that are slow with current JS engines and that people would like to be doing but can't because of said slowness, whatever those things are, and benchmarking those.
Well... Mozilla _has_ concentrated on low RAM usage in the past. The actual memory usage of Gecko is significantly lower than its competitors if you load some pages and measure it.
At this point, they're actually trading off space for performance (e.g. making some core objects slightly bigger to improve certain performance characteristics).
70% share is not the worry; that's about where IE was a year ago.
It's 90+% that would be a problem; at that point MS could hold back HTML5 by simply not implementing any more of it, since clearly they don't need it to get back market share.
As for adoption.... I'm for adoption of whatever that leaves no rendering engine (as opposed to browser, note) with more than 50% of the market.
> I don't see Flash going anywhere for at least a decade
No one cares that Flash exists. What's important is that it be possible to develop tomorrow's web sites without having to use Flash, and that it be possible to browse the web at least somewhat reasonably without having Flash (e.g. not all sites need to work, but there should be sites in a given category that work without Flash). That's a somewhat realistic goal right now; for example very few banks require Flash (though some do).
> Silverlight won't have the install base of HTML5
The goal is to keep it that way, yes.
> Apple doesn't have enough influence to change the direction of the web.
You apparently haven't had to deal with the "if it's on a cell phone it must be Webkit" mindset of developers of "mobile" sites. See the part dealing with -webkit-text-size-adjust at http://blogs.msdn.com/b/iemobile/archive/2010/05/10/javascript-and-css-changes-in-ie-mobile-for-windows-phone-7.aspx which Microsoft was forced to take out later. Note that there have been calls for Gecko to similarly add support on mobile for some of the -webkit-* stuff Apple has been pushing people to use. Those calls have been resisted so far, but as for the future.... who knows.
The war now is, generally speaking against Flex, Silverlight, and some of the things Apple seems to want to do with Webkit that are much like IE back in the IE4/5 days.
That is, now that we don't have a monopoly we'd really like it to _stay_ that way.
When viewing scaled video, it's a huge factor. And when using web applications (as opposed to reading the news) it's a significant factor. Oh, and when scrolling, not like anyone ever does that with webpages.
And we're not talking tenths of a millisecond here. If each scroll operation takes you 200ms (easy to run into without hardware acceleration on some sites out there that are sticking video or large translucent images over fixed-position backgrounds), you just lose.
> I've never noticed whether my browser has fast, or slow, or any GPL acceleration.
The #1 reason people give for switching browsers over the last few years, no matter what they're switching from and to, is that the old one was slow... So you're very much not representative of the market, sadly.
> OK, so the display portion that takes milliseconds
I've done a lot of various Gecko-based browsers in the last 10 years. At this point, for your typical "this page is being slow" case, 30-80% of the time is spent painting. The hardware accelerated builds speed this up a lot.
And do note that painting happens 60 times per second, so as soon as it takes more than 16ms or so you start seeing lag.
Er, slashdot totally munged the source there. The relevant function is:
998 nsresult 999 nsHTMLFormElement::WalkFormElements(nsFormSubmission* aFormSubmission) 1000 { 1001 nsTArray<nsGenericHTMLFormElement*> sortedControls; 1002 nsresult rv = mControls->GetSortedControls(sortedControls); 1003 NS_ENSURE_SUCCESS(rv, rv); 1004 1005 // 1006 // Walk the list of nodes and call SubmitNamesValues() on the controls 1007 // 1008 PRUint32 len = sortedControls.Length(); 1009 for (PRUint32 i = 0; i < len; ++i) { 1010 // Tell the control to submit its name/value pairs to the submission 1011 sortedControls[i]->SubmitNamesValues(aFormSubmission); 1012 } 1013 1014 return NS_OK; 1015 }
> Unless firefox has magically cleaned up its entire tree in the past 4 years
So about half the life of the project? A lot of cleanup has in fact happened, yes.
> Was there a nice neat "Form Business Logic" object somewhere that was triggered with a > list of all the stuff in the form that simply iterated over a standard > get-me-your-submit-value interface for each?
You mean like this function?
nsHTMLFormElement::WalkFormElements(nsFormSubmission* aFormSubmission)
{...
PRUint32 len = sortedControls.Length();
for (PRUint32 i = 0; i SubmitNamesValues(aFormSubmission);
}
Sure looks like a standard interface for each control that tells it to add itself to the data being submitted. Just querying the "submit value" wouldn't work as well, since some of the values are strings and some are sets of multiple strings and some are stream objects representing files on disk.
Oh, and this code has looked like this since 2002, so it was certainly like this in 2006 when you say you looked at it.
> The click event kinda walked up the tree to see if there was a form to submit somewhere > up there
Are you and I looking at the same code? The default click event handling on a submit control just fires a submit event at its mForm member. As long as your mForm is set right, it'll Just Work even if the isn't inside the in the DOM. Which is no surprise, because the way HTML parsing works that happens pretty often. So it has to be supported.
You do have to have the mForm member set correctly, of course.
> which allows you to set the destination on the fly from the button) to do essentially a > getElementById() on it.
Sounds like you were doing this completely backwards, for what it's worth. The right way to handle [input form="something"] is to set the mForm based on the "something". Then everything else should more or less Just Work. The complications arise from the fact that the element "something" references can change; in 2006 there was no good general system for keeping track of such changes; now there is.
> That and the fact that you needed an assembler
Unlike what other open-source layout engines? Heck, chromium includes a third-party assembler as part of their source checkout because they depend on it for vp8 stuff.
And on the other hand, if you're compiling C++ I figure you have an assembler installed already, right? So what's the issue?
> python, gawk AND perl
Yeah, the fact that multiple scripting languages are used in the build system is somewhat of an annoyance if you change the build system and reflects the fact that different scripting languages were popular in 1998 and 2006 and that rewriting a bunch of perl in python is low on the priority list. How does that affect the readability or editability of the codebase, though? Unless, as I said, you're changing the perl parts of the build system, in which case you have to read perl.
What's interesting about conformance tests is that unless they're exhaustive the only thing you can tell from them is how much a browser cares about conformance... by looking at the score when the test is first published, before people go and fix just the issues that are tested for.
As far as I know Opera is not "cheating" on its sputnik results but is in fact "cheating" (in the effect of implementing effectively bare-minimum functionality needed to pass) on some of the Acid3 bits. Precisely the ones Firefox is not passing, as it happens.
Like most "well known facts", this one is more of an urban legend than anything else. Whatever you may think of the Mozilla codebase (and it's not as painful to work with as people seem to think; Webkit used to be somewhat simpler when it did a lot less and got more complicated as it gained feature parity), the JS engine was never a horrible mess any more so than any other JS engine I've read the source for. And I've read the source of at least V8, JSC, and Spidermonkey at some length.
One thing it _did_ have against it is that it wasn't designed from the ground up to work with a jit, unlike V8. And reworking an existing codebase to a new design while keeping API compatibility (which Spidermonkey has aimed to do until recently) is much harder than writing something from scratch. For example, some optimizations that make a lot of sense in an interpreter are actually slowdowns when you're using a jit; several such had to be effectively undone before the jit could get faster.
For example, no one currently making a browser actually wants to make a browser per se. They all have an agenda they would like to push, using their browser as leverage.
> Long standing problems that may or may not be fixed in the 4.0 branch
You mean long-standing problems like not being able to print preview in Chrome? Or long-standing problems like broken CSS selector matching in Chrome (and other Webkit browsers)?
Complex software tends to have some long-standing problems. A number of such got fixed in Firefox 4.0; not all of them did. That doesn't equate to no work happening.
I also disagree with your characterization of the video tag situation, but only time will tell which of us is right.
Slashdot Mirror
Well, sure. Just because they have an agenda doesn't mean they're saying things that are completely false, though.
Is the performance difference across the board, or is it particularly noticeable on some subtests?
It's curious, given your theory, that there are all sorts of tests in that benchmark on which Firefox does worse than other browsers.
But in general, you do expect that if a vendor thinks something is important enough to benchmark they also think that it's important to optimize for. Every single other browser vendor that has released a benchmark has created it, optimized for it, _then_ released it (much more so than is the case for Kraken).
I agree this is a real lack; I think EcmaScript 5 is working on addressing it.
I'm not smoking anything. I'm just describing my personal experience, as well as that of various people who've sat down and meaured this. See for example:
http://cybernetnews.com/browser-comparison-internet-explorer-firefox-chrome-safari-opera/ (search for "memory usage tests" in the page).
http://www.zdnet.com/blog/hardware/browser-memory-usage-the-good-the-bad-and-the-down-right-ugly/2024
http://www.favbrowser.com/browser-memory-ram-usage-firefox-35-rc-safari-4-opera-10-beta-google-chrome-30-dev/ (ignore the Chrome bit, because they were adding up memory used by processes that actually have some memory mappings shared)
http://lifehacker.com/5457242/browser-speed-tests-firefox-36-chrome-4-opera-105-and-extensions (seach for "memory use, no extensions" and "Memory use with extensions") as well as the other tests lifehacker has done (e.g. follow the "last batch of browser tests" from that page)
So you tell me, what am I smoking and how did I get the rest of the world to smoke it too?
Ah.. Firebug definitely has memory leaks where it grabs onto things and "caches" them for the lifetime of the process, especially if you actually use it. But even just having it installed will leak some.
The overall sluggishness is a separate issue; I agree that it's there, and it's being worked on.
Are you comparing your 11180.7 number to the numbers in the article submission to conclude that "Chrome is still king"? Or did you run other browsers on your hardware and OS? The hardware and OS _do_ make a difference, you know... especially that hardware bit.
> So yeah, I'd expect FF 3.6 to be the slowest of the bunch.
Right. Which is consistent with other JS tests, so no mysteries or benchmark-skewing by Mozilla needed there... ;)
> Actually, there are probably several empty leagues between FF/Chrome/Safari/Opera and IE
That may or may not be the case with the IE9 betas, depending on which benchmarks you use. Though there's some weirdness; see http://blog.mozilla.com/rob-sayre/2010/09/09/js-benchmarks-closing-in/ the paragraph starting "One last issue that can crop up".
Interesting. One question. Is this a Firefox profile with extensions installed? If so, which ones? Whenever I've looked over here (Linux and Mac), recent Firefox has used less memory than recent Opera....
You forgot the part about Apple winning in their test (sunspider), and the curious cache of the values of the sin() function in their JS engine that just happens to be the right size for that test.
Fundamentally, browser makers optimize their engine for what they consider important. They also put the things they consider important into benchmarks. The result is somewhat predictable.
Now Peacekeeper is an interesting mention, except I've actually looked at its code. This is a benchmark that measures things like 10,000 calls each of which removes 20 elements from an array that starts with 100,000 elements. It has (failed, interestingly) attempts to browser-sniff and run different code in different browsers. I wouldn't take its numbers to mean much of anything, in general, without some careful study of the exact tests you're looking at. Of course it's also measuring a lot more than just JavaScript; in that sense it's better than most of the benchmarks out there, if you think it manages to correctly measure the things it claims it's measuring.
One other thing, by the way: I fully expect that on Kraken shipping Chrome and Opera are faster than Firefox 3.6.
Because the amount of time it takes to run the javascript on the top sites is pretty small (which is what the IE team was talking about around IE8's release). Performance on those sites mostly doesn't depend on whether your JS engine is the one in Chrome dev or the one in IE7. I only say "mostly" because I wouldn't be surprised if gmail is in the top 200. ;)
If you're going to worry specifically about JS performance (which is an assumption; the IE team is still saying that this focus is a mistake and to some extent they're right), you want to be benchmarking things that are gated on JS performance. That means identifying t the things that are slow with current JS engines and that people would like to be doing but can't because of said slowness, whatever those things are, and benchmarking those.
Well... Mozilla _has_ concentrated on low RAM usage in the past. The actual memory usage of Gecko is significantly lower than its competitors if you load some pages and measure it.
At this point, they're actually trading off space for performance (e.g. making some core objects slightly bigger to improve certain performance characteristics).
It's impossible today, yes. A bit of thinking ahead to two years from now is sometimes warranted, though.
70% share is not the worry; that's about where IE was a year ago.
It's 90+% that would be a problem; at that point MS could hold back HTML5 by simply not implementing any more of it, since clearly they don't need it to get back market share.
As for adoption.... I'm for adoption of whatever that leaves no rendering engine (as opposed to browser, note) with more than 50% of the market.
> I don't see Flash going anywhere for at least a decade
No one cares that Flash exists. What's important is that it be possible to develop tomorrow's web sites without having to use Flash, and that it be possible to browse the web at least somewhat reasonably without having Flash (e.g. not all sites need to work, but there should be sites in a given category that work without Flash). That's a somewhat realistic goal right now; for example very few banks require Flash (though some do).
> Silverlight won't have the install base of HTML5
The goal is to keep it that way, yes.
> Apple doesn't have enough influence to change the direction of the web.
You apparently haven't had to deal with the "if it's on a cell phone it must be Webkit" mindset of developers of "mobile" sites. See the part dealing with -webkit-text-size-adjust at http://blogs.msdn.com/b/iemobile/archive/2010/05/10/javascript-and-css-changes-in-ie-mobile-for-windows-phone-7.aspx which Microsoft was forced to take out later. Note that there have been calls for Gecko to similarly add support on mobile for some of the -webkit-* stuff Apple has been pushing people to use. Those calls have been resisted so far, but as for the future.... who knows.
The real war 5 years was against ActiveX and IE6.
The war now is, generally speaking against Flex, Silverlight, and some of the things Apple seems to want to do with Webkit that are much like IE back in the IE4/5 days.
That is, now that we don't have a monopoly we'd really like it to _stay_ that way.
When viewing scaled video, it's a huge factor. And when using web applications (as opposed to reading the news) it's a significant factor. Oh, and when scrolling, not like anyone ever does that with webpages.
And we're not talking tenths of a millisecond here. If each scroll operation takes you 200ms (easy to run into without hardware acceleration on some sites out there that are sticking video or large translucent images over fixed-position backgrounds), you just lose.
> I've never noticed whether my browser has fast, or slow, or any GPL acceleration.
The #1 reason people give for switching browsers over the last few years, no matter what they're switching from and to, is that the old one was slow... So you're very much not representative of the market, sadly.
> OK, so the display portion that takes milliseconds
I've done a lot of various Gecko-based browsers in the last 10 years. At this point, for your typical "this page is being slow" case, 30-80% of the time is spent painting. The hardware accelerated builds speed this up a lot.
And do note that painting happens 60 times per second, so as soon as it takes more than 16ms or so you start seeing lag.
Er, slashdot totally munged the source there. The relevant function is:
998 nsresult
999 nsHTMLFormElement::WalkFormElements(nsFormSubmission* aFormSubmission)
1000 {
1001 nsTArray<nsGenericHTMLFormElement*> sortedControls;
1002 nsresult rv = mControls->GetSortedControls(sortedControls);
1003 NS_ENSURE_SUCCESS(rv, rv);
1004
1005 //
1006 // Walk the list of nodes and call SubmitNamesValues() on the controls
1007 //
1008 PRUint32 len = sortedControls.Length();
1009 for (PRUint32 i = 0; i < len; ++i) {
1010 // Tell the control to submit its name/value pairs to the submission
1011 sortedControls[i]->SubmitNamesValues(aFormSubmission);
1012 }
1013
1014 return NS_OK;
1015 }
> Unless firefox has magically cleaned up its entire tree in the past 4 years
So about half the life of the project? A lot of cleanup has in fact happened, yes.
> Was there a nice neat "Form Business Logic" object somewhere that was triggered with a
> list of all the stuff in the form that simply iterated over a standard
> get-me-your-submit-value interface for each?
You mean like this function?
nsHTMLFormElement::WalkFormElements(nsFormSubmission* aFormSubmission) ...
{
PRUint32 len = sortedControls.Length();
for (PRUint32 i = 0; i SubmitNamesValues(aFormSubmission);
}
Sure looks like a standard interface for each control that tells it to add itself to the data being submitted. Just querying the "submit value" wouldn't work as well, since some of the values are strings and some are sets of multiple strings and some are stream objects representing files on disk.
Oh, and this code has looked like this since 2002, so it was certainly like this in 2006 when you say you looked at it.
> The click event kinda walked up the tree to see if there was a form to submit somewhere
> up there
Are you and I looking at the same code? The default click event handling on a submit control just fires a submit event at its mForm member. As long as your mForm is set right, it'll Just Work even if the isn't inside the in the DOM. Which is no surprise, because the way HTML parsing works that happens pretty often. So it has to be supported.
You do have to have the mForm member set correctly, of course.
> which allows you to set the destination on the fly from the button) to do essentially a > getElementById() on it.
Sounds like you were doing this completely backwards, for what it's worth. The right way to handle [input form="something"] is to set the mForm based on the "something". Then everything else should more or less Just Work. The complications arise from the fact that the element "something" references can change; in 2006 there was no good general system for keeping track of such changes; now there is.
> That and the fact that you needed an assembler
Unlike what other open-source layout engines? Heck, chromium includes a third-party assembler as part of their source checkout because they depend on it for vp8 stuff.
And on the other hand, if you're compiling C++ I figure you have an assembler installed already, right? So what's the issue?
> python, gawk AND perl
Yeah, the fact that multiple scripting languages are used in the build system is somewhat of an annoyance if you change the build system and reflects the fact that different scripting languages were popular in 1998 and 2006 and that rewriting a bunch of perl in python is low on the priority list. How does that affect the readability or editability of the codebase, though? Unless, as I said, you're changing the perl parts of the build system, in which case you have to read perl.
What's interesting about conformance tests is that unless they're exhaustive the only thing you can tell from them is how much a browser cares about conformance... by looking at the score when the test is first published, before people go and fix just the issues that are tested for.
As far as I know Opera is not "cheating" on its sputnik results but is in fact "cheating" (in the effect of implementing effectively bare-minimum functionality needed to pass) on some of the Acid3 bits. Precisely the ones Firefox is not passing, as it happens.
Register allocation is handled separately on x86-64 and x86, yes.
Like most "well known facts", this one is more of an urban legend than anything else. Whatever you may think of the Mozilla codebase (and it's not as painful to work with as people seem to think; Webkit used to be somewhat simpler when it did a lot less and got more complicated as it gained feature parity), the JS engine was never a horrible mess any more so than any other JS engine I've read the source for. And I've read the source of at least V8, JSC, and Spidermonkey at some length.
One thing it _did_ have against it is that it wasn't designed from the ground up to work with a jit, unlike V8. And reworking an existing codebase to a new design while keeping API compatibility (which Spidermonkey has aimed to do until recently) is much harder than writing something from scratch. For example, some optimizations that make a lot of sense in an interpreter are actually slowdowns when you're using a jit; several such had to be effectively undone before the jit could get faster.
> Christian missionaries do that, too
Sure. So does everyone.
For example, no one currently making a browser actually wants to make a browser per se. They all have an agenda they would like to push, using their browser as leverage.
> Long standing problems that may or may not be fixed in the 4.0 branch
You mean long-standing problems like not being able to print preview in Chrome? Or long-standing problems like broken CSS selector matching in Chrome (and other Webkit browsers)?
Complex software tends to have some long-standing problems. A number of such got fixed in Firefox 4.0; not all of them did. That doesn't equate to no work happening.
I also disagree with your characterization of the video tag situation, but only time will tell which of us is right.