Slashdot Mirror
Clashing Scores In the HTML5 Compatibility Test Wars
Andreas(R) writes "Microsoft has published a set of HTML5 tests comparing Internet Explorer 9 to other web browsers. In Microsoft's own tests, IE9 performs 100% on all tests. However, the Internet Explorer 9 HTML5 Canvas Campaign has published results that show that Internet Explorer gets 0% on all their tests." The results reported here are selected with tongue in cheek: "Therefore, we'll also present shameless results from tests which have been carefully selected to give the results that the PR department has demanded."
...with MS HTML# 5.0
93rd rule of Slashdot: No matter how obvious my sarcasm is, my comment will be taken seriously by someone.
First off if this is a technical discussion, we should probably be talking about layout engines -- not browsers. Secondly their HTML5 capabilities are well documented. You can come up with whatever perventage you want from those charts as some things (Video) might be deal breakers compared to others (MathML).
My work here is dung.
Yeah that's certainly entertaining and all but...do we have any real tests with real results?
And by "real" I mean tests that includes all HTML5 specifications...
When you can't beat em, change the rules.
The real Sig captains the Northwestern. This one captains
Well, let knowledgeable slashdotters point us novices to a set of a "standard" HTML5 test site to which we can run and establish the fact.
Ohh wait, I forgot that there is yet to be any agreement on the HTML5 standard itself! This is why I think Apple is just bluffing with their campaign against Flash. It also demonstrates the weaknesses we all have to work around.
Clearly, the independent, third-party tests are flawed. Microsoft would never create a biased benchmarking test to promote their own product.
Seriously though? The only people that understand what HTML5 is and what these results actually mean are going to understand that it is complete nonsense.
TFA: "The first table is a summary of the test results with the May 2010 IE Platform Preview and each of the major shipping browsers running on Windows."
So...IE8 isn't a "major shipping browser" that runs on Windows?
If IE8 scores so terribly that Microsoft is embarrassed to post its scores, that's fine, but it would be less dishonest and more informative then to include recent betas of their competitors' browsers in addition to the latest shipping version.
High-speed Road Trip (18.000KPH)
In other news, something surprising happened.
Coming up next: are angry hobos stalking your Facebook account? What you don't know might eat you. Film at 11.
That's like saying slashdotters are 100% successful sexually.
If the tests that include the opposite sex are excluded.
...that they benchmarked IE trunk against OLD versions of other browsers. They didn't even use Chrome 5.0!
In some places it's a significant difference.
I also did some benchmarks of my own on non-Microsoft controlled sites. See the first comment on that page for results. Suffice it to say IE9 has improved since IE8 but still has a ways to go.
Fuck that. Car analogy.
It's like testing several cars with similar performance on a very long, winding racetrack covered with obstacles and comparing their lap times to that of another car that drives around a very short and straight loop.
It's easy to say who gets first place when you get to choose the conditions of winning. Doesn't matter, though, the point here isn't to make Internet Explorer more popular, it's to make Microsoft look like they're competing fairly and remaining relevant.
"Most people, I think, don't even know what a rootkit is, so why should they care about it?"
Where's Bad Analogy Guy when you need him?
Yeah we all knew that already but when I went to the site a few days ago, when hackernews showed it, it claimed chrome was failing tests where you could see with your own eyes it was succeeding. Fuck Microsoft and their bashing other browsers when theres still falls flat on all other tests /rant
If I was witty I'd put something funny here but, as it stands, I am not and have just wasted seconds of your life
Like IE8, IE7 and IE6 before them, windows users will be forced to upgrade to 9 sooner or later anyway. You can bet it is likely part of Windows 7 SP1 or SP2.
I run: Windows, OS X, Linux, FreeBSD. Just because you have a hammer, doesn't mean everything is a nail.
http://html5test.com/
things like this will have to do until we see something like ACID support HTML5.
I work for the Department of Redundancy Department.
We have a winner! :D
Cheers,
"What in the name of Fats Waller is that?"
"A four-foot prune."
Discussion of test results
Based on the tests that we have performed, it is very clear that there is a very big difference between the best and worst browsers. Therefore we can only conclude that the results are valid and true.
Now if that isn't a rigorous application of the scientific method I don't know what is!
Microsoft is 100% Microsoft compatible (restrictions and exclusions apply).
this was pointless.
Here's a difference that everyone should note. When the later Acid tests were formulated they were written by Webkit and Gecko developers and were specifically biased against those engines. If one of the two did not fail, it didn't go in. That way it motivates them to improve. When MS writes a test suite it's biased in favor of their engine, so they can claim to be "ahead" and have no motivation to improve. It's an excellent example of who values technical excellence and who values marketing.
No, where's Pizza Analogy Guy when you need him?
His Analogy would hit that bullseye, then the rest of the dominoes will fall like a house of cards. Checkmate.
The Kruger Dunning explains most post on
I'm not sure about the part about the tests seeming legit, but I do think you make a good point that the tests are at least legit enough to cast light on the shortcomings of the other browsers. But mainly, I want to express my appreciation that you didn't just post another version of the standard (and boring and too easy) Slashdot response. So thank you for actually trying to focus this discussion on the issues. I hope you're not flamed or just ignored, but I'm not holding my breath.
I am adequately shocked and awed by how Microsoft's own testing shows that IE9 will be superior to any other major browser out today! ... except that IE9 isn't out today, and won't be for months.
Either tell me how well IE8 handles those same tests in comparison to Firefox 3.6, or give me a comparison of IE9 to Firefox 4. Otherwise, it's just fluff.
Did any of you 'holier than thou' slashdotters actually evaluate the tests that Microsoft published before ripping them apart? They seem legit so far, and they are pointing out issues that SHOULD be fixed in the other browsers as well.
If so, then good for them. It would be a dramatic change in behavior, given the complete disregard for standards in previous versions of IE.
There is an obvious methodology problem with this test that makes its result questionable: They are testing an unreleased product (IE9) against the shipping version of other browsers. That is like bragging that a PC that will ship in nine months is faster than one that has been on sale for a year. Other browsers are actively working on the same issues, and will be better by the time any real users get IE9. Firefox and Chrome are open source, and provide bleeding edge builds that they could have tested. They chose not to test the shipping version of their own browser.
Being skeptical that the company with the worst track record in standards compliance has completely changed based on a benchmark of their own choosing, comparing an alpha product against stable versions of competing products, is not "holier than thou". It is common sense.
Well, Bad Analogy Guy is kind of like a car. And the radio only gets two stations on AM, but there's an eight track with a copy of "Journey's Greatest Hits" stuck in it. If you look at it that way then this discussion is something like an eight hour drive from Tulsa, OK to one of the Portlands. I can't remember which one, but it's eight hours away by car. Now the car has wood grain paneling on the right side and some kind spray-on granite countertop on the left, so the driver can lean out of the window and chop tomatoes as long as the passenger leans over to take the wheel.
The rest of us are the two pedigreed schnoodles sitting in the back seat, trying to eat bacon and egg sandwiches.
Does that answer your question?
His Analogy would hit that bullseye, then the rest of the dominoes will fall like a house of cards. Checkmate.
Game, set and match.
... and then they built the supercollider.
It's all so clear now... my days of nihilism are finally over...
"Most people, I think, don't even know what a rootkit is, so why should they care about it?"
The reason why most tests failed with browsers other than IE:
1st) Since HTML5 is still in a very early state, many browsers (AKA Webkit, Gecko, Presto) used prefixes for most tags and CSS properties. Example: round borders is -moz-border-radius in gecko, and -webkit-border-radius in Chrome. Some latest versions have taken some out of beta and also read border-radius, but most still don't. IE obviously uses border-radius, and that's why other's don't work. ... except they are not, because they use HTML5 styles and tags, and they do not validate. Validator says: The document located at was tentatively checked as XHTML 1.0 Strict. This means that with the use of some fallback or override mechanism, we successfully performed a formal validation using an SGML, HTML5 and/or XML Parser(s). In other words, the document would validate as XHTML 1.0 Strict if you changed the markup to match the changes we have performed automatically, but it will not be valid until you make these changes.
2nd) The JS is tricky at best. Go and check it out. Lots of lines of code to perform a simple task, and those lines are carefully selected to fail in other browsers. I downloaded the tests, and they work on ALL browsers (I tested Chrome, Firefox and Opera, all on GNU/Linux, all on their latest version). That JS was crafted to fail on all browsers and work only on IE
3rd) I took the time to run the source of many of their scripts through the W3C validator. Most scripts have several warnings, some errors, etc. They DO NOT VALIDATE.
4th) The tests aren't really HTML5. Only the HTML5 tests are actual HTML5, the others are XHTML 1.0 strict
It's microsoft ... never forget about that. This is business as usual.
WTF am I doing replying to an AC at 5 A.M on a Friday night?
Sorry to reply to myself, but I forgot a few things:
1st: The actual ietestcenter fails validation with 12 errors and 6 warnings: http://validator.w3.org/check?uri=http://samples.msdn.microsoft.com/ietestcenter/&charset=(detect+automatically)&doctype=Inline&group=0
Including some serious ones, like no Character encoding specified.
None of the tests specify a character encoding either.
WTF am I doing replying to an AC at 5 A.M on a Friday night?
agreeded like whats with dev build of IE and an old build of Chrome?
epic sig..... ya i got nothing
There was a time when tabbed browsing was a new idea. If I remember correctly, IE was the last browser to get on board with it. In fact, I'm pretty sure that IE is the last browser to get on board with just about everything. Eventually, though, IE does come on board with everyone else (it may take years, but hey - possessing the lion's share of the market share has its advantages). I don't see any reason to believe that this will be any different with HTML5. And then (as now) the so-called 'Browser Wars' will boil down to a simple matter of personal preference.
Which browser does the best pizza graph?
Which does the best Paul Klee faces?
Which cuts up the image best for different resolutions, or multiple monitors?
Which one requires yellow cheese for best results?
--
Andrew Romanoff for Czar.
On what day do you offer the authentic Neapolitan pizza chef school?
At this point, no. I've wasted a lot of time with Microsoft and I don't really care to evaluate their marketing material anymore. I really am done with that circus.
Your last point still stands: developers of other layout engines should look at the results, maybe there's something new there.
If they wanted to be at all relevant, they should have tested their development version against the development versions of the other browsers. At the moment they're comparing where they are now, where Opera was in March where and Mozilla was in January. Fair.
...
The rest of us are the two pedigreed schnoodles sitting in the back seat, trying to eat bacon and egg sandwiches.
Does that answer your question?
You forgot the freaky right-wing rant!
We live, as we dream -- alone....
I don't really care about HTML5, until I can get box-shadow (they already have the rounded corners in 9). That saves so much extra crap in layout if you use images, or you've got to code some ugly hack if you use filters.
Too bad, so sad.
as the user "DigitalFeonix" wrote in comments on the link to the original post: "Really? FF3 got 100% on WebSockets?! I'd like to see the tests. Firefox won't support WebSockets until 4." This is ridiculous. For once Microsoft is working to get something good out of their labs, and is openly sharing data (like the say it out LOUD that they can't pass ACID3 further than 69/100 YET), we should give them a chance
MS needs to fuully separate the render engine fron the browser in a way so that they can be updated individually. There are features in the browser which users may like or dislike, may require or not. Let the browser handle things like plugins just provide hooks for compositing in the render engine. Let the browser handle security, etc. Let the render "render".
A fool throws a stone into a well and a thousand sages can not remove it.
Exactly...the IE9 team wants to show that the next version of IE will be an improvement over what's out there currently. That's the whole point of the IE9 project. There's no point in comparing current technologies...they already know IE8 won't pass and they've already run the comparisons against current browsers and they know how well it does against them. There's also no point in comparing the IE9 against nightly builds of various other browser engines, because none of them are complete, and there's better things to do with their time.
Has anyone else noticed that many of the "HTML 5 Selection" test cases only fail in Chrome because each test contains a function called checkSelectionAttributes() and when you remove the call to this function the test performs correctly.
IE 9 doesn't come close to emulating IE 6 correctly.
Who the hell wrote these stupid tests? Whoever it is was a fucking moron.
I went down the list and picked the first one that Firefox supposedly failed on... “Call select() on a text field.” Firefox fails this test? What the fuck? I’ve used select() on text fields many a time and Firefox supports it just fine. Something’s fishy. So.......... I hit up the test page and read the source code.
If there’s a javascript error, the test obviously fails. A bunch of the attributes of window.getSelection() are checked, and if those don’t match what the test expected, the test fails. So I added a descriptive message to each fail condition, ran it again, and came up with the following list of exceptions:
Umm... so let’s see...
In order to pass this test, after selecting some text, the anchor node must be null, the offset zero, the focus node null, its offset zero, and the range count must be zero.
WHAT THE FUCK?
http://www.w3.org/TR/html5/editing.html#selection-0:
They wrote the exact opposite of these conditions for what you should expect after selecting text.
Strangely enough, Opera 10.53 also “passed”.
This is beyond incompetent, it is sleazy. They wrote their test to call everything that IE fails at a “pass” and anything that does it correctly will “fail”.
The correct test results should be:
Alexander Peter Kristopeit bought his basement from his mommy for one dollar.
Yes.
They test lots and lots of semi-related attributes to verify that they are at the correct value. Missing one of them causes the test to fail. No description is given of what points caused the test to fail; it just gives the PASS/FAIL grade at the end.
For instance, the first test: “Add a range to the selection”. When they create a new range, the anchor node of the range is the <body> element. When they call range.selectNode(document.getElementById("p1")) then add that range to the selection, they then check to make sure that the start container of the range (which was originally the <body> element) is now identical to the anchor node of the selection (which is the <p id="p1"> element). If it isn’t, the test fails.
However, there is no reason why the anchor node of the range has to be the <p> element; the W3C just says that the start container and end container must be the same, and the <p> element is located inside the <body> element and its contents could also be selected using offsets. In Firefox, the start container of the range does become the <p> element when you select it with selectNode(). In Opera 10.53, the start container is still the <body> element, and the start and end offsets are merely changed to select the contents of the <p> element which is itself contained inside the <body> element. This causes the test to fail in Opera.
Not only that but I don’t see any reason why the anchor node of the selection returned by window.getSelection couldn’t also be the <body> element, so as long as the range it created and the selection it created ended up with the same root element, their test would fail to even detect the difference whether that was <p> or <body>.
Alexander Peter Kristopeit bought his basement from his mommy for one dollar.
1st) Since HTML5 is still in a very early state, many browsers (AKA Webkit, Gecko, Presto) used prefixes for most tags and CSS properties. Example: round borders is -moz-border-radius in gecko, and -webkit-border-radius in Chrome. Some latest versions have taken some out of beta and also read border-radius, but most still don't. IE obviously uses border-radius, and that's why other's don't work.
This is a technicality, but it's fair enough. The other browsers should unprefix their implementations. In some cases, they may be using prefixes because their prefixed versions actually behave differently from the final standard.
2nd) The JS is tricky at best. Go and check it out. Lots of lines of code to perform a simple task, and those lines are carefully selected to fail in other browsers. I downloaded the tests, and they work on ALL browsers (I tested Chrome, Firefox and Opera, all on GNU/Linux, all on their latest version). That JS was crafted to fail on all browsers and work only on IE
Give examples and justification. From looking at the test names, at least some of them seem fair. For instance, the tests of SVG in text/html fail in non-IE9 browsers because in fact, no other tested browser implements that feature. That's legitimate.
3rd) I took the time to run the source of many of their scripts through the W3C validator. Most scripts have several warnings, some errors, etc. They DO NOT VALIDATE.
This is totally irrelevant to the tests' legitimacy, unless you can point to an actual case where it causes an incorrect failure.
4th) The tests aren't really HTML5. Only the HTML5 tests are actual HTML5, the others are XHTML 1.0 strict ... except they are not, because they use HTML5 styles and tags, and they do not validate. Validator says: The document located at was tentatively checked as XHTML 1.0 Strict. This means that with the use of some fallback or override mechanism, we successfully performed a formal validation using an SGML, HTML5 and/or XML Parser(s). In other words, the document would validate as XHTML 1.0 Strict if you changed the markup to match the changes we have performed automatically, but it will not be valid until you make these changes.
It makes no difference. HTML5 and CSS define processing for documents regardless of doctype or validation status. Indeed, you want some tests that are deliberately invalid, to test error handling.
In summary, there may be legitimate objections to the test (obviously including selective coverage), but you haven't made them.
MediaWiki developer, Total War Center sysadmin
I think that was him, he probably got modded to oblivion so often his karma's in the Guatamalan sinkhole.
Free Martian Whores!
That none of the browsers is yet 100% ready for HTML5.
That word doesn't mean what you think it means.
I' running a version of Chrome 6 and Firefox 3.6. A pre-release of IE should be compared to the latest beta/alapha of the other browsers...
All the HTML5 tests I've seen to date runs well on one / two browsers and fail on others.... (Apple's works perfectly on safari, with video / VR / audio failing on Chrome, Opera doing horribly in the typography / image-effect tests and Firefox ion general doing just as bad as Opera, mostly failing in different areas...)
A set of tests / demoes trying to test each of the features of the standard would help a lot to properly asses browser's progress....
Microsoft should shut up and ship.
Until CSS3 is at least a recommendation rather than a draft, the CSS3-draft-related features remain proprietary extensions to CSS and the prefixes are appropriate.
Since the final standard does not exist yet, it is rather premature to remove prefixes on the assumption that the behavior (even if it matches the current draft) will match the final standard.
Until CSS3 is at least a recommendation rather than a draft, the CSS3-draft-related features remain proprietary extensions to CSS and the prefixes are appropriate.
Since the final standard does not exist yet, it is rather premature to remove prefixes on the assumption that the behavior (even if it matches the current draft) will match the final standard.
The features we're talking about, like border-radius, have reached Candidate Recommendation. According to informally-agreed-upon CSSWG process (see last sentence), prefixes can be dropped then. They don't have to wait until Recommendation – CSS 2.1 isn't even at REC, and won't reach it for some time. If IE has dropped a prefix after CR and another browser hasn't, it's fair to call that a standards-compliance win for IE.
Features specified in draft standards are not proprietary, in any case. And there is no "CSS3" standard: CSS 3 consists of a number of standards, which range from Proposed Recommendation (Selectors – further than CSS 2.1!) to not even written (like Math).
MediaWiki developer, Total War Center sysadmin