Slashdot Mirror
Clashing Scores In the HTML5 Compatibility Test Wars
Andreas(R) writes "Microsoft has published a set of HTML5 tests comparing Internet Explorer 9 to other web browsers. In Microsoft's own tests, IE9 performs 100% on all tests. However, the Internet Explorer 9 HTML5 Canvas Campaign has published results that show that Internet Explorer gets 0% on all their tests." The results reported here are selected with tongue in cheek: "Therefore, we'll also present shameless results from tests which have been carefully selected to give the results that the PR department has demanded."
...with MS HTML# 5.0
93rd rule of Slashdot: No matter how obvious my sarcasm is, my comment will be taken seriously by someone.
First off if this is a technical discussion, we should probably be talking about layout engines -- not browsers. Secondly their HTML5 capabilities are well documented. You can come up with whatever perventage you want from those charts as some things (Video) might be deal breakers compared to others (MathML).
My work here is dung.
When you can't beat em, change the rules.
The real Sig captains the Northwestern. This one captains
Well, let knowledgeable slashdotters point us novices to a set of a "standard" HTML5 test site to which we can run and establish the fact.
Ohh wait, I forgot that there is yet to be any agreement on the HTML5 standard itself! This is why I think Apple is just bluffing with their campaign against Flash. It also demonstrates the weaknesses we all have to work around.
TFA: "The first table is a summary of the test results with the May 2010 IE Platform Preview and each of the major shipping browsers running on Windows."
So...IE8 isn't a "major shipping browser" that runs on Windows?
If IE8 scores so terribly that Microsoft is embarrassed to post its scores, that's fine, but it would be less dishonest and more informative then to include recent betas of their competitors' browsers in addition to the latest shipping version.
High-speed Road Trip (18.000KPH)
...that they benchmarked IE trunk against OLD versions of other browsers. They didn't even use Chrome 5.0!
In some places it's a significant difference.
I also did some benchmarks of my own on non-Microsoft controlled sites. See the first comment on that page for results. Suffice it to say IE9 has improved since IE8 but still has a ways to go.
No, at least not before we have a real HTML5 spec.
Discussion of test results
Based on the tests that we have performed, it is very clear that there is a very big difference between the best and worst browsers. Therefore we can only conclude that the results are valid and true.
Now if that isn't a rigorous application of the scientific method I don't know what is!
Wait, what? No Windows Service Pack has ever forced an update of Internet Explorer; maybe NT 4.0 did as I can't remember that far back, but definitely nothing since Windows 2000 onwards. Windows XP SP3 will install fine with IE 6.0 (XP bundled version). They'd be breaking their own support policy by even doing so, as Microsoft commits to supporting the version of IE that is shipped with every Windows version for the lifetime of support for that OS release. Seriously, where do you trolls get your garbage? You're not picking exceptions, you're claiming shit that has never happened.
Here's a difference that everyone should note. When the later Acid tests were formulated they were written by Webkit and Gecko developers and were specifically biased against those engines. If one of the two did not fail, it didn't go in. That way it motivates them to improve. When MS writes a test suite it's biased in favor of their engine, so they can claim to be "ahead" and have no motivation to improve. It's an excellent example of who values technical excellence and who values marketing.
Well, Bad Analogy Guy is kind of like a car. And the radio only gets two stations on AM, but there's an eight track with a copy of "Journey's Greatest Hits" stuck in it. If you look at it that way then this discussion is something like an eight hour drive from Tulsa, OK to one of the Portlands. I can't remember which one, but it's eight hours away by car. Now the car has wood grain paneling on the right side and some kind spray-on granite countertop on the left, so the driver can lean out of the window and chop tomatoes as long as the passenger leans over to take the wheel.
The rest of us are the two pedigreed schnoodles sitting in the back seat, trying to eat bacon and egg sandwiches.
Does that answer your question?
The reason why most tests failed with browsers other than IE:
1st) Since HTML5 is still in a very early state, many browsers (AKA Webkit, Gecko, Presto) used prefixes for most tags and CSS properties. Example: round borders is -moz-border-radius in gecko, and -webkit-border-radius in Chrome. Some latest versions have taken some out of beta and also read border-radius, but most still don't. IE obviously uses border-radius, and that's why other's don't work. ... except they are not, because they use HTML5 styles and tags, and they do not validate. Validator says: The document located at was tentatively checked as XHTML 1.0 Strict. This means that with the use of some fallback or override mechanism, we successfully performed a formal validation using an SGML, HTML5 and/or XML Parser(s). In other words, the document would validate as XHTML 1.0 Strict if you changed the markup to match the changes we have performed automatically, but it will not be valid until you make these changes.
2nd) The JS is tricky at best. Go and check it out. Lots of lines of code to perform a simple task, and those lines are carefully selected to fail in other browsers. I downloaded the tests, and they work on ALL browsers (I tested Chrome, Firefox and Opera, all on GNU/Linux, all on their latest version). That JS was crafted to fail on all browsers and work only on IE
3rd) I took the time to run the source of many of their scripts through the W3C validator. Most scripts have several warnings, some errors, etc. They DO NOT VALIDATE.
4th) The tests aren't really HTML5. Only the HTML5 tests are actual HTML5, the others are XHTML 1.0 strict
It's microsoft ... never forget about that. This is business as usual.
WTF am I doing replying to an AC at 5 A.M on a Friday night?
While you are sitting back and waiting for someone else to write these comprehensive test suites the only entity really taking the W3C Test Suite projects seriously at all is Microsoft. They've submitted thousands of test cases for CSS2.1 and have been working to submit hundreds of tests for CSS3.0 and HTML5. These are the very tests that are being disparaged here. It can be claimed that Microsoft is stacking these tests intentionally, except that these tests are publically available for comment and scrutiny. The Test Suite projects are open for submissions by others as well so Google and Mozilla and anyone else is free to submit tests of behaviors that IE9 fails and that their browser passes. The more comprehensive these test case projects are the more everyone benefits by having a real target to implement.
ACID is not an authority. They cherry pick a relative handful of problematic yet pointless tests and construct something cute. ACID3 tests about a hundred different things from a mix of technologies. The W3C Test Suite for CSS2.1 currently has nearly 8,000 tests. Which do you think is more comprehensive?
Well, that's the problem with HTML. The W3C doesn't create an acceptance test, so there's really no objective way to measure how compliant a particular browser truly is. People love to use the ACID tests, but ACID tests only a small portion of the relevant standards. And the portions tested aren't even the major, important parts; ACID tests for very obscure, esoteric parts of the standards.
On one hand, you can look at the ACID tests and say 'well, at least it's an indication of interest in conforming to the standard.' But is that true either? ACID tests have become another marketing point: 'my browser got to 100% compliance before your browser.' Aiming for 100% on the ACID tests doesn't necessarily indicate a desire to be highly compliant, it indicates a desire to score 100% on the ACID tests.
You could perhaps consider the instantaneous behavior of the tests: how compliant various browsers are upon release of the new test. There's a certain logic to that; developers which are truly interested in compliance, and not just marketing, will do well in a previously unseen test. But ACID tests aren't developed in isolation either. They're politically justified, an effort to encourage compliance, and as such they test for specific behaviors which major browsers were getting wrong (i.e., a browser could be 99.9% compliant, and ACID would target the 0.1% they get wrong).
So to answer your question: No. There's no comprehensive compliance/acceptance test for any of W3C's standards, so don't expect one either. The only evidence of compliance is anecdotal, and the plural of anecdote is not data. Microsoft's test results are completely unsurprising and generally meaningless for anybody familiar with normal development practices, and W3C standards, but it's a nice indication that they're aiming for at least some level of standards-compliance in IE9.