First Look At the ACID3 Browser Test
ddanier writes "Now that all major browsers have mastered the ACID2 test (at least in some preview versions), work on ACID3 has begun. The new test will focus on ECMAScript, DOM Level 3, Media Queries, and data: URLs. 100 tests will be put into functions each returning either true or false depending on the result of the test. The current preview of ACID3 is still missing 16 tests."
u-bend
Final scores of course are subject to change on the final test:
* - script takes long enough to run that browser prompts you to kill it.
No, it isn't.
Which is why the GP shouldn't be modded as "Insightful."
The ACID Tests are meant to test certain parts of the proposed standards.
Passing the Test doesn't imply standards compliance.
BUT
Standards compliances DOES imply passing the tests.
It's created by an advanced, custom-built browser which, for certain input, correctly renders a perfectly standards-compliant reference image. Just don't ask to use the browser on any other input.
Help poke pirates in the eyepatch, arr.
How do we know no mistake was made when creating the reference image?
You don't
I remember an article by the Apple guy who made ACID2 work on Safari (I think this was the first browser to make it work). One of the steps to get it working was to fix a bug in the test, when he couldn't make the reference result fit with what the test HTML said.
This is an easy to reproduce set of bugs someone else found on their browser.
I would be glad to receive bug reports with an easy to use test case. It saves me the trouble of determining if it is a bug or not, coming up with a test case, the pain of communicating back and forth with the customer trying to find out what they are doing and how the bug is being triggered, etc. Also, this test suite will improve compatibility with other browsers so it will reduce bug reports in the long run.
Why the heck would they be pissed?
Yeah, it was David Hyatt who was working on getting Saf to pass (and got it to be the first browser to pass in any build, and the first to have a generally available release (i.e., a non-development build, even if public) -- the latter being the only thing that truly counts for passing the test).
http://weblogs.mozillazine.org/hyatt/archives/2005_04.html#008011 details the bug (in this case, it was the test itself that was wrong -- not the reference). The reference rendering for Acid3 is likely correct as the actual rendering isn't overly complex (the complexity is in the ECMAScript and DOM support), though with the complexity of some tests there could easily be bugs in the test again.
We write those tests too, they're called test suites and if you look at my site you'll find literally hundreds if not thousands of them:
http://hixie.ch/tests/adhoc/
The Acid tests are easier for the less technically inclined to get a hold of. In practice, the browser vendors take Acid tests and turn them into small tests of the kind you describe before fixing them. For Acid2, I was the one who did a number of those small tests for Opera (I worked for Opera at the time) -- you can see them here:
http://www.hixie.ch/tests/evil/acid/002/opera001.html
http://www.hixie.ch/tests/evil/acid/002/opera002.html
http://www.hixie.ch/tests/evil/acid/002/opera003.html
http://www.hixie.ch/tests/evil/acid/002/opera004.html
http://www.hixie.ch/tests/evil/acid/002/opera005.html
http://www.hixie.ch/tests/evil/acid/002/opera006.html
http://www.hixie.ch/tests/evil/acid/002/opera007.html
http://www.hixie.ch/tests/evil/acid/002/opera008.html
http://www.hixie.ch/tests/evil/acid/002/opera009.html
http://www.hixie.ch/tests/evil/acid/002/opera010.html
http://www.hixie.ch/tests/evil/acid/002/opera011.html
They're not as exciting as the smiley face, so they don't get the media's attention in the same way.