W3C DOM Level 1 Conformance Test Suite Released
Dimitris Dimitriadis of Ontologicon , W3C Representative to the DOM Test Suite Group, writes with an announcement also signed by Philippe Le Hégaret, W3C DOM Activity lead, and Mary Brady of NIST: "The DOM Test Suite Group releases the first version of the DOM
Conformance Test Suite, Level 1 Core.
The first version of the DOM Conformance Test Suite for Level 1 Core has
been released by the DOM Test Suite Group. The DOM TS aims at helping
implementors test their implementations' conformance with the W3C DOM
Level 1 specification. This work, launched by W3C and NIST, is a publically developed and open
framework to test the DOM Level 1 Core implementations. Read about the
Document Object Model (DOM) Conformance Test Suites at
http://www.w3.org/DOM/Test, where you can also download the DOM TS
distribution. Comments are appreciated and need to be sent to www-dom-ts@w3.org
(online archive at http://lists.w3.org/Archives/Public/www-dom-ts)."
I'm sure there are a few implementations out there, but does any of them have any significant benefit over another?
Just tested ECMAScript on Mozilla 0.9.8. I got 25 failures and 63 errors out of 290 runs! It is not very good, isn't it? IE6 gives 18 failures and 10 errors. Really sad... Hope mozilla.org will do something about it...
Were you never warned not to compare apples and oranges. The raw test failure count is not a good metric (unless it is zero). Several points: 1) The DOM L1 test suite currently only contains tests that exercise the XML and SVG implementations and contains no HTML tests. For MS IE, this means testing whatever is loaded to MSXML2.DomDocument or the Adobe SVG control. For Mozilla, this means the internal XML DOM implementation. We are awaiting submission of additional tests that will test HTML implementations of DOM L1. 2) Many of the Mozilla test failures are due to Mozilla not parsing the DTD and providing default values for attributes. 3) This is the first of several test suites for different levels and modules of the DOM. Don't be making conclusions just on the basis of the L1 DOM Core suite. 4) Exceptions throw to the JScript binding from MSXML do not adhere to the DOMException interface (from the public documentation for Windows Scripting, it does not appear possible to craft the exception class that is generated for a COM failure HRESULT). The test adapter for MSXML compensates for this. The compensation is justified by the third paragraph in http://www.w3.org/TR/2000/WD-DOM-Level-1-20000929/ level-one-core.html#ID-17189187 which allows for the limitation of object system. If this compensation is eliminated, then MSXML will fail every test that checks the code for an exception which will radically change the raw numbers (find MSXMLDocumentBuilder_isDOMExceptionCode(ex, code) in DOMTestSuite.js and add "return (ex.code == code);" in case you are curious.)
So, don't compare apples to oranges, don't rely on default attribute values for Mozilla's XML DOM and don't rely on DOMException.code on MSXML.
p.s. I've participated in the DOM Test Suite work, but the opinions expressed in this message are my personal opinions and not that of the DOM Test Suite group or any other group of the W3C.
p.s.s. A unofficial superficial analysis of the test failures for an earlier version of Mozilla against an working draft versions is available at http://lists.w3.org/Archives/Public/www-dom-ts/200 1Nov/0011.html