Slashdot Mirror
Crowdsourcing JavaScript Testing
snitch writes "John Resig, creator of the jQuery JavaScript library, has released Test Swarm, a platform for distributed continuous integration testing for client-side JavaScript. Frustrated with traditional JavaScript testing environments that don't scale, John's new project, which is currently in private alpha, aims to provide a system for outsourcing browser-related testing to large groups of people or communities."
Javascript 1.5 (aka ECMAScript ECMA-262 Edition 3) is what most developers target for good reason. But there are supersets found primarily in Gecko based browsers. But then you throw in the various DOM quirks between browsers and before you know it, programming anything large in Javascript that will be used across a wide variety of browsers can really start to suck due to minor quirks between different implementations. It will be interesting to see how their test cases support/address layout issues, if they do at all.
There may only be a handful of browsers, but that doesn't mean there are only a handful of test cases. When it comes to javascript, there are quite a few variations that can cause problems beyond just the browser name.
One example that I've been specifically dealing with on MooTools involves bugs relating to font antialiasing on Internet Explorer. The issue presents itself on IE 7 when system font smoothing is enabled, but NOT on IE6 or IE8. Furthermore, in only presents itself if the user utilizes "Cleartype" (recommended for LCDs) but not if they use Standard font smoothing. It took me quite some time to narrow down exactly where the problem was.
This isn't an issue on Firefox 2,3, Safari 2,3 (although other kinds of font issues may arise with FF on OSX, and with Safari on Windows). Counting the variations of system settings, major OS and browser versions, that makes for 51 test cases, not including point versions of the OS or browsers, beta versions, and limiting it to the big 3 of browsers.
I'm out of my mind right now, but feel free to leave a message.....
Correction, 40 test cases:
Firefox 2,3 on WinXP,WinVista * 3 system font smoothing settings
Firefox 2,3 on OSX
Safari 2,3 on Win2K,WinXP,WinVista,OSX
IE 6, FF 2,3, on Win2k
IE 6 on WinXP * 3 system font smoothing settings
IE 7,8 on WinXP,WinVista * 3 system font smoothing settings
And of course other issues will have larger sets of test cases. I'm able to narrow it down here because Safari has its own font smoothing that is unaffected by which version of Windows it is running on or what the system font smoothing is set to, and Win2k doesn't have any native font smoothing.
I'm out of my mind right now, but feel free to leave a message.....
Before jumping into some gauntlet of distributed (or aggregated or what ever you want to call it) JavaScript testing. Shouldn't there be a FOSS [cross]platform for unit testing JavaScript under multiple "rendering kits/engines" that can be plugged into Rake, TAP or Nose or any other unit testing harness.
For most applications things like Selenium are more trouble or cost to much money then they are worth. I would like to see something that would allow me to test my JS code across multiple browsers with a simple "make test" within my build environment. Of course there are things like JSLint and Rhino etc... but they don't really handle the issue of making the tests easy to write and run (which means dev's will actually write the tests) on top of Rhino not really representing an actual or even a mock of a particular browsers DOM. I don't relish the idea of having to build an special application into my test harness and then direct my browser to a URL in order to run my JavaScript unit tests. There are plenty of systems out there that compile test reports across multiple platforms from a standard harness (CPAN Testers for one). The problem is not test reporting - it's having an easy way to write and run the tests.