Slashdot Mirror
A Look At Microsoft's 'Mini Internet' For Testing IE
MrSeb writes "With the grandiose bluster that only an aging juggernaut can pull off, Microsoft has detailed the Internet Explorer Performance Lab and its extraordinary efforts to ensure IE9 is competitive and IE10 is the fastest browser in the world. Here are a few bullet points: 128 test computers, 20,000 tests per day, over 850 metrics analyzed, 480GB of runtime data per day, and a granularity of just 100 nanoseconds. The data is reported to 11 server-class (16-core, 16GB of RAM) computers, and the data is stored on a 24-core, 64GB SQL server. The 'mini internet' has content servers, DNS servers, and network emulators (to model various different latencies, throughputs, packet loss)."
The only thing Firefox does fast anymore is update.
They wanted to account for any kind of lag, so by having it all in house and disconnected from even their internal network, they have control over all variables so everything is equal.
They did this post on their blog yesterday http://blogs.msdn.com/b/b8/
And when all we care about is the fastest browser - in nanoseconds! - will we begin to forget the truly important criteria for choosing a browser?
Or better still, by the time IE is on par with Chrome the actual browser will be irrelevant because mobile platforms - in which IE has little share - will do to traditional computers what Cromagnons did to Neanderthals. The next generation will use integrated devices, unaware they were using a browser, and with little or no need for even a choice.
Help fight poverty: Punch a poor person.
And still unable to correctly implement CSS2 and HTML4
Fixed that for ya.
Because you can not have reproducable results on the real Internet. Only a fake one, where eveyrthing is controlled and reproducable, can be used for testing and making sure some settings do not make the browser slower.
Correctly according to whom? Neither CSS3 nor HTML5 are completed standards and various portions remain in flux. Of the more mature bits IE9 and IE10 implement quite a bit of it and do so quite comprehensively.
MS is also one of very few organizations that is very actively involved with the W3C Test Suite by submitting test cases for each portion of the standard under various circumstances to demonstrate correct behaviors. What Google and Mozilla do instead is slap together a partial implementation and call it a day. More than once has their implementations been found to be not only incomplete but also incorrect.
Stop relying on scores given by non-authoritative tests demonstrating exceedingly limited and selective interpretations of non-standardized functionality. Oh, and HTML5 Video does NOT specify a codec, in fact it was designed to handle many simultaneous codecs, including h.264, which is explicitly referenced in the draft.
You have to figure out what the variables that you have problems with in real world usage, before you can start optimising your product to account for them.
There has to be iterative cycles of real world, then fake internet testing to really make it work well.
It would also help if you were able to test your competition alogn the same lines.
I additionally wonder if they are accouting for all of the different behavious of all of the various webservers out there. If they are only testing agianst iis, well, that's not very good.
Well.. maybe. Or Maybe not. But Definitely not sort of.
MSDN blogs are often very technically detailed, written by people who know this stuff from the inside, and if it's about a topic that's of general computing interest then it seems that's a good thing. And the blog in question is chock full of some really good detailed stuff about how they're doing performance testing, reasons why the lab is architected the way it is, detailed graphics on how they measure performance, how they analyze it...on and on.
Frankly, this seems more akin to old Slashdot than a lot of the nonsense we see here today. (That story the other day about a girl sent home from school because her lunch wasn't healthy, and then quickly called into question over what happened? Really? What was that topic even doing on Slashdot in the first place?) Whatever you think about Microsoft, having this extremely detailed look into how one of the world's biggest software vendors (or are they the biggest now?) goes about performance testing, and how they ensure consistent results, should be really, really interesting to anyone involved in IT.