Published in News

Microsoft “cheats” at benchmark test

by on19 November 2010

Feature in IE allows superspeeds to be recorded
A bug in Microsoft's Internet Exploder 9 appears to have the advantage that it allows the browser to sale through the SunSpider JavaScript performance benchmark. The benchmark was devised by the developers of the WebKit browser engine, is used and quoted widely as a measure of browser scripting performance.

IE 9 did extremely well in the test and this had people scratching their heads a bit. One of them Mozilla developer, Rob Sayre, made some minor changes to the test and saw Internet Explorer 9 slow down considerably. He filed a bug against Internet Explorer.

Sayre suggested that an optimization that Internet Explorer 9's Chakra JavaScript engine was performing was fragile. It was easily disabled by minor alterations to the code that it should ignore.

Some more cynical hacks claimed that Redmond must have done it all deliberately and that Internet Explorer 9 had been designed to cheat in the test. The allegation is that Microsoft has built a specific optimisation into Chakra that detects, and bypasses, the specific code in SunSpider, but which has no other purpose.

Such things have been done before, but Ars Technica thinks that was what happened here. It thinks that the benchmark has dead code under the bonnet. This is the kind of code that can be reached and executed, but whose results are never used. Sometimes this code is ignored by a compiler.

SunSpider has a test which computes the sine and cosine of a number using a CORDIC algorithm. JavaScript contains built-in sine and cosine functionality, functionality that will be much faster than performing the computation in this way, so it is not something real programs would ever do. The test does not bother using the results that it has computed and as a result the entire test susceptible to dead code elimination.

Internet Explorer 9 accurately treats the entire test as dead code, and so removes the whole lot. This makes for a very fast benchmark result. Actually this is a good thing, as it means that IE 9 is capable of ignoring a lot of dead code. However it does boost the performance figures a bit too much.

Rate this item
(11 votes)

Read more about: