Featured Articles

IHS teardown reveals Galaxy S5 BOM

IHS teardown reveals Galaxy S5 BOM

Research firm IHS got hold of Samsung’s new flagship smartphone and took it apart to the last bolt to figure out…

More...
Galaxy S5, HTC One M8 available selling well

Galaxy S5, HTC One M8 available selling well

Samsung’s Galaxy S5 has finally gone on sale and it can be yours for €699, which is quite a lot of…

More...
Intel lists Haswell refresh parts

Intel lists Haswell refresh parts

Intel has added a load of Haswell refresh parts to its official price list and there really aren’t any surprises to…

More...
Respawn confirms Titanfall DLC for May

Respawn confirms Titanfall DLC for May

During his appearance at PAX East panel and confirmed on Twitter, Titanfall developer Respawn confirmed that the first DLC pack for…

More...
KFA2 GTX 780 Ti Hall Of Fame reviewed

KFA2 GTX 780 Ti Hall Of Fame reviewed

KFA2 gained a lot of overclocking experience with the GTX 780 Hall of Fame (HOF), which we had a chance to…

More...
Frontpage Slideshow | Copyright © 2006-2010 orks, a business unit of Nuevvo Webware Ltd.
Friday, 19 November 2010 10:25

Microsoft “cheats” at benchmark test

Written by Nick Farell
ie

Feature in IE allows superspeeds to be recorded
A bug in Microsoft's Internet Exploder 9 appears to have the advantage that it allows the browser to sale through the SunSpider JavaScript performance benchmark. The benchmark was devised by the developers of the WebKit browser engine, is used and quoted widely as a measure of browser scripting performance.

IE 9 did extremely well in the test and this had people scratching their heads a bit. One of them Mozilla developer, Rob Sayre, made some minor changes to the test and saw Internet Explorer 9 slow down considerably. He filed a bug against Internet Explorer.

Sayre suggested that an optimization that Internet Explorer 9's Chakra JavaScript engine was performing was fragile. It was easily disabled by minor alterations to the code that it should ignore.

Some more cynical hacks claimed that Redmond must have done it all deliberately and that Internet Explorer 9 had been designed to cheat in the test. The allegation is that Microsoft has built a specific optimisation into Chakra that detects, and bypasses, the specific code in SunSpider, but which has no other purpose.

Such things have been done before, but Ars Technica thinks that was what happened here. It thinks that the benchmark has dead code under the bonnet. This is the kind of code that can be reached and executed, but whose results are never used. Sometimes this code is ignored by a compiler.

SunSpider has a test which computes the sine and cosine of a number using a CORDIC algorithm. JavaScript contains built-in sine and cosine functionality, functionality that will be much faster than performing the computation in this way, so it is not something real programs would ever do. The test does not bother using the results that it has computed and as a result the entire test susceptible to dead code elimination.

Internet Explorer 9 accurately treats the entire test as dead code, and so removes the whole lot. This makes for a very fast benchmark result. Actually this is a good thing, as it means that IE 9 is capable of ignoring a lot of dead code. However it does boost the performance figures a bit too much.

Nick Farell

E-mail: This e-mail address is being protected from spambots. You need JavaScript enabled to view it
blog comments powered by Disqus

Comments  

 
+7 #1 lucserp 2010-11-19 10:31
So what's the news?! No surprise here! :lol:
 
 
+6 #2 yourma2000 2010-11-19 14:48
still did shit in the FutureMark PeaceKeeper benchmark compared to the likes of Chrome
 
 
+2 #3 Reavenk 2010-11-19 16:24
Cheats? Really? It doesn't sound like Microsoft’s fault for having valid optimizations. Sounds like it's SunSpider's fault for not making a more thorough benchmark. No one accuses ATI or NVidia of cheating when their GLSL or HLSL compilers cut out dead code, or GCC or VC++ for doing the same (in release compiles).
 
 
+5 #4 ghelyar 2010-11-19 17:05
So add an "alert(result)" or something similar, eliminating the dead code. There should really not be dead code in a benchmark anyway.

This is actually *very* common for modern compilers and IDEs will even give warnings on the fly for this, just as short circuit evaluation is common, for example.

I doubt there is much dead code in javascript on the web though.

That is assuming that it is actually just optimising the dead code, and isn't cheating on one specific test, which is equally plausible.
 
 
+7 #5 ghelyar 2010-11-19 17:15
If you actually look at the changes, all that was added was a superfluous "true;" line, which shouldn't have affected the test at all and if the interpreter was optimising as suggested, should have even been removed entirely. Dead code optimization would not have been affected by this at all and would have still skipped over it.

*if* this was the only change *and* it made a significant difference then there can be no doubt that it was cheating on the test. No software engineer worth their salt would argue that.
 
 
0 #6 hoohoo 2010-11-20 02:16
Word police:

'sail' not 'sale'.

(Unless that was bit of Nick Farell subtlety!)
 
 
+3 #7 Fud_u 2010-11-20 02:19
But the problem remain... How is MS convincing people to update their aging IE6. IE sicken me as a web developer. It so fragmented.
 
 
+4 #8 JAB Creations 2010-11-21 21:42
Quoting Reavenk:
Cheats? Really? It doesn't sound like Microsoft’s fault for having valid optimizations.


If you are in a 50 mile bike race that's a circle and ride a thousand feet through the woods finishing the course in say 10 minutes versus other people's several hours then that's cheating.

What M$ did was essentially detect if a test was occurring and avoided doing the heavy work. Ars linkage: http://arstechnica.com/microsoft/news/2010/11/lies-damned-lies-and-benchmarks-is-ie9-cheating-at-sunspider.ars
 

To be able to post comments please log-in with Disqus

 

Facebook activity

Latest Commented Articles

Recent Comments