That's actually an 82% increase on performance (where performance is defined as the amount of work done per unit of time). Kraken does a fixed amount of work and reports the time it took to complete. Therefore lower scores represent higher performance.
I *think* some benchmarks uses a score (higher is better) while others uses time to complete (where lower is better), which is very confusing when you have no information about that and they're all in the same table.
Good. Plenty of other people have noticed the problems with the charts as well. It should be broken into 2 tables: "higher is better" and "lower is better."
As it stands, to compare IE Exp to Chrome, which is right next to it, you have to go over to "Percentage Change" and assume that means "Improvement", which it has to for the chart to not be a complete clusterf...then compare IE to IE Exp using that percentage to determine direction for that row, then finally compare IE Exp to Chrome. Just repeat that 5 more times and you're good. And remember to keep track of who won each time so you can get an overall impression at the end.
It's easy if you just peruse the chart (using the real definition of peruse)
Those are some mighty impressive numbers. This makes me happy as now Chrome has more incentive to improve performance (I don't use IE except for testing).
Well at least you use it for testing. Some jerks out there make sure it works in their world and that's it.
I really like that they're keeping the compatibility modes. They come in handy in IE11 already. If their extension support is improved and they've made it easy to port extensions, we should hopefully see all the heavy hitters release a Spartan-compatible version.
To be fair to other web devs who neglect their testing, generally if you are targeting IE 11, it requires the least testing as it renders the closest to the standard of the major browsers (the Firefox, then Chrome and Safari). The thing I see that most doesn't work in IE is draft standards that just aren't implemented in IE. Microsoft is ahead of the curve on security, rendering accuracy and even implementation of fully-ratified new HTML features...but they are behind the curve on implementing draft features which, for web devs, is pretty annoying. Just implement the frickin' datetime input types already! (which is happening in Spartan, other devs will be happy to learn).
Right, I realize that and I've been pleased with Microsoft's close adherence to modern standards in IE11. I should have been more specific. I'm primarily thinking about devs who use crap that ISN'T standard and simply don't care because it works in Safari or it works in Chrome.
In mobile this is particularly bad. Especially as certain mobile web sites send IE the junky legacy site. In some situations the problem was so bad that in the latest IE11 mobile variant (as found on WP 8.1), MS added a wrapper for some commonly used non-standard stuff used in mobile sites. Combine this with the user agent string saying it isn't IE (which is a tactic MS must also use out of necessity), and you get the better version of the site up and running. This results in a better experience for their users, even though that nonsense shouldn't have been necessary and is actually kind of embarrassing.
If they add _proper_ extension support, with the same flexibility (and ease of use for devs) as other browsers i'll switch to Spatan. They now have good performance and i'm expecting far more regular updates for the browser too.
IE has had much better GPU acceleration performance for a long time. It's so much smoother with scrolling on a touch screen, or even without touch. And any type of animations on a page are usually smoother, they don't drop frames/stutter. The UI also scales properly with high DPI. I also like the new UI look and the features shown so far.
So just sort the extensions out and i'll drop Chrome.
Agree with you here, IE seemed to me smoother all around compared to chrome, and I don't regret switching. I do miss some of the more useful chrome extensions.
That was the reason I stopped using chrome scrolling performance is horrible on heavy pages. FF is better so that is my choice, but even FF is behind IE in GPU accleration.
Why are you so eager to switch? Just curious. The performance is similar or worse than Chrome/Firefox, at least in the current version. Support for extension is promising, on paper, but nobody really knows. And MS has never updated a browser at a decent pace. Which meant that even if the browser was "competitive" at the time of release, it was outperformed in a matter of weeks.
Unless the numbers clearly show huge benefits, I'll wait a couple of generations before considering to switch. I got burned with IE twice already; I left it in 2002 and never looked back. And I have not regretted it for a moment.
I am very eager to see how Mozilla's Servo does since they target better single threaded perf but the real goal is multi-threading so it could be interesting on all the octa core in mobile. With that in mind, any clue how multi-threaded Spartan is and any chance to test power consumption too in an update or future articles.If the world is mobile (including laptops) power is pretty relevant. Another issue, there is a significant push for SSL and lots of the most popular websites default on https. This should be relevant for both hardware and software (on the hardware side A53 and A57 are much faster in encryption and Qualcomm might actually have some problems but they could be on their way to fixing it). Point being that when you test web browsing perf and battery , you should really not forget about including encrypted sites.
Looks good, but we've gone down this road before. I think just because Microsoft COULD conceivably update the new browser via the Windows Store more often... I don't know that that means they will.
History suggests it will not make much difference.
I have to believe that sometime after Windows 8.1 when the apps team was working on Windows 10, they just decided to leave the current apps as is and put all their effort into brand new, universal apps. One would hope for continuous improvements after that though.
I'm a card-carrying Windows-hater, but I'm not sure the article is really accurate when it says that IE wasn't on a rapid upgrade cycle like Firefox and Chrome. It wasn't on a rapid _version numbering_ cycle to be sure, but various components of the browser and rendering engine are constantly being updated during the weekly Windows update downloads. And another thing, IE was pretty much the performance leader by a longshot for a long time, but they slacked off once they won the browser wars. Now they see Google as a credible threat to many areas of their business, and in general seem to be shedding the "good enough if it's market leading" attitude of the Gates-Ballmer era.
"This means that even though Chrome and Safari were both based on Webkit, performance can be very different because of the different script engines in each browser."
There is a third very important issue that affects PERCEIVED PERFORMANCE --- caching and preloading. My guess is that caching doesn't matter match, because I assume all the browsers cache to the maximum allowed by the headers of each asset. (The headers are frequently wrong and broken, but that's a different issue.)
However preloading is something where browsers can follow very different strategies. This starts with some fairly obvious ideas like resolving all the addresses of links on a page, but can graduate to using browser history and previous user behavior during this session to start predicting WHICH of those links the user might click and hitting those pages in advance...
I think it is about time for AnandTech to update your user comment section. It has stayed the same for at least 4-5 years, and has fallen behind other sites IMO. If it is a relatively easy change, I would recommend increasing the number of entries displayed per page. Bandwidth and capacity has improved vastly in the last 4-5 years.
So IE is performing poorly in Google made (Octane) and Mozilla made (Kraken) tests, big surprise! which browser do you think will be on top in a Microsoft javascript test? And when its performing better on Sunspider test, you are claming its optimized for that! without any evidence. Who cares if a browser is optimized for a test or not, it means nothing for real usage scenarios. You don't think Chrome is optimized for Octane test!?
You do realize that IE Experimental actually had the highest score of all tested browsers in the Octane test now?
In any case, the tests are there to show the speed improvements inside IE, not necessarily a comparison to other browsers, those are mostly for reference.
If memory serves, it's not so much that Sunspider is optimized for IE, it's that Sunspider is poorly written for its purpose as a benchmark. Remember the old days of the Dhrystone benchmark? The first version of it was written to calculate a bunch of stuff, but never used the results of the calculation. Once C compilers started doing "dead code elimination", the program that actually ran no longer matched the distribution of operations that the Dhrystone authors based their benchmark on. They had to come out with a new version that actually used the calculated values (and nowadays might well have to generate random numbers to calculate, lest "constant propagation" optimize the calculations away again).
Sunspider has, or at least at one time had, the same problem the first Dhrystone benchmark had; it is or I hope now was, full of dead code, and IE's Javascript did dead code elimination--so it looked better on the benchmark, but those results wouldn't mean diddly for performance in real life.
I recall an MS web demo that gratuitously endlessly switched the background from a solid gray to a solid gray (different data, but not perceptibly different to the human eye). Firefox (and Chrome? it's been long enough I'm not sure) didn't cache backgrounds, and hence the needless flailing away at the background made it look bad, though no sane web developer would create a web site that generated a lot of overhead that nobody could possibly see the result of.
Great... Another f***ckin ms web browser that's going to have quirks galore and waste weeks of my life. MS has wasted MILLIONS of hours of web developers lives. AND THAT WAS JUST LAST WEEK.
I don't care about performance I don't care about features THE ONLY THING MS NEEDS IS A BROWSER THAT ADHERES TO HTML CSS, AND ECMA STANDARDS TO THE TEE. END OF STORY. WE DONT NEED TWO BROWSERS WITH DUAL RENDERING ENGINES ITS JUST MORE WORK THAT I DONT GET PAID FOR THAT IN THE END WILL JUST PLAIN SUCK.
I'm pretty sure the idea is that spartan will auto-update and stay current. Supporting legacy versions of IE should eventually go away. Not sure what else you want. Microsoft must ship windows with a web browser. Sure this is a change that should have happened with windows 8 and not windows 10, but at least it's finally happening.
Speed is nice but correctness should come first, speed second.
Neither IE nor Chrome displays correct colors on a color profiled monitor.
Any incorrect display of a webpage, including displaying the wrong colours, should be awarded a fail and no score, however fast the incorrect display is generated.
I think Google and Apple have (for lack of a better term) last-level compilers that do extra work optimizing heavily run code. Wonder if IE has one now, or if those results are really about DOM access speed not JS itself.
"Internet Explorer has never been on the fast update cycle that other browsers are on – namely Google’s Chome browser and Mozilla’s Firefox."
That's a common misconception. Internet Explorer actually is updated quite frequently via Windows Update; it just doesn't reflect these incremental, comulative updates by inflationary abusing its major version counter like the other ones. As of January 2015, the current version of IE11 is 11.0.15, which means it has already received 15 incremental update since Windows 8.1 has been released.
"WebM" support is a ambiguous here. Does it also mean native VP8, VP9, and/or Opus support, or does it only signify a simple parser for the _container_ with no guaranteed extra codec support?
A new native video decoder in IE is HUGE news, so it'd be great if you could verify!
from what I see, yea it's an improvement but it still is losing to firefox and chrome overall. I currently use chrome and it's worked the fastest and most stable than any other browser i've used in my 22 years of using the net. I've tried all the major browsers and in my opinion chrome is the snappiest. For MS to get me to leave chrome they can't just come close to or just slightly edge out chrome they have to give me a noticeable improvement in speed and stability and they are def not there yet.
I agree overall chrome still looks faster overall.
But I'd have to look at the text rendering and smoothness of scrolling before I can really decide. In the past IE and chrome have been significantly different in this regard.
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
49 Comments
Back to Article
tuxRoller - Sunday, January 25, 2015 - link
http://arewefastyet.com/#machine=29Your relative performance values don't line up with their bot.
tuxRoller - Sunday, January 25, 2015 - link
Ignore.I misread the chart.
Chalabala - Sunday, January 25, 2015 - link
Shouldn't the Kraken score be a 45% decrease in performance instead of a jump?madmilk - Sunday, January 25, 2015 - link
That's actually an 82% increase on performance (where performance is defined as the amount of work done per unit of time). Kraken does a fixed amount of work and reports the time it took to complete. Therefore lower scores represent higher performance.BlueScreenJunky - Sunday, January 25, 2015 - link
I *think* some benchmarks uses a score (higher is better) while others uses time to complete (where lower is better), which is very confusing when you have no information about that and they're all in the same table.althaz - Sunday, January 25, 2015 - link
If it helps, all of those performance changes are improvements.deV14nt - Sunday, January 25, 2015 - link
Good. Plenty of other people have noticed the problems with the charts as well. It should be broken into 2 tables: "higher is better" and "lower is better."As it stands, to compare IE Exp to Chrome, which is right next to it, you have to go over to "Percentage Change" and assume that means "Improvement", which it has to for the chart to not be a complete clusterf...then compare IE to IE Exp using that percentage to determine direction for that row, then finally compare IE Exp to Chrome. Just repeat that 5 more times and you're good. And remember to keep track of who won each time so you can get an overall impression at the end.
It's easy if you just peruse the chart (using the real definition of peruse)
B3an - Sunday, January 25, 2015 - link
It should be made clear in the article:SunSpider: Lower is better.
Kraken: Lower is better.
B3an - Sunday, January 25, 2015 - link
And all other tests: Higher is better.Gothmoth - Sunday, January 25, 2015 - link
everone with 3 brain cells should get that without additional infos.....Murloc - Sunday, January 25, 2015 - link
still, I had to look twice and compare the numbers to get it, they should write it.Brett Howse - Sunday, January 25, 2015 - link
Updated the chart sorry for the confusion it was late last night!althaz - Sunday, January 25, 2015 - link
Those are some mighty impressive numbers. This makes me happy as now Chrome has more incentive to improve performance (I don't use IE except for testing).Alexvrb - Sunday, January 25, 2015 - link
Well at least you use it for testing. Some jerks out there make sure it works in their world and that's it.I really like that they're keeping the compatibility modes. They come in handy in IE11 already. If their extension support is improved and they've made it easy to port extensions, we should hopefully see all the heavy hitters release a Spartan-compatible version.
althaz - Monday, January 26, 2015 - link
To be fair to other web devs who neglect their testing, generally if you are targeting IE 11, it requires the least testing as it renders the closest to the standard of the major browsers (the Firefox, then Chrome and Safari). The thing I see that most doesn't work in IE is draft standards that just aren't implemented in IE. Microsoft is ahead of the curve on security, rendering accuracy and even implementation of fully-ratified new HTML features...but they are behind the curve on implementing draft features which, for web devs, is pretty annoying. Just implement the frickin' datetime input types already! (which is happening in Spartan, other devs will be happy to learn).Alexvrb - Monday, January 26, 2015 - link
Right, I realize that and I've been pleased with Microsoft's close adherence to modern standards in IE11. I should have been more specific. I'm primarily thinking about devs who use crap that ISN'T standard and simply don't care because it works in Safari or it works in Chrome.In mobile this is particularly bad. Especially as certain mobile web sites send IE the junky legacy site. In some situations the problem was so bad that in the latest IE11 mobile variant (as found on WP 8.1), MS added a wrapper for some commonly used non-standard stuff used in mobile sites. Combine this with the user agent string saying it isn't IE (which is a tactic MS must also use out of necessity), and you get the better version of the site up and running. This results in a better experience for their users, even though that nonsense shouldn't have been necessary and is actually kind of embarrassing.
So again, thank you.
B3an - Sunday, January 25, 2015 - link
If they add _proper_ extension support, with the same flexibility (and ease of use for devs) as other browsers i'll switch to Spatan. They now have good performance and i'm expecting far more regular updates for the browser too.IE has had much better GPU acceleration performance for a long time. It's so much smoother with scrolling on a touch screen, or even without touch. And any type of animations on a page are usually smoother, they don't drop frames/stutter. The UI also scales properly with high DPI. I also like the new UI look and the features shown so far.
So just sort the extensions out and i'll drop Chrome.
maximumGPU - Sunday, January 25, 2015 - link
Agree with you here, IE seemed to me smoother all around compared to chrome, and I don't regret switching. I do miss some of the more useful chrome extensions.Murloc - Sunday, January 25, 2015 - link
adblock plus and downthemall are what I need, but I seriously doubt downthemall will be ported....jeffkibuule - Sunday, January 25, 2015 - link
Rumor is Chrome extensions will work natively in Spartan with minimal work.Makaveli - Sunday, January 25, 2015 - link
That was the reason I stopped using chrome scrolling performance is horrible on heavy pages. FF is better so that is my choice, but even FF is behind IE in GPU accleration.yankeeDDL - Monday, January 26, 2015 - link
Why are you so eager to switch? Just curious.The performance is similar or worse than Chrome/Firefox, at least in the current version. Support for extension is promising, on paper, but nobody really knows.
And MS has never updated a browser at a decent pace. Which meant that even if the browser was "competitive" at the time of release, it was outperformed in a matter of weeks.
Unless the numbers clearly show huge benefits, I'll wait a couple of generations before considering to switch. I got burned with IE twice already; I left it in 2002 and never looked back. And I have not regretted it for a moment.
damianrobertjones - Tuesday, January 27, 2015 - link
2002.... ?jjj - Sunday, January 25, 2015 - link
I am very eager to see how Mozilla's Servo does since they target better single threaded perf but the real goal is multi-threading so it could be interesting on all the octa core in mobile.With that in mind, any clue how multi-threaded Spartan is and any chance to test power consumption too in an update or future articles.If the world is mobile (including laptops) power is pretty relevant.
Another issue, there is a significant push for SSL and lots of the most popular websites default on https. This should be relevant for both hardware and software (on the hardware side A53 and A57 are much faster in encryption and Qualcomm might actually have some problems but they could be on their way to fixing it). Point being that when you test web browsing perf and battery , you should really not forget about including encrypted sites.
mayankleoboy1 - Sunday, January 25, 2015 - link
Holy cr@p at that Octane score!Drumsticks - Sunday, January 25, 2015 - link
I knew microsoft had a good browser hidden in them somewhere :) IE11 was decent but this is just fantastic.HisDivineOrder - Sunday, January 25, 2015 - link
Looks good, but we've gone down this road before. I think just because Microsoft COULD conceivably update the new browser via the Windows Store more often... I don't know that that means they will.History suggests it will not make much difference.
jeffkibuule - Sunday, January 25, 2015 - link
I have to believe that sometime after Windows 8.1 when the apps team was working on Windows 10, they just decided to leave the current apps as is and put all their effort into brand new, universal apps. One would hope for continuous improvements after that though.DanNeely - Sunday, January 25, 2015 - link
In the back compatibility slide, why does MS continue to provide compatibility to IE5.5 but not IE6?ABR - Sunday, January 25, 2015 - link
I'm a card-carrying Windows-hater, but I'm not sure the article is really accurate when it says that IE wasn't on a rapid upgrade cycle like Firefox and Chrome. It wasn't on a rapid _version numbering_ cycle to be sure, but various components of the browser and rendering engine are constantly being updated during the weekly Windows update downloads. And another thing, IE was pretty much the performance leader by a longshot for a long time, but they slacked off once they won the browser wars. Now they see Google as a credible threat to many areas of their business, and in general seem to be shedding the "good enough if it's market leading" attitude of the Gates-Ballmer era.name99 - Sunday, January 25, 2015 - link
"This means that even though Chrome and Safari were both based on Webkit, performance can be very different because of the different script engines in each browser."There is a third very important issue that affects PERCEIVED PERFORMANCE --- caching and preloading. My guess is that caching doesn't matter match, because I assume all the browsers cache to the maximum allowed by the headers of each asset. (The headers are frequently wrong and broken, but that's a different issue.)
However preloading is something where browsers can follow very different strategies. This starts with some fairly obvious ideas like resolving all the addresses of links on a page, but can graduate to using browser history and previous user behavior during this session to start predicting WHICH of those links the user might click and hitting those pages in advance...
gamoniac - Monday, January 26, 2015 - link
I think it is about time for AnandTech to update your user comment section. It has stayed the same for at least 4-5 years, and has fallen behind other sites IMO. If it is a relatively easy change, I would recommend increasing the number of entries displayed per page. Bandwidth and capacity has improved vastly in the last 4-5 years.I hope this will reach the decision maker (Ryan?)
nathanddrews - Monday, January 26, 2015 - link
Just click "View All Comments" if you want to read everything at once. Easy.Cliff34 - Monday, January 26, 2015 - link
Wow. I learn something new. It would be nice if it is not just black text. Maybe an underline will help make it easier to spot.USGroup1 - Monday, January 26, 2015 - link
So IE is performing poorly in Google made (Octane) and Mozilla made (Kraken) tests, big surprise! which browser do you think will be on top in a Microsoft javascript test?And when its performing better on Sunspider test, you are claming its optimized for that! without any evidence. Who cares if a browser is optimized for a test or not, it means nothing for real usage scenarios. You don't think Chrome is optimized for Octane test!?
nevcairiel - Monday, January 26, 2015 - link
You do realize that IE Experimental actually had the highest score of all tested browsers in the Octane test now?In any case, the tests are there to show the speed improvements inside IE, not necessarily a comparison to other browsers, those are mostly for reference.
jejones3141 - Friday, March 20, 2015 - link
If memory serves, it's not so much that Sunspider is optimized for IE, it's that Sunspider is poorly written for its purpose as a benchmark. Remember the old days of the Dhrystone benchmark? The first version of it was written to calculate a bunch of stuff, but never used the results of the calculation. Once C compilers started doing "dead code elimination", the program that actually ran no longer matched the distribution of operations that the Dhrystone authors based their benchmark on. They had to come out with a new version that actually used the calculated values (and nowadays might well have to generate random numbers to calculate, lest "constant propagation" optimize the calculations away again).Sunspider has, or at least at one time had, the same problem the first Dhrystone benchmark had; it is or I hope now was, full of dead code, and IE's Javascript did dead code elimination--so it looked better on the benchmark, but those results wouldn't mean diddly for performance in real life.
I recall an MS web demo that gratuitously endlessly switched the background from a solid gray to a solid gray (different data, but not perceptibly different to the human eye). Firefox (and Chrome? it's been long enough I'm not sure) didn't cache backgrounds, and hence the needless flailing away at the background made it look bad, though no sane web developer would create a web site that generated a lot of overhead that nobody could possibly see the result of.
dsumanik - Monday, January 26, 2015 - link
Great... Another f***ckin ms web browser that's going to have quirks galore and waste weeks of my life. MS has wasted MILLIONS of hours of web developers lives. AND THAT WAS JUST LAST WEEK.I don't care about performance I don't care about features THE ONLY THING MS NEEDS IS A BROWSER THAT ADHERES TO HTML CSS, AND ECMA STANDARDS TO THE TEE. END OF STORY. WE DONT NEED TWO BROWSERS WITH DUAL RENDERING ENGINES ITS JUST MORE WORK THAT I DONT GET PAID FOR THAT IN THE END WILL JUST PLAIN SUCK.
andrewaggb - Monday, January 26, 2015 - link
I'm pretty sure the idea is that spartan will auto-update and stay current. Supporting legacy versions of IE should eventually go away. Not sure what else you want. Microsoft must ship windows with a web browser. Sure this is a change that should have happened with windows 8 and not windows 10, but at least it's finally happening.CSMR - Monday, January 26, 2015 - link
Speed is nice but correctness should come first, speed second.Neither IE nor Chrome displays correct colors on a color profiled monitor.
Any incorrect display of a webpage, including displaying the wrong colours, should be awarded a fail and no score, however fast the incorrect display is generated.
Hulk - Monday, January 26, 2015 - link
I switched to Chrome many, many years ago. And from the results of this table it looks like that's where I'll stay.Michael Bay - Tuesday, January 27, 2015 - link
Enjoy your bloat.Da W - Monday, January 26, 2015 - link
Yay. Another fast browser. Seriously who need performance to Watch porn?Tunnah - Monday, January 26, 2015 - link
Damn 4chan has forever ruined the phrase "large gains"twotwotwo - Monday, January 26, 2015 - link
Huh, Octane and Kraken gains look substantial.I think Google and Apple have (for lack of a better term) last-level compilers that do extra work optimizing heavily run code. Wonder if IE has one now, or if those results are really about DOM access speed not JS itself.
(see https://www.webkit.org/blog/3362/introducing-the-w... for WebKit's new JIT, for instance.)
sevenacids - Tuesday, January 27, 2015 - link
"Internet Explorer has never been on the fast update cycle that other browsers are on – namely Google’s Chome browser and Mozilla’s Firefox."That's a common misconception. Internet Explorer actually is updated quite frequently via Windows Update; it just doesn't reflect these incremental, comulative updates by inflationary abusing its major version counter like the other ones. As of January 2015, the current version of IE11 is 11.0.15, which means it has already received 15 incremental update since Windows 8.1 has been released.
mavere - Tuesday, January 27, 2015 - link
"WebM" support is a ambiguous here. Does it also mean native VP8, VP9, and/or Opus support, or does it only signify a simple parser for the _container_ with no guaranteed extra codec support?A new native video decoder in IE is HUGE news, so it'd be great if you could verify!
Laststop311 - Tuesday, January 27, 2015 - link
from what I see, yea it's an improvement but it still is losing to firefox and chrome overall. I currently use chrome and it's worked the fastest and most stable than any other browser i've used in my 22 years of using the net. I've tried all the major browsers and in my opinion chrome is the snappiest. For MS to get me to leave chrome they can't just come close to or just slightly edge out chrome they have to give me a noticeable improvement in speed and stability and they are def not there yet.andrewaggb - Tuesday, January 27, 2015 - link
I agree overall chrome still looks faster overall.But I'd have to look at the text rendering and smoothness of scrolling before I can really decide. In the past IE and chrome have been significantly different in this regard.