The Snapdragon 865 Performance Preview: Setting the Stage for Flagship Android 2020
by Andrei Frumusanu on December 16, 2019 7:30 AM EST- Posted in
- Mobile
- Qualcomm
- Smartphones
- 5G
- Cortex A77
- Snapdragon 865
Machine Learning Inference Performance
AIMark 3
AIMark makes use of various vendor SDKs to implement the benchmarks. This means that the end-results really aren’t a proper apples-to-apples comparison, however it represents an approach that actually will be used by some vendors in their in-house applications or even some rare third-party app.
In AIMark 3, the benchmark uses each vendor’s proprietary SDK in order to accelerate the NN workloads most optimally. For Qualcomm’s devices, this means that seemingly the benchmark is also able to take advantage of the new Tensor cores. Here, the performance improvements of the new Snapdragon 865 chip is outstanding, posting in 2-3x performance compared to its predecessor.
AIBenchmark 3
AIBenchmark takes a different approach to benchmarking. Here the test uses the hardware agnostic NNAPI in order to accelerate inferencing, meaning it doesn’t use any proprietary aspects of a given hardware except for the drivers that actually enable the abstraction between software and hardware. This approach is more apples-to-apples, but also means that we can’t do cross-platform comparisons, like testing iPhones.
We’re publishing one-shot inference times. The difference here to sustained performance inference times is that these figures have more timing overhead on the part of the software stack from initialising the test to actually executing the computation.
AIBenchmark 3 - NNAPI CPU
We’re segregating the AIBenchmark scores by execution block, starting off with the regular CPU workloads that simply use TensorFlow libraries and do not attempt to run on specialized hardware blocks.
Starting off with the CPU accelerated benchmarks, we’re seeing some large improvements of the Snapdragon 865. It’s particularly the FP workloads that are seeing some big performance increases, and it seems these improvements are likely linked to the microarchitectural improvements of the A77.
AIBenchmark 3 - NNAPI INT8
INT8 workload acceleration in AI Benchmark happens on the HVX cores of the DSP rather than the Tensor cores, for which the benchmark currently doesn’t have support for. The performance increases here are relatively in line with what we expect in terms of iterative clock frequency increases of the IP block.
AIBenchmark 3 - NNAPI FP16
FP16 acceleration on the Snapdragon 865 through NNAPI is likely facilitated through the GPU, and we’re seeing iterative improvements in the scores. Huawei’s Mate 30 Pro is in the lead in the vast majority of the tests as it’s able to make use of its NPU which support FP16 acceleration, and its performance here is quite significantly ahead of the Qualcomm chipsets.
AIBenchmark 3 - NNAPI FP32
Finally, the FP32 test should be accelerated by the GPU. Oddly enough here the QRD865 doesn’t fare as well as some of the best S855 devices. It’s to be noted that the results here today were based on an early software stack for the S865 – it’s possible and even very likely that things will improve over the coming months, and the results will be different on commercial devices.
Overall, there’s again a conundrum for us in regards to AI benchmarks today, the tests need to be continuously developed in order to properly support the hardware. The test currently doesn’t make use of the Tensor cores of the Snapdragon 865, so it’s not able to showcase one of the biggest areas of improvement for the chipset. In that sense, benchmarks don’t really mean very much, and the true power of the chipset will only be exhibited by first-party applications such as the camera apps, of the upcoming Snapdragon 865 devices.
Facebook
Twitter
RSS
178 Comments
View All Comments
michael2k - Wednesday, December 18, 2019 - link
I know you want to think that, but the 855 right now barely competes with the A13. The gap may be getting smaller, but when Apple has a multiple year lead in performance, it will take multiple years of iteration to catch up, and that assumes Apple isn't growing either.From the article, the 865 wasn't very competitive with the older A12:
On the integer side, the A77 still trails Apple’s Monsoon cores in the A11, but the new Arm design now has been able to trounce it in the FP suite. We’re still a bit far away from the microarchitectures catching up to Apple’s latest designs, but if Arm keeps up this 25-30% yearly improvement rate, we should be getting there in a few more iterations.
...
The QRD865 really didn’t manage to differentiate itself from the rest of the Android pack even though it was supposed to be roughly 20-25% ahead in theory. I’m not sure what the limitation here is, but the 5-10% increases are well below what we had hoped for. For now, it seems like the performance gap to Apple’s chips remains significant.
...
There’s one apparent issue here when looking at the chart rankings; although there’s an improvement in the peak performance, the end result is that the QRD865 still isn’t able to reach the sustained performance of Apple’s latest A13 phones.
...
Looking at the estimated power draw of the phone, it indeed does look like Qualcomm has been able to sustain the same power levels as the S855, but the improvements in performance and efficiency here aren’t enough to catch up to either the A12 or A13, with Apple being both ahead in terms of performance, power and efficiency.
The 855 was released early this year and was not very competitive with the slightly older A11:
https://www.anandtech.com/show/14031/samsung-galax...
Deboo - Monday, January 27, 2020 - link
Isn't apple cpu cores are arm basedmichael2k - Tuesday, December 17, 2019 - link
The formal term for it is induction.Apple's GPU is very competitive in phones, sometimes taking the top spot, sometimes not:
https://www.anandtech.com/show/13392/the-iphone-xs...
But for their tablets, Apple proceeds to beef up the GPU over 2x the iPhone, in this case by having 75% more CPU cores, much better thermal capacity, and higher clockspeed:
https://www.anandtech.com/show/13661/the-2018-appl...
It is beaten by the GTX 1060, but beats the Ryzen 7, in PC space, and soundly beats the iPhone by 80% when not CPU bound.
So the inductive part is that, given the 865 approaches the performance of an iPhone, it won't approach the performance of an iPad.
id4andrei - Tuesday, December 17, 2019 - link
One thing to remember when you compare GPUs. Benchmarks usually run half precision operations on mobile compared to what they run on desktop - FP16 vs FP32. Also on ios they run Metal while on Windows DX12 or Vulkan or OpenGL. Not the same thing.IUU - Thursday, December 26, 2019 - link
Apple cores beating the ryzen 7 is a very big word and requires a big leap of faithph00ny - Monday, December 16, 2019 - link
Probably. Android tablet is pretty dead and there is no real solid demand for a "faster" hardwaregeneralako - Monday, December 16, 2019 - link
That's a stupid exaggeration. If it were dead how come Huawei just released the Mediapad M6 this summer with Kirin 980, and is currently releasing an even more premium tablet right now? How come Samsung makes a new Tab S every year, along with iterations of cheaper models every now and then? Same with Xiaomi, Lenovo and others. Low-end market is sprawling with new tablets from all sorts of brands. Even LG released a tablet this year. The tablet market may not be huge, and certainly lackluster in a lot of ways on Android (mainly due to Google neglecting it since Nexus 9 was a flop; you don’t follow-up Nexus 7 v2—the best tablet ever made—with that nonsense), but it very much exists and is desired by a lot of customers. This reality is true for Apple too, that waited 4-5 years before it released a new iPad Mini iteration.As an avid tablet user myself, even with a lot to be desired on Android, I still very much like many offerings by them the past 2 years. I'm currently using both an iPad Mini 5 and Mediapad M6 8.4”. Seeing as I, like most other people, use tablett for media consumption (YouTube, streaming movies/shows, reading books, Reddit, browsing, Spotify), the available supported apps are equally good for both platforms. If I were to dive deeper to dedicated application, sure, iPad has much better support, I really never do as I use a tablet for the specific uses it was made for. I can see this be a complaint if you're an iPad Pro user and use it for professional work, but I don't really see that being a desire for even those using regular iPad or iPad Mini. Maybe if you’re a gamer, but that’s really it.
I use the Huawei more than the iPad Mini due to how much more intuitive Android is. At the end of the day, they both run oversized variants of their smartphone OS, and Android is simply more intuitive in a lot of respects, with iOS use still feeling like having one hand behind my back. Where Apple is fantastic though, is in its hardware implementation, like its excellent screen calibration and touch latency, or having a 3.5mm input (unlike M6 or newer Tab S, sadly) with really good DAC that properly drives my HD650. Mediapad wins in more effective and ease-of-use OS, 16:10 aspect ratio. Android also makes it easier to do things like torrenting (which is great for downloading/streaming movies, football matches, etc.), local file management and sharing and more. Ironically, Apple has made up for the Mediapad’s lack of jack with its fantastic 3.5mm Type-C for $7, which beats DACs upwards of 10x its price (no joke – Apple really knocked it out of the park).
imaheadcase - Tuesday, December 17, 2019 - link
Let alone the thousands of Samsung tablets with android that sell like hot cakes. Given the sell over holidays on them, they are even more attractive.Sure Samsung is terrible at updating software, but not a deal breaker by any means, i tried parents Ipad and it just felt wrong using it, i can't quite put finger on reason, just seems so limited with apps and just not responsive.
Things just work great on Android variant for me.
Oliseo - Thursday, December 19, 2019 - link
One look at Android App Stores tells you just how successful Android Tablets are. That's something everyone can do, and not just take some random fanboys drivel on a forum./end debate
Lois - Friday, December 27, 2019 - link
This is just your feeling and not a fact… Some people feel better on iPadOS and some others on Android. Now the fact is that iPad is more powerfull than andoid Tablets…