SPEC2006 Performance & Efficiency

HiSilicon made some big promises for the Kirin 980, claiming up to 75% higher performance while improving efficiency by 58%. This time around for our analysis, I’m starting with the results off the bat and going into the more detailed analysis later.

We’re picking off with our mobile SoC SPEC2006 results where we left off in our Apple A12 analysis, and add the scores of the new Kirin 980 to the new set.

As a reminder, as the scores aren’t submitted to SPEC, we have to put a disclaimer that these are just estimates as they aren’t officially validated. Naturally, we verify that the tests are run correctly in-house.

When measuring performance and efficiency, it’s important to take three metrics into account: Evidently, the performance and runtime of a benchmark, which in the graphs below is represented on the right axis, growing from the right. Here the bigger the figures, the more performant a SoC/CPU has benchmarked. The labels represent the SPECspeed scores.

On the left axis, the bars are representing the energy usage for the given workload. The bars grow from the left, and a longer bar means more energy used by the platform. A platform is more energy efficient when the bars are shorter, meaning less energy used. The labels showcase the average power used in Watts, which is still an important secondary metric to take into account in thermally constrained devices, as well as the total energy used in Joules, which is the primary efficiency metric.

Both the SPECint2006 and SPECfp2006 overall scores paint a much better picture than HiSilicon lead us to believe in their estimates, and the figures are also ahead of what I had personally estimated using Arm’s marketing materials during the Cortex-A76 launch.

Against the Kirin 970, the Kirin 980 is sporting about flat out double the SPEC2006 performance. Naturally this is a comparison between SoCs two CPU generations apart, but it’s still an outstanding showing because the improvements were not only in the benefit of performance, but also energy efficiency.

The Kirin 980’s energy usage while completing the workloads is among the best in the space, ending about in the same range as Apple’s A12 SoC, slightly eding it out in SPECint2006 as well as SPECfp2006.

We have to remember that the energy usage isn’t the same as power efficiency: While energy usage of a workload is an immensely important metric that will very much correlate to the battery life of a device, the power efficiency of a CPU and SoC is something needs to take into consideration the actual performance as well. In this case, the actual perf/W of the Kirin 980 is only about 28% better than the Kirin 970. So if performance has doubled, and energy usage has also gone down, then it means that something had to give, and that is the power usage.

The new Cortex-A76 cores and the memory subsystem of the Kirin 980 are a lot more power hungry, reaching 2.14W average in SPECint and 2.65W in SPECfp, a notable increase over the 1.38W and 1.72W of the Kirin 970. In a sense, Arm’s new microarchitectures, including the Cortex-A75 of the Snapdragon 845, have been able to increase their performance in a more linear fashion alongside power.

Now this is not really a negative as long as the relationship between gained performance and raised power usage is kept in check. The best example is obviously Apple’s SoCs, which do sport very high power figures, but also come with very high performance. The best counter-example is the Exynos 9810’s higher frequency states, which come with similarly high power requirements, however don’t showcase an equal increase in performance, thus resulting in a big efficiency disadvantage.

Looking at the wider range of historical SPEC2006 scores, we see the Kirin 980 just losing out to Apple’s A10 in terms of performance. As I had expected some months ago, the A76 largely puts the vastly bigger and more complex Exynos M3 to shame as it manages to post better performance while at the same time using much less power as well as using about half the energy to complete the benchmark.

While Arm has already disclosed the key aspects and improvements of the Cortex-A76 microarchitecture, we can still go over the more detailed SPEC2006 subtests to see if we can extract any further meaningful information:

In SPECint2006, the Kirin 980’s gains are quite relatively even across the board, possibly showcasing a more balanced approach towards the different aspects of the microarchitecture. The biggest generational gains were found in 403.gcc where we see a 2.67x improvement over the Kirin 970. It’s a bit unfortunate that we don’t have a better “apples-to-apples” comparison to the Cortex-A75; the Snapdragon 845’s DRAM memory latency isn’t very good due to its “L4” system cache block, which does handicap it a tad in SPEC.

456.hmmer and 464.h264ref are the two most execution backend bound tests in the suite, and the Cortex-A76 again performs excellently here, showcasing scores that are about in line what you’d expect from the 4-wide microarchitecture as well as clock frequency. I might sound like broken record by now, but again this comes at a great contrast versus Samsung’s M3 core, which in theory just should perform much better than it does.

Something that’s also obvious here is that even though the Cortex-A76 and Kirin 980 are able to show good improvements, it’s not enough to even remotely close the big performance gap in memory latency and bandwidth sensitive tests  – here Apple’s monstrous memory subsystem is just that much further ahead.

In the SPECfp2006 results, we again see some really big improvements of the Kirin 980 versus the 970. Again the generational improvement over the A75 is a bit clouded through the comparison to the Snapdragon 845, which in these memory sensitive tests didn’t manage to separate itself much from the previous Cortex A73 based Snapdragon 835.

Still, all, in all, we see very rounded off improvements across the board in all benchmarks, which fares very well for the Kirin 980 both in terms of performance gains as well as energy usage improvements.

Overall, the Kirin 980 as well as Arm’s Cortex-A76 both delivered on their promises on the CPU side, and even managed to surpass by initial performance projections of the new core. No, the Kirin 980 certainly is not able to match Apple’s A12, or even A11 for that matter, and it’s likely this situation won’t change all that much in the next few generations, at least until the Android SoC vendors invest in significantly better and more robust memory subsystems.

The Kirin 980’s performance here should largely represent what we’ll see in the next generation Snapdragon as well – I expect Qualcomm to be able to push the clocks just slightly more, but the big question here is what they will do on the memory subsystem side and if they’ll be able to get rid of the latency penalty that was introduced with the L4 system cache.

For Samsung, the Cortex-A76 is just scary (Apple aside). If the next generation M4 core is just an iterative microarchitecture, I have a really hard time seeing it compete. Here we’ll need to see major improvements both in performance as well as power efficiency in order to have the next gen Exynos to be able to match the Kirin 980, yet alone beat it.

The Kirin 980 - A Recap Overview Second Generation NPU - NNAPI Tested
Comments Locked

141 Comments

View All Comments

  • cha0z_ - Tuesday, November 20, 2018 - link

    I don't like how the best phone samsung made this year is not here (the note 9). That phone has a lot bigger body vs s9 and 3 times bigger heatpipe that is also better (the body of the phone heats, but the SOC is not throttled. Actually note 9 in heavy use is hotter than the s9+ but sustains better ;) ) + it's tweaked not for peak performance, but sustained performance + samsung DID improve the kernel and the control of the exynos 9810. I am sure all the factors will lead to noticeable difference compared to the s9 exynos tests from the start of the year.

    I know that you are tired of the exynos 9810, we all know that chip is far worse than the rivals, but still it would be better to show it in it's best light instead of the all negativity. Comparing in a single table a phone twitce smaller than the other and drawing conclusions about the SOCs inside is plain wrong.
  • eastcoast_pete - Tuesday, November 20, 2018 - link

    @Andrei:Any statement from Huawei on how long they will continue to provide OS updates for, and how quickly after Google releases them? With prices approaching 1000 dollars/euros/pounds, the old "release and abandon" would be a bit too much. Thanks!
  • abufrejoval - Tuesday, November 20, 2018 - link

    It doesn't get any better: Here you have all the hardware to turn into a credible workstation with sufficient compute, gaming and even inference power to do 90% of what normal PC users would need with UPS, storage and a high resolution touch screen included at pocket size and laptop budgets....

    But you simply cannot get the power onto a screen large enough to work with all day (Miracast is really doesn't have acceptable fidelity)

    And they simply won't let you take control over what could be a very personal and very portable workstation, because they deny you control over the computer you purchased (no rooting).

    All that power in a form factor that precludes putting it to work just drives me knocking my head into the wall!
  • whyamihere - Thursday, November 22, 2018 - link

    If possible do you think you could look at the power consumption of a BOE screen on the Mate 20 pro. I'm wondering if the battery issues you saw on the pro model had to do with the LG screen, as LG screens on the pro model seem to have issues such as really bad green tint that gets worse over time.
  • Jalk44 - Thursday, November 22, 2018 - link

    A: it's not the first QHD phone by Huawei,that's was the mate 9 pro

    B: it's also not the first phone with both front and back full curved glass, that would be the mate rs
  • MyFluxi - Thursday, November 22, 2018 - link

    hey, can you do a screen battery test with the BOE screen and also a general review on the boe screen. some saying that the vibrancy is less on the BOE but the uniformity is better
  • ballsystemlord - Friday, November 23, 2018 - link

    Hi. Your local S+G corrector here. Todays mistake is an obvious one, the word "if" should be substituted by the word "is".
    "Here acceleration if facilitated through the HVX DSPs."
    --
    "Here acceleration is facilitated through the HVX DSPs."
    I lightly read the last 3 pages. I got tired of reading everything.
  • salbashi - Tuesday, November 27, 2018 - link

    Did Anandtech notice any benchmark mode on Mate 20 or Mate 20 Pro this time around?
    Cause that would be Huawei caught cheating again right after P20 and P20 Pro.
  • Davidsic - Wednesday, November 28, 2018 - link

    Hello, my first Mate 20 Pro had the same brightness anomaly you are talking about (LG screen) and my second one that i recieved yesterday have the same issue and it's a BOE screen !
  • AlexTi - Sunday, March 17, 2019 - link

    Mate 20 having Qi wireless charging possibility seems to be a mistake in specifications. The one I've just bought definitely lacks it (model number HMA-L29), and specs on Huawei website do not include this feature for non-Pro Mate 20.

Log in

Don't have an account? Sign up now