Dissecting Intel's EPYC Benchmarks: Performance Through the Lens of Competitive Analysis
by Johan De Gelas & Ian Cutress on November 28, 2017 9:00 AM EST- Posted in
- CPUs
- AMD
- Intel
- Xeon
- Skylake-SP
- Xeon Platinum
- EPYC
- EPYC 7601
Enterprise & Cloud Benchmarks
Below you can find Intel's internal benchmarking numbers. The EPYC 7601 is the reference (performance=1), the 8160 is represented by the light blue bars, the top of the line 8180 numbers are dark blue. On a performance per dollar metric, it is the light blue worth observing.
Java benchmarks are typically unrealistically tuned, so it is a sign on the wall when an experienced benchmark team is not capable to make the Intel 8160 shine: it is highly likely that the AMD 7601 is faster in real life.
The node.js and PHP runtime benchmarks are very different. Both are open source server frameworks to generate for example dynamic page content. Intel uses a client load generator to generate a real workload. In the case of the PHP runtime, MariaDB (MySQL derivative) 10.2.8 is the backend.
In the case of Node.js, mongo db is the database. A node.js server spawns many different single threaded processes, which is rather ideal for the AMD EPYC processor: all data is kept close to a certain core. These benchmarks are much harder to skew towards a certain CPU family. In fact, Intel's benchmarks seem to indicate that the AMD EPYC processors are pretty interesting alternatives. Surely if Intel can only show a 5% advantage with a 10% more expensive processor, chances are that they perform very much alike in the real world. In that case, AMD has a small but tangible performance per dollar advantage.
The DPDK layer 3 Network Packet Forwarding is what most of us know as routing IP packets. This benchmark is based upon Intel own Data Plane Developer Kit, so it is not a valid benchmark to use for an AMD/Intel comparison.
We'll discuss the database HammerDB, NoSQL and Transaction Processing workloads in a moment.
The second largest performance advantage has been recorded by Intel testing the distributed object caching layer memcached. As Intel notes, the benchmark was not a processing-intensive workload, but rather a network-bound workload. As AMD's dual socket system is seen as a virtual 8-socket system, due to the way that AMD has put four dies onto each processor and each die has a sub-set of PCIe lanes linked to it, AMD is likely at a disadvantage.
Intel's example of network bandwidth limitations in a pseudo-socket configuration
Suppose you have two NICs, which is very common. The data of the first NIC will, for example, arrive in NUMA node 1, Socket 1, only to be accessed by NUMA node 4, Socket 1. As a result, there is some additional latency incurred. In Intel's case, you can redirect a NIC to each socket. With AMD, this has to be locally programmed, to ensure that the packets that are sent to each NICs are processed on each virtual node, although this might incur additional slowdown.
The real question is whether you should bother to use a 2S system for Memached. After all, it is distributed cache layer that scales well over many nodes, so we would prefer a more compact 1S system anyway. In fact, AMD might have an advantage as in the real world, Memcached systems are more about RAM capacity than network or CPU bottlenecks. Missing the additional RAM-as-cache is much more dramatic than waiting a bit longer for a cache hit from another server.
The virtualization benchmark is the most impressive for the Intel CPUs: the 8160 shows a 37% performance improvement. We are willing to believe that all the virtualization improvements have found their way inside the ESXi kernel and that Intel's Xeon can deliver more performance. However, in most cases, most virtualization systems run out of DRAM before they run out of CPU processing power. The benchmarking scenario also has a big question mark, as in the footnotes to the slides Intel achieved this victory by placing 58 VMs on the Xeon 8160 setup versus 42 VMs on the EPYC 7601 setup. This is a highly odd approach to this benchmark.
Of course, the fact that the EPYC CPU has no track record is a disadvantage in the more conservative (VMware based) virtualization world anyway.
Facebook
Twitter
RSS
105 Comments
View All Comments
Johan Steyn - Monday, December 18, 2017 - link
I have stated before that Anandtech is on Intel's payroll. You could see it especially with the first Threadripper review, it was horrendous to say the least. This article goes the same route. You see, two people can say the same thing, but project a completely different picture. I do not disagree that Intel has it strengths over EPYC, but this article basically just agrees with Intel,s presentation. Ha ha, that would have been funny, but it is not.Intel is corrupt company and Anandtech is missing the point on how they present their "facts." I now very rarely read anything Anandtech publishes. In the 90's they were excellent - those were the days...
Jumangi - Tuesday, November 28, 2017 - link
Maybe you have herd of Google..or Facebook. Not only d9 they build but they design their own rack systems to suit their massive needs.Samus - Wednesday, November 29, 2017 - link
Even mom and pop shops shouldn't have servers built from scratch. Who's going to support and validate that hardware for the long haul?HP and Dell have the best servers in my opinion. Top to bottom. Lenovo servers are at best just rehashes of their crappy workstations. If you want to get exotic (I don't) one could consider Supermicro...friends in the industry have always mentioned good luck with them, and good support. But my experience is with the big three.
Ratman6161 - Wednesday, November 29, 2017 - link
You are both wrong in my experience. These days the software that runs on servers usually costs more (often by a wide margin) than the hardware it runs on. I was once running a software package the company paid $320K for on a VM environment of five two socket Dell servers and a SAN where the total hardware cost was $165K. But that was for the whole VM environment that ran many other servers besides the two that ran this package. Even the $165K for the VM environment included VMWare licensing so that was part software too. Considering the resources the two VMs running this package used, the total cost for the project was probably somewhere around 10% hardware and 90% software licensing.For my particular usage, the virtualization numbers are the most important so if we accept these numbers, Intel seems to be the way to go. The $10K CPU's seem pretty outlandish though. For virtualization purposes it seems like there might be more bang for the buck by going with the 8160 and just adding more hosts. Would have to get down to actually doing the math to decide on that one.
meepstone - Thursday, December 7, 2017 - link
So I'm not sure who has the bigger e-peen between eek2121 and CajunArson. The drama in the comments were more entertaining than the article!ddriver - Tuesday, November 28, 2017 - link
Take a chill pill you intel shill :)Go over to servethehome and check results from someone who is not paid to pimp intel. Epyc enjoys ample lead against similarly priced xeons.
The only niche it is at a disadvantage is the low core count high clock speed skus, simply because for some inexplicable reason amd decided to not address that important market.
Lastly, nobody buys those 10+k $$$ xeons with his own money. Those are bought exclusively with "others' money" by people who don't care about purchase value, because they have deals with intel that put a percent of that money right back into their pockets, which is their true incentive. If they could put that money in their pockets directly, they would definitely seek the best purchase value rather than going through intel to essentially launder it for them.
iwod - Tuesday, November 28, 2017 - link
This. Go to servethehome and make up your own mind.lazarpandar - Tuesday, November 28, 2017 - link
It's one thing to sound like a dick, it's another thing to sound like a dick and be wrong at the same time.mkaibear - Tuesday, November 28, 2017 - link
Er, yes, if you want just 128Gb of RAM it may cost you $1,500, but if you actually want to use the capacity of those servers you'll want a good deal more than that.The server mentioned in the Intel example can take 1.5Tb of ECC RAM, at a total cost of about $20k - at which point the cost of the CPU is much less of an impact.
As CajunArson said, a full load of RAM on one of these servers is expensive. Your response of "yes well if you only buy 128Gb of RAM it's not that expensive", while true, is a tad asinine - you're not addressing the point he made.
eek2121 - Tuesday, November 28, 2017 - link
Not every workload requires that the RAM be topped off. We are currently in the middle of building our own private cloud on Hyper-V to replace our AWS presence, which involves building out at multiple datacenters around the country. Our servers have half a terabyte of RAM. Even with that much RAM, CPUs like this would still be (and are) a major factor in the overall cost of the server. The importance for our use case is the ability to scale, not the ability to cram as many VMs into one machine as possible. 2 servers with half a terabyte of RAM are far more valuable to us than 1 server with 1-1.5 terabytes due to redundancy.