The 2018 GPU Benchmark Suite & the Test

Another year marks another update to our GPU benchmark suite. This time, however, is more in line with a maintenance update than it is a complete overhaul. Although we've done some extended compute and deep learning benchmarking in the past year, and even some HDR gaming impressions, our compute and synthetic lineup remains largely the same. But before getting into the details, let's start with the bulk of benchmarking, and the biggest reason for these cards anyhow: games.

Joining the 2018 game list is Far Cry 5, Wolfenstein II, Final Fantasy XV and Middle-earth: Shadow of War. We are also bringing in F1 2018 and Total War: Warhammer II. Returning from last year is Battlefield 1, Ashes of the Singularity: Escalation, and Grand Theft Auto V. All-in-all, these games span multiple genres, differing graphics workloads, and contemporary APIs, with a nod towards modern and relatively intensive games.

AnandTech GPU Bench 2018 Game List
Game Genre Release Date API(s)
Battlefield 1 FPS Oct. 2016 DX11
(DX12)
Far Cry 5 FPS Mar. 2018 DX11
Ashes of the Singularity: Escalation RTS Mar. 2016 DX12
(DX11, Vulkan)
Wolfenstein II: The New Colossus FPS Oct. 2017 Vulkan
Final Fantasy XV: Windows Edition JRPG Mar. 2018 DX11
Grand Theft Auto V Action/Open world Apr. 2015 DX11
Middle-earth: Shadow of War Action/RPG Sep. 2017 DX11
F1 2018 Racing Aug. 2018 DX11
Total War: Warhammer II RTS Sep. 2017 DX11
(DX12)

That said, Ashes as a DX12 trailblazer may not be as hot and fresh as it once was, especially considering that the pace of DX12 and Vulkan adoption in new games has waned. The circumstances are worth an investigation on their own, but the learning curve required in modern low-level API and the subsequent return may not be convincing right now. As a more general remark, most developers and publishers tend not to advertise or document DX12 support as much as they used to, nor is it clearly labelled in game specifications as many times DX11 is the unmentioned default.

Particularly for NVIDIA and GeForce RTX, pushing DXR and raytracing means pushing DX12, of which DXR is a component. The API has a backstop in the form of Xbox consoles and Windows 10, and if multi-GPU is to make a comeback, whether that's via compatible workloads (VR), flexible usage (ray tracing workload topologies), or just the plain old inevitability of Moore's Law. So this is less likely to be the slow end of DX12.

In terms of data collection, measurements were gathered either using built-in benchmark tools or with AMD's open-source Open Capture and Analytics Tool (OCAT), which is itself powered by Intel's PresentMon. 99th percentiles were obtained or calculated in a similar fashion, as OCAT natively obtains 99th percentiles. In general, we prefer 99th percentiles over minimums, as they more accurately represent the gaming experience and filter out any artificial outliers.

We've also swapped out Blenchmark, which seems to have been abandoned in terms of updates, in favor of a BMW render from the Blender Institute Cycles Benchmark, and a more recent one from a Cycles benchmark developer on Blenderartists.org. There were concerns with Blenchmark's small tile size, which is not very applicable to GPUs, and in terms of usability we also ran into some GPU detection errors which were linked to inaccurate Blenchmark Python code.

Otherwise, we are also keeping an eye on a few trends and upcoming developments:

  • MLPerf machine learning benchmark suite
  • Blender Benchmark
  • Futuremark's 3DMark DirectX Raytracing benchmark
  • DXR and Vulkan raytracing extension support in games

Another point is that we do not have a permanent HDR monitor for our testbed, which would be necessary to incorporate HDR game testing in the near future; 5 games in our list actually support HDR. And as we look at technologies that enhance or alter image quality (e.g. HDR, Turing's DLSS), we will want to find a better way of comparing differences. This is particularly tricky with HDR as screenshots are inapplicable and even taking accurate photographs will most likely be viewed on an SDR screen. With DLSS, there is a built-in reference quality based on 64x supersampling, which in deep learning terms is the 'ground truth'; an intuitive solution would be to use a neural network based method of analyzing quality differences, but that is likely beyond our scope.

The following tech demos and test applications were provided via NVIDIA:

  • Star Wars 'Reflections' Demo (includes real time ray tracing and DLSS support)
  • Final Fantasy XV Official Benchmark (includes DLSS support)
  • Asteroids Demo (features mesh shading and variable LOD)
  • Epic Infiltrator Demo (features DLSS)

The Testbed

Because NVIDIA is not productizing any other reference-quality GeForce RTX 2080 Ti and 2080 card besides the Founders Editions, which are non-reference by specifications, we've gone ahead and emulated the true reference specifications with a 90MHz downclock and lowering the TDP by roughly 10W. This is to keep comparisons standardized and apples-to-apples, as we always look at reference-to-reference results.

In a classic case of Murphy's Law, our usual PSU started malfunctioning around the time of the review, but given the time constraints we couldn't do a 1:1 replacement in time. As it is a digital PSU, we were beginning to use it for PCIe power readings to augment system measurements, but for now we will have to stick power draw at the wall. For the time being, we've swapped it out with another high-quality and high-wattage PSU.

CPU: Intel Core i7-7820X @ 4.3GHz
Motherboard: Gigabyte X299 AORUS Gaming 7 (F9g)
Power Supply: Corsair AX860i
EVGA 1000 G3
Hard Disk: OCZ Toshiba RD400 (1TB)
Memory: G.Skill TridentZ DDR4-3200 4 x 8GB (16-18-18-38)
Case: NZXT Phantom 630 Windowed Edition
Monitor: LG 27UD68P-B
Video Cards: AMD Radeon RX Vega 64 (Air Cooled)
NVIDIA GeForce RTX 2080 Ti Founders Edition
NVIDIA GeForce RTX 2080 Founders Edition

NVIDIA GeForce GTX 1080 Ti Founders Edition
NVIDIA GeForce GTX 1080 Founders Edition
NVIDIA GeForce GTX 980 Ti
NVIDIA GeForce GTX 980
Video Drivers: NVIDIA Release 411.51 Press
AMD Radeon Software Adrenalin Edition 18.9.1
OS: Windows 10 Pro (April 2018 Update)
Spectre/Meltdown Mitigations Yes, both
Meet The GeForce RTX 2080 Ti & RTX 2080 Founders Editions Cards Battlefield 1
Comments Locked

337 Comments

View All Comments

  • milkod2001 - Thursday, September 20, 2018 - link

    NV is messing with us. Even with no competition from AMD those price hikes at such low performance gains are laughable. This generation of new GPU seems like just a stop gap before NV will have something more serious to show next year.
  • willis936 - Thursday, September 20, 2018 - link

    No they seem like they will be exactly the same as the 1000 series: they are what they are, you pay what they ask, and they will be the only decent option they offer for the next two years.

    Maybe if Radeon ever gets their shit together the landscape might look different in 2-3 years but trust me: for now, expect more of the same.
  • milkod2001 - Thursday, September 20, 2018 - link

    Yeah we are pretty much getting into Intel vs AMD scenario when Intel dominates for a years and bring customers overpriced products with very slow performance upgrades. There is a hope AMD will at least try to do something about it.
  • yhselp - Thursday, September 20, 2018 - link

    The temperature and noise results are shocking. The results are much closer to what you'd expect from a blower, rather than an open-air cooler. Previous gen OEM solutions do much better than this. What's the reason for this?
  • milkod2001 - Thursday, September 20, 2018 - link

    Chips are much bigger than previous gen.
  • iwod - Thursday, September 20, 2018 - link

    I think we need DLSS and Hybrid Ray Tracing to judge whether it is worth it. At the moment, we could have the nearly double the performance of 1080Ti if we simply have a 7xxmm2 Die of it.

    I think the idea Nvidia had is that we have reached the plateau of Graphic Gaming. Imagine what you could do with a 7nm 7xxm2 Die of 2080Ti? Move the 4K Ultra High Quality frame rate from ~60 to 100? That is in 2019, in 2022 3nm, double that frame rate from 100 to 200?

    The industry as a whole needs to figure out how to extract even more Graphics Quality with less transistors, simpler software while at the same time makes 3D design modelling easier. The graphics assets from gaming are now worth 100s to millions. Just the asset, not engine programming, network, 3D interaction etc, nothing to do with code. Just the Graphics. And Hybrid Ray tracing is an attempt to bring more quality graphics without the ever increasing cost of Engine and graphics designer simulating those effect.

    What is interesting is that we will have 8 Core 5Ghz CPU and 7nm GPU next year.
  • Chawitsch - Thursday, September 20, 2018 - link

    Given how much die space is dedicated to the new features software support will definitely be the key for these cards' success. Otherwise their price is just too high for what they offer today. Buying these cards now is somewhat of a gamble, but nVidia does have excellent relations with developers however, so support should come. As someone who would like to have a capable GPU for 100+ FPS gaming at 1440p, especially one that is future proof, I would much rather take my chances with these new cards.

    To me the question is this, would it really be worth focusing even more on 4k gaming, when it is a fairly niche market segment still due to monitor prices (especially ones with low latency for gaming). Arguably these high end cards are niche too, but when we can already have 4k@60 FPS, with maxed graphics settings, other considerations become more important. At any given resolution and feature level pure performance becomes meaningless after a certain point, at least for gaming. Arguing that reaching 100 FPS at 4k definitely has merit in my opinion, but by the time really good 4k monitors take over we'll get there, even with the path nVidia took.

    Regarding graphics quality and transistor count, ray tracing should be a win here, if not now in the future certainly. There are diminishing returns with rasterization as you approach more realistic scenes and ray tracing makes you jump through less hoops to if you want to create a correct looking scene.
  • MadManMark - Thursday, September 20, 2018 - link

    "I think the idea Nvidia had is that we have reached the plateau of Graphic Gaming. Imagine what you could do with a 7nm 7xxm2 Die of 2080Ti?"

    Yes, but that is probably why they stuck with 12nmFF actually. Note the die size, plus each card has its own GPU, rather than binned selection from the same GPU (kudos to Nate for also ruminating briefly on this in text). This means maximizing yiled is particularly important, and so begs for a mature, efficient process. TSMC achieved great things with their current 7nm process, no knock on it, but it is still UV-based, it's been long documented that there are yield challengels with that. IMO Nvidia will wait to hitch their wagons to TSMC's next process (expected next year), EUV-based 7nm+, which is expected to mitigate a lot of these yield concerns.

    In other words it will be very interesting to see what the 2180 Ti looks like next year -- yes, I built a lot of assumptions into that sentence ;)
  • eddman - Thursday, September 20, 2018 - link

    Come on, the naming is already set; 1080, 2080, 3080. What the hell is a "2180"?

    P.S. OCD
  • Lolimaster - Saturday, September 22, 2018 - link

    That works if you expect graphics to be stagnant, tons of mini effects and polygon count will chunk a current 1080ti to 10fps in 2021.

Log in

Don't have an account? Sign up now