Technology Lab / Information Technology

Inside the second: Gaming performance with today’s CPUs

Does the processor you choose still matter?

The Elder Scrolls V: Skyrim

We tested performance using Fraps while taking a stroll around the town of Whiterun in Skyrim. The game was set to the graphical quality settings shown above. Note that we're using fairly high quality visual settings, basically the "ultra" presets at 1920x1080 but with FXAA instead of MSAA. Our test sessions lasted 90 seconds each, and we repeated them five times per CPU.

Frame time
in milliseconds
8.3 120
16.7 60
20 50
25 40
33.3 30
50 20

The plots above show the time required to render each frame from a single test run. You can click on the buttons to switch to results for different brands and classes of processors. Notice that, because we're reporting frame times, lower numbers are preferable to higher ones. You can even see some spikes representing long frame times in each plot. For the confused, we've included the table on the right, which converts some key frame time thresholds into their FPS equivalents. Also note that faster solutions tend to produce more total frames than the slower ones during the test period.

To get a sense of the performance range we're dealing with, flip between the far-left "AMD budget" and far right "Intel extreme" buttons a few times. The fastest Intel processors produce very few frame times above 16.7 milliseconds—that is, they churn out a nearly steady stream of frames at roughly 60 FPS or better. Meanwhile, the slowest budget processors see regular spikes into 30 or 40 millisecond territory. That's not a devastating outcome, but there are substantially more slowdowns than we see with the fastest processors.

All of the CPUs achieve FPS averages near or above the supposedly golden 60 FPS mark. Based purely on common FPS expectations, one might argue that any of them should be more than sufficient for playing Skyrim well.

There are some warning signs here, though. The Intel processors line up as one might expect, roughly in order of age and then model number, with only the slowest legacy dual-core trailing anything from AMD. The three Ivy Bridge parts, in lighter blue, fare well. However, the AMD processors don't quite behave as expected. A prior-gen CPU, the Phenom II X4 980, takes the top spot among them. The three FX processors, in lighter green, don't finish in the order expected, either. The low-end FX-4170 is the fastest of the three, although by a slim margin.

In the past, we might have dismissed these results as part of the noise and as not terribly relevant given the fairly high FPS averages. Look what happens when we turn the focus to frame latencies, though.

This is a snapshot of the frame latency picture; it's the point below which 99% of all frames have been rendered. We're simply excluding the last 1% of frames, many of them potential outliers, to get a sense of overall smoothness.

Again, the Intel processors perform well. All but one of them render the great majority of frames in under 23 milliseconds, which translates to a steady-state frame rate of just under 50 FPS. There is some reshuffling in the move from FPS average to a latency-sensitive metric—the "big iron" Core i7-3820 with its large cache and quad memory channels moves up the ranks, for instance—but the changes are what one might expect, given the hardware in question.

Meanwhile, the AMD FX processors suffer in this comparison. The FX-8150, which is ostensibly AMD's top-of-the-line desktop processor, trails two older Phenom IIs and the FX-4170. The FX-6200 falls behind the A8-3850, a budget APU based on AMD's prior CPU microarchitecture. The absolute numbers aren't stellar, either. The FX processors are cranking out 99% of the frames in 33 milliseconds or so, which translates to a steady rate of 30 FPS—much lower than even the slower Intel processors.

What's the problem? The broader latency curve suggests some answers.

The "tail" of the curve for the AMD processors is telling. Although the FX chips keep pace with the Phenom II X6 1100T in the first 95% or so of frames rendered, their frame times spike upward to meet the slower A8-3850 budget APU and Phenom II X4 850 in the last ~5% of frames. In the most crucial function of gaming performance, latency avoidance, the more expensive FX processors essentially perform like low-end CPUs.

Why? I think the answer is suggested by the relatively strong performance of the FX-4170 compared to the FX-6200 and FX-8150. As we noted, the FX-4170 actually has the highest base and Turbo clock speeds of the FX lineup. That means it likely has the highest per-thread performance of any FX chip, and that appears to translate into better latency mitigation. (I also suspect the FX-4170 spends more time operating near its peak Turbo speed, since it only has to fit two "modules" and four cores into the same 125W thermal envelope as the higher-end FX chips.)

Looks to me like the FX CPUs have an Amdahl's Law problem. Even though they have a relatively large amount of cores for their given product segments, their per-thread performance is fairly weak. The Bulldozer architecture combines relatively low instruction throughput per cycle with clock speeds that aren't as high as AMD probably anticipated. That adds up to modest per-thread performance—and even with lots of cores on hand, the execution speed of a single thread can limit an application's throughput.

Thus, the FX-6200 and FX-8150 processors aren't as well-suited to our Skyrim test scenario as their predecessors in the Phenom II lineup. Only the FX-4170 outperforms the CPU it replaces, the Phenom II X4 850, whose lack of L3 cache and modest 3.3GHz clock frequency aren't doing it any favors.

Do the results from our new methods mean that some AMD processors are inadequate for Skyrim? Not quite. One of our key metrics for frame latency problems involves adding up all of the time spent working on frames above a certain time threshold. We consider it a measure of "badness," giving us a sense of how severe the slowdowns are. We typically start at a threshold of 50 milliseconds, which translates to 20 FPS, since taking longer than that to produce a frame is likely to interrupt the illusion of motion. The thing is, none of the CPUs we tested spends any real time above the 50 ms threshold. They're all adequate enough to deliver relatively decent gameplay. In fact, we've omitted the graph for this threshold, since it doesn't show much.

However, if we crank down the tolerance to 16.7 milliseconds, the equivalent of 60 FPS, then the differences become apparent. The FX processors again fare poorly, relatively speaking. If you covet glassy smoothness, where the system pumps out frames consistently at low latencies close to your display's refresh rate, then you'll want a newer Intel processor. In this scenario, no entry in the FX lineup comes as close to delivering that experience as a Phenom II X4 980 or a Core i5-655K.

Expand full story

You must to comment.