Intel® VTune™ Performance Analyzer Basics: What is CPI and how do I use it?

If you have used the Sampling Wizard before to profile your application, you have probably noticed that the VTune Analyzer reports a metric called CPI as part of your results. CPI is commonly referred to as “clocks per instruction”. It is calculated by dividing the number of unhalted processor cycles (“clockticks”) by the number of instructions retired. (On each processor the exact events used to count clockticks and instructions retired may be different, but VTune Analyzer will know the correct ones to use.)

What is the significance of CPI?
The CPI value of an application or function is an indication of how much latency affected its execution. Higher CPI values mean there was more latency in your system – on average, it took more clockticks for an instruction to retire. Latency in your system can be caused by cache misses, I/O, or other bottlenecks.

How do I know when CPI is high?
Unfortunately there are no hard and fast rules. The CPI of a workload will depend both on the code, the processor, and the system configuration. So, the main way to use CPI is by comparing a current CPI value to a baseline CPI for the same workload. For example, suppose you made a change to your system or your code and then ran the VTune Analyzer and collected CPI. If the performance of the application decreased after the change, one way to try to understand what may have happened is to look for functions where CPI increased. If you have made an optimization that improved the runtime of your application, you can look at VTune Analyzer data to see if CPI decreased. If it did you can use that information to help direct you toward further investigations. What caused CPI to decrease? Was it a reduction in cache misses, fewer memory operations, lower memory latency, etc.

What are the pitfalls of using CPI?
CPI can be misleading, so you should understand the pitfalls. CPI (latency) is not the only factor affecting the performance of your code on your system. The other major factor is the number of instructions executed (sometimes called path length). All optimizations or changes you make to your code will affect either the time to execute instructions (CPI) or the number of instructions to execute, or both. Using CPI without considering the number of instructions executed can lead to incorrect interpretation of your results. For example, supposed you vectorized your code and converted your math operations to operate on multiple pieces of data at once. This would have the effect of replacing many single-data math instructions with fewer multiple-data math instructions. This would reduce the number of instructions executed overall in your code, but it would likely raise your CPI because multiple-data instructions are more complex and take longer to execute. In many cases, this vectorization would increase your performance, even though CPI went up.

It is important to be aware of your total instructions executed as well. The number of instructions executed is generally called INST_RETIRED in the VTune Analyzer. If your instructions retired is remaining fairly constant, CPI can be a good indicator of performance (this is the case with system tuning, for example). If both number of instructions and CPI are changing, than you need to look at both metrics to understand why your performance increased or decreased. Finally, an alternative to looking at CPI is to look at stall cycles. Stall cycles are discussed further in the whitepaper Using Intel® VTune™ Performance Analyzer to Optimize Software on Intel® Core™ i7 Processors.
