title: What considerations go into predicting latency for operations on modern superscalar processors and how can I calculate them by hand?
body: <p>I want to be able to predict, by hand, exactly how long arbitrary arithmetical (i.e. no branching or memory, though that would be nice too) x86-64 assembly code will take given a particular architecture, taking into account instruction reordering, superscalarity, latencies, CPIs, etc.</p>
  
  <p>What / describe the rules must be followed to achieve this?</p>
  
  <hr>
  
  <p>I think I've got some preliminary rules figured out, but I haven't been able to find any references on breaking down any example code to this level of detail, so I've had to take some guesses. (For example, the Intel optimization manual barely even <em>mentions</em> instruction reordering.)</p>
  
  <p>At minimum, I'm looking for (1) confirmation that each rule is correct or else a correct statement of each rule, and (2) a list of any rules that I may have forgotten.</p>
  
  <ul>
  <li>As many instructions as possible are issued each cycle, starting in-order from the current cycle and potentially as far ahead as the reorder buffer size.</li>
  <li>An instruction can be issued on a given cycle if:
  
  <ul>
  <li>No instructions that affect its operands are still being executed. And:</li>
  <li>If it is a floating-point instruction, every floating-point instruction before it has been issued (floating-point instructions have static instruction re-ordering). And:</li>
  <li>There is a functional unit available for that instruction on that cycle. Every (?) functional unit is pipelined, meaning it can accept 1 new instruction per cycle, and the number of total functional units is 1/CPI, for the CPI of a given function class (nebulous here: presumably e.g. <code>addps</code> and <code>subps</code> use the same functional unit?  How do I determine this?). And:</li>
  <li>Fewer than the superscalar width (typically <code>4</code>) number of instructions have already been issued this cycle.</li>
  </ul></li>
  <li>If no instructions can be issued, the processor simply doesn't issue any—a condition called a "stall".</li>
  </ul>
  
  <hr>
  
  <p>As an example, consider the following example code (which computes a cross-product):</p>
  
  <pre><code>shufps   xmm3, xmm2, 210
  shufps   xmm0, xmm1, 201
  shufps   xmm2, xmm2, 201
  mulps    xmm0, xmm3
  shufps   xmm1, xmm1, 210
  mulps    xmm1, xmm2
  subps    xmm0, xmm1
  </code></pre>
  
  <p>My attempt to predict the latency for Haswell looks something like this:</p>
  
  <pre><code>; `mulps`  Haswell latency=5, CPI=0.5
  ; `shufps` Haswell latency=1, CPI=1
  ; `subps`  Haswell latency=3, CPI=1
  
  shufps   xmm3, xmm2, 210   ; cycle  1
  shufps   xmm0, xmm1, 201   ; cycle  2
  shufps   xmm2, xmm2, 201   ; cycle  3
  mulps    xmm0, xmm3        ;   (superscalar execution)
  shufps   xmm1, xmm1, 210   ; cycle  4
  mulps    xmm1, xmm2        ; cycle  5
                             ; cycle  6 (stall `xmm0` and `xmm1`)
                             ; cycle  7 (stall `xmm1`)
                             ; cycle  8 (stall `xmm1`)
  subps    xmm0, xmm1        ; cycle  9
                             ; cycle 10 (stall `xmm0`)
  </code></pre>