title: Why does a loop transitioning from having its uops fed by the Uop Cache to LSD cause a spike in branch-misses?
body: <p>All benchmarks are run on either
  <strong><a href="https://ark.intel.com/content/www/us/en/ark/products/196597/intel-core-i7-1065g7-processor-8m-cache-up-to-3-90-ghz.html" rel="nofollow noreferrer">Icelake</a></strong>
  or <strong><a href="https://ark.intel.com/content/www/us/en/ark/products/149091/intel-core-i7-8565u-processor-8m-cache-up-to-4-60-ghz.html" rel="nofollow noreferrer">Whiskey Lake</a></strong> (In Skylake Family).</p>
  <h2>Summary</h2>
  <p>I am seeing a strange phenomina where it appears that when a loop
  transitions from running out of the <strong>Uop Cache</strong> to running out of
  the <strong>LSD (Loop Stream Detector)</strong> there is a spike in <strong>Branch
  Misses</strong> that can cause severe performance hits. I tested on both
  Icelake and Whiskey Lake by benchmarking a nested loop with the outer loop
  having a sufficiently large body s.t the entire thing did not fit in
  the <strong>LSD</strong> itself, but with an inner loop small enough to fit in the
  <strong>LSD</strong>.</p>
  <p>Basically once the inner loop reaches some iteration count decoding
  seems to switch for <strong>idq.dsb_uops (Uop Cache)</strong> to <strong>lsd.uops (LSD)</strong>
  and at that point there is a large increase in <strong>branch-misses</strong>
  (without a corresponding jump in <strong>branches</strong>) causing a severe
  performance drop. <strong>Note: This only seems to occur for nested
  loops. <a href="https://github.com/travisdowns/x86-loop-test" rel="nofollow noreferrer">Travis Down's Loop
  Test</a> for example will
  not show any meaningful variation in branch misses</strong>. AFAICT this has
  something to do with when a loop transitions from running out of the
  <strong>Uop Cache</strong> to running out of the <strong>LSD</strong>.</p>
  <h2>Questions</h2>
  <ol>
  <li><p>What is happening when the loop transitions from running out of the
  <strong>Uop Cache</strong> to running out of the <strong>LSD</strong> that causes this spike in
  <strong>Branch Misses</strong>?</p>
  </li>
  <li><p>Is there a way to avoid this?</p>
  </li>
  </ol>
  <h2>Benchmark</h2>
  <p>This is the minimum reproducible example I could come up with:</p>
  <p>Note: If the <code>.p2align</code> statements are removed both loops will fit in
  the LSD and there will not be a transitions.</p>
  <pre><code>#include &lt;assert.h&gt;
  #include &lt;stdint.h&gt;
  #include &lt;stdio.h&gt;
  #include &lt;stdlib.h&gt;
  #include &lt;string.h&gt;
  
  #define BENCH_ATTR __attribute__((noinline, noclone, aligned(4096)))
  
  static const uint64_t outer_N = (1UL &lt;&lt; 24);
  
  
  static void BENCH_ATTR
  bench(uint64_t inner_N) {
      uint64_t inner_loop_cnt, outer_loop_cnt;
      asm volatile(
          &quot;.p2align 12\n&quot;
          &quot;movl   %k[outer_N], %k[outer_loop_cnt]\n&quot;
          &quot;.p2align   6\n&quot;
          &quot;1:\n&quot;
          &quot;movl   %k[inner_N], %k[inner_loop_cnt]\n&quot;
          // Extra align surrounding inner loop so that the entire thing
          // doesn't execute out of LSD.
          &quot;.p2align   10\n&quot;
          &quot;2:\n&quot;
          &quot;decl   %k[inner_loop_cnt]\n&quot;
          &quot;jnz    2b\n&quot;
          &quot;.p2align   10\n&quot;
          &quot;decl   %k[outer_loop_cnt]\n&quot;
          &quot;jnz    1b\n&quot;
          : [ inner_loop_cnt ] &quot;=&amp;r&quot;(inner_loop_cnt),
            [ outer_loop_cnt ] &quot;=&amp;r&quot;(outer_loop_cnt)
          : [ inner_N ] &quot;ri&quot;(inner_N), [ outer_N ] &quot;i&quot;(outer_N)
          :);
  }
  int
  main(int argc, char ** argv) {
      assert(argc &gt; 1);
      uint64_t inner_N = atoi(argv[1]);
      bench(inner_N);
  }
  </code></pre>
  <p><strong>Compile</strong>: <code>gcc -O3 -march=native -mtune=native &lt;filename&gt;.c -o &lt;filename&gt;</code></p>
  <p><strong>Run Icelake</strong>: <code>sudo perf stat -C 0 --all-user -e cycles -e branches -e branch-misses -x, -e idq.ms_uops -e idq.dsb_uops -e lsd.uops taskset -c 0 ./&lt;filename&gt; &lt;N inner loop iterations&gt;</code></p>
  <p><strong>Run Whiskey Lake</strong>: <code>sudo perf stat -C 0 -e cycles -e branches -e branch-misses -x, -e idq.ms_uops -e idq.dsb_uops -e lsd.uops taskset -c 0 ./&lt;filename&gt; &lt;N inner loop iterations&gt;</code></p>
  <h2>Graphs</h2>
  <p>Edit: x label is N iterations of inner loop.</p>
  <p>Below is a graph of <strong>Branch Misses</strong>, <strong>Branches</strong>, and <strong>LSD Uops</strong>.</p>
  <p>Generally you can see that 1) there is no corresponding jump in <strong>Branches</strong>. 2) that the number of added <strong>Branch Misses</strong> stabilizes at a constant. And 3) That there is a strong relationship between the <strong>Branch Misses</strong> and <strong>LSD Uops</strong>.</p>
  <p><strong>Icelake Graph</strong>:</p>
  <p><a href="https://i.stack.imgur.com/2j8Ej.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/2j8Ej.png" alt="icl-branch" /></a></p>
  <p><strong>Whiskey Lake Graph</strong>:</p>
  <p><a href="https://i.stack.imgur.com/LjLZO.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/LjLZO.png" alt="skl-branch" /></a></p>
  <p>Below is a graph of <strong>Branch Misses</strong>, <strong>Cycles</strong>, and <strong>LSD Uops</strong>
  for <strong>Icelake</strong> only as performance is not affected nearly as much on:</p>
  <p><a href="https://i.stack.imgur.com/3IwQZ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/3IwQZ.png" alt="icl-cycles" /></a></p>
  <h2>Analysis</h2>
  <p>Hard numbers below.</p>
  <p>For <strong>Icelake</strong> starting at <code>N = 22</code> and finishing at <code>N = 27</code> there
  is some fluctuation in the number of uops coming from the <strong>LSD</strong> vs
  <strong>Uop Cache</strong> and during that time <strong>Branch Misses</strong> increases by
  roughly 3 order of magnitude from <em>10^4 -&gt; 10^7</em>. During this period
  <strong>Cycles</strong> also increased by a factor of <em>2</em>. For all <code>N &gt; 27</code>
  <strong>Branch Misses</strong> stays at around <em>1.67 x 10^7</em> (roughly <code>outer_loop_N</code>). For <code>N = [17, 40]</code>
  <strong>branches</strong> continues to only increase linearly.</p>
  <p>The results for <strong>Whiskey Lake</strong> look different in that 1) <code>N</code> begins fluctuating at <code>N = 35</code> and continues to fluctuate until <code>N = 49</code>. And 2) there is less of a
  performance impact and more fluctuation in the data. That being said
  the increase in <strong>Branch-Misses</strong> corresponding with transitions from
  uops being fed by <strong>Uop Cache</strong> to being fed by <strong>LSD</strong> still exists.</p>
  <h2>Results</h2>
  <p>Data is mean result for 25 runs.</p>
  <p><strong>Icelake Results</strong>:</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>N</th>
  <th>cycles</th>
  <th>branches</th>
  <th>branch-misses</th>
  <th>idq.ms_uops</th>
  <th>idq.dsb_uops</th>
  <th>lsd.uops</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>1</td>
  <td>33893260</td>
  <td>67129521</td>
  <td>1590</td>
  <td>43163</td>
  <td>115243</td>
  <td>83908732</td>
  </tr>
  <tr>
  <td>2</td>
  <td>42540891</td>
  <td>83908928</td>
  <td>1762</td>
  <td>49023</td>
  <td>142909</td>
  <td>100690381</td>
  </tr>
  <tr>
  <td>3</td>
  <td>50725933</td>
  <td>100686143</td>
  <td>1782</td>
  <td>47656</td>
  <td>142506</td>
  <td>117440256</td>
  </tr>
  <tr>
  <td>4</td>
  <td>67533597</td>
  <td>117461172</td>
  <td>1655</td>
  <td>52538</td>
  <td>186123</td>
  <td>134158311</td>
  </tr>
  <tr>
  <td>5</td>
  <td>68022910</td>
  <td>134238387</td>
  <td>1711</td>
  <td>53405</td>
  <td>204481</td>
  <td>150954035</td>
  </tr>
  <tr>
  <td>6</td>
  <td>85543126</td>
  <td>151018722</td>
  <td>1924</td>
  <td>62445</td>
  <td>141397</td>
  <td>167633971</td>
  </tr>
  <tr>
  <td>7</td>
  <td>84847823</td>
  <td>167799220</td>
  <td>1935</td>
  <td>60248</td>
  <td>160146</td>
  <td>184563523</td>
  </tr>
  <tr>
  <td>8</td>
  <td>101532158</td>
  <td>184570060</td>
  <td>1709</td>
  <td>60064</td>
  <td>361208</td>
  <td>201100179</td>
  </tr>
  <tr>
  <td>9</td>
  <td>101864898</td>
  <td>201347253</td>
  <td>1773</td>
  <td>63827</td>
  <td>459873</td>
  <td>217780207</td>
  </tr>
  <tr>
  <td>10</td>
  <td>118024033</td>
  <td>218124499</td>
  <td>1698</td>
  <td>59480</td>
  <td>177223</td>
  <td>234834304</td>
  </tr>
  <tr>
  <td>11</td>
  <td>118644416</td>
  <td>234908571</td>
  <td>2201</td>
  <td>62514</td>
  <td>422977</td>
  <td>251503052</td>
  </tr>
  <tr>
  <td>12</td>
  <td>134627567</td>
  <td>251678909</td>
  <td>1679</td>
  <td>57262</td>
  <td>133462</td>
  <td>268435650</td>
  </tr>
  <tr>
  <td>13</td>
  <td>285607942</td>
  <td>268456135</td>
  <td>1770</td>
  <td>74070</td>
  <td>285032524</td>
  <td>315423</td>
  </tr>
  <tr>
  <td>14</td>
  <td>302717754</td>
  <td>285233352</td>
  <td>1731</td>
  <td>74663</td>
  <td>302101097</td>
  <td>15953</td>
  </tr>
  <tr>
  <td>15</td>
  <td>321627434</td>
  <td>302010569</td>
  <td>81796</td>
  <td>77831</td>
  <td>319192830</td>
  <td>1520819</td>
  </tr>
  <tr>
  <td>16</td>
  <td>337876736</td>
  <td>318787786</td>
  <td>71638</td>
  <td>77056</td>
  <td>335904260</td>
  <td>1265766</td>
  </tr>
  <tr>
  <td>17</td>
  <td>353054773</td>
  <td>335565563</td>
  <td>1798</td>
  <td>79839</td>
  <td>352434780</td>
  <td>15879</td>
  </tr>
  <tr>
  <td>18</td>
  <td>369800279</td>
  <td>352344970</td>
  <td>1978</td>
  <td>79863</td>
  <td>369229396</td>
  <td>16790</td>
  </tr>
  <tr>
  <td>19</td>
  <td>386921048</td>
  <td>369119438</td>
  <td>1972</td>
  <td>84075</td>
  <td>385984022</td>
  <td>16115</td>
  </tr>
  <tr>
  <td>20</td>
  <td>404248461</td>
  <td>385896655</td>
  <td>29454</td>
  <td>85348</td>
  <td>402790977</td>
  <td>510176</td>
  </tr>
  <tr>
  <td>21</td>
  <td>421100725</td>
  <td>402673872</td>
  <td>37598</td>
  <td>83400</td>
  <td>419537730</td>
  <td>729397</td>
  </tr>
  <tr>
  <td>22</td>
  <td>519623794</td>
  <td>419451095</td>
  <td>4447767</td>
  <td>91209</td>
  <td>431865775</td>
  <td>97827331</td>
  </tr>
  <tr>
  <td>23</td>
  <td>702206338</td>
  <td>436228323</td>
  <td>12603617</td>
  <td>109064</td>
  <td>427880075</td>
  <td>327661987</td>
  </tr>
  <tr>
  <td>24</td>
  <td>710626194</td>
  <td>453005538</td>
  <td>12316933</td>
  <td>106929</td>
  <td>432926173</td>
  <td>344838509</td>
  </tr>
  <tr>
  <td>25</td>
  <td>863214037</td>
  <td>469782765</td>
  <td>14641887</td>
  <td>121776</td>
  <td>428085132</td>
  <td>614871430</td>
  </tr>
  <tr>
  <td>26</td>
  <td>761037251</td>
  <td>486559974</td>
  <td>13067814</td>
  <td>113011</td>
  <td>438093034</td>
  <td>418124984</td>
  </tr>
  <tr>
  <td>27</td>
  <td>832686921</td>
  <td>503337195</td>
  <td>16381350</td>
  <td>113953</td>
  <td>421924080</td>
  <td>556915419</td>
  </tr>
  <tr>
  <td>28</td>
  <td>854713119</td>
  <td>520114412</td>
  <td>16642396</td>
  <td>124448</td>
  <td>420515666</td>
  <td>598907353</td>
  </tr>
  <tr>
  <td>29</td>
  <td>869873144</td>
  <td>536891629</td>
  <td>16572581</td>
  <td>119280</td>
  <td>421188631</td>
  <td>629696780</td>
  </tr>
  <tr>
  <td>30</td>
  <td>889642335</td>
  <td>553668847</td>
  <td>16717446</td>
  <td>120116</td>
  <td>420086570</td>
  <td>668628871</td>
  </tr>
  <tr>
  <td>31</td>
  <td>906912275</td>
  <td>570446064</td>
  <td>16735759</td>
  <td>126094</td>
  <td>419970933</td>
  <td>702822733</td>
  </tr>
  <tr>
  <td>32</td>
  <td>923023862</td>
  <td>587223281</td>
  <td>16706519</td>
  <td>132498</td>
  <td>420332680</td>
  <td>735003892</td>
  </tr>
  <tr>
  <td>33</td>
  <td>940308170</td>
  <td>604000498</td>
  <td>16744992</td>
  <td>124365</td>
  <td>419945191</td>
  <td>770185745</td>
  </tr>
  <tr>
  <td>34</td>
  <td>957075000</td>
  <td>620777716</td>
  <td>16726856</td>
  <td>133675</td>
  <td>420215897</td>
  <td>802779119</td>
  </tr>
  <tr>
  <td>35</td>
  <td>974557538</td>
  <td>637554932</td>
  <td>16763071</td>
  <td>134871</td>
  <td>419764866</td>
  <td>838012637</td>
  </tr>
  <tr>
  <td>36</td>
  <td>991110971</td>
  <td>654332149</td>
  <td>16772560</td>
  <td>130903</td>
  <td>419641144</td>
  <td>872037131</td>
  </tr>
  <tr>
  <td>37</td>
  <td>1008489575</td>
  <td>671109367</td>
  <td>16757219</td>
  <td>138788</td>
  <td>419900997</td>
  <td>904638287</td>
  </tr>
  <tr>
  <td>38</td>
  <td>1024971256</td>
  <td>687886583</td>
  <td>16772585</td>
  <td>139782</td>
  <td>419663863</td>
  <td>938988917</td>
  </tr>
  <tr>
  <td>39</td>
  <td>1041404669</td>
  <td>704669411</td>
  <td>16776722</td>
  <td>137681</td>
  <td>419617131</td>
  <td>972896126</td>
  </tr>
  <tr>
  <td>40</td>
  <td>1058594326</td>
  <td>721441018</td>
  <td>16773492</td>
  <td>142959</td>
  <td>419662133</td>
  <td>1006109192</td>
  </tr>
  <tr>
  <td>41</td>
  <td>1075179100</td>
  <td>738218235</td>
  <td>16776636</td>
  <td>141185</td>
  <td>419601996</td>
  <td>1039892900</td>
  </tr>
  <tr>
  <td>42</td>
  <td>1092093726</td>
  <td>754995452</td>
  <td>16776233</td>
  <td>142793</td>
  <td>419611902</td>
  <td>1073373451</td>
  </tr>
  <tr>
  <td>43</td>
  <td>1108706464</td>
  <td>771773224</td>
  <td>16776359</td>
  <td>139500</td>
  <td>419610885</td>
  <td>1106976114</td>
  </tr>
  <tr>
  <td>44</td>
  <td>1125413652</td>
  <td>788549886</td>
  <td>16764637</td>
  <td>143717</td>
  <td>419889127</td>
  <td>1139628280</td>
  </tr>
  <tr>
  <td>45</td>
  <td>1142023614</td>
  <td>805327103</td>
  <td>16778640</td>
  <td>144397</td>
  <td>419558217</td>
  <td>1174329696</td>
  </tr>
  <tr>
  <td>46</td>
  <td>1158833317</td>
  <td>822104321</td>
  <td>16765518</td>
  <td>148045</td>
  <td>419889914</td>
  <td>1206833484</td>
  </tr>
  <tr>
  <td>47</td>
  <td>1175665684</td>
  <td>838881537</td>
  <td>16778437</td>
  <td>148347</td>
  <td>419562885</td>
  <td>1241397845</td>
  </tr>
  <tr>
  <td>48</td>
  <td>1192454164</td>
  <td>855658755</td>
  <td>16778865</td>
  <td>153651</td>
  <td>419552747</td>
  <td>1275006511</td>
  </tr>
  <tr>
  <td>49</td>
  <td>1210199084</td>
  <td>872436025</td>
  <td>16778287</td>
  <td>152468</td>
  <td>419599314</td>
  <td>1307945613</td>
  </tr>
  <tr>
  <td>50</td>
  <td>1226321832</td>
  <td>889213188</td>
  <td>16778464</td>
  <td>155552</td>
  <td>419572344</td>
  <td>1341893668</td>
  </tr>
  <tr>
  <td>51</td>
  <td>1242886388</td>
  <td>905990406</td>
  <td>16778745</td>
  <td>155401</td>
  <td>419559249</td>
  <td>1375589883</td>
  </tr>
  <tr>
  <td>52</td>
  <td>1259559053</td>
  <td>922767623</td>
  <td>16778809</td>
  <td>154847</td>
  <td>419554082</td>
  <td>1409206082</td>
  </tr>
  <tr>
  <td>53</td>
  <td>1276875799</td>
  <td>939544839</td>
  <td>16778460</td>
  <td>162521</td>
  <td>419576455</td>
  <td>1442424993</td>
  </tr>
  <tr>
  <td>54</td>
  <td>1293113199</td>
  <td>956322057</td>
  <td>16778931</td>
  <td>154913</td>
  <td>419550955</td>
  <td>1476316161</td>
  </tr>
  <tr>
  <td>55</td>
  <td>1310449232</td>
  <td>973099274</td>
  <td>16778534</td>
  <td>157364</td>
  <td>419578102</td>
  <td>1509485876</td>
  </tr>
  <tr>
  <td>56</td>
  <td>1327022109</td>
  <td>989876491</td>
  <td>16778794</td>
  <td>162881</td>
  <td>419562403</td>
  <td>1543193559</td>
  </tr>
  <tr>
  <td>57</td>
  <td>1344097516</td>
  <td>1006653708</td>
  <td>16778906</td>
  <td>157486</td>
  <td>419567545</td>
  <td>1576414302</td>
  </tr>
  <tr>
  <td>58</td>
  <td>1362935064</td>
  <td>1023430928</td>
  <td>16778959</td>
  <td>315120</td>
  <td>419583132</td>
  <td>1609691339</td>
  </tr>
  <tr>
  <td>59</td>
  <td>1381567560</td>
  <td>1040208143</td>
  <td>16778564</td>
  <td>179997</td>
  <td>419661259</td>
  <td>1640660745</td>
  </tr>
  <tr>
  <td>60</td>
  <td>1394829416</td>
  <td>1056985359</td>
  <td>16778779</td>
  <td>167613</td>
  <td>419575969</td>
  <td>1677034188</td>
  </tr>
  <tr>
  <td>61</td>
  <td>1411847237</td>
  <td>1073762626</td>
  <td>16778071</td>
  <td>166332</td>
  <td>419613028</td>
  <td>1710194702</td>
  </tr>
  <tr>
  <td>62</td>
  <td>1428918439</td>
  <td>1090539795</td>
  <td>16778409</td>
  <td>168073</td>
  <td>419610487</td>
  <td>1743644637</td>
  </tr>
  <tr>
  <td>63</td>
  <td>1445223241</td>
  <td>1107317011</td>
  <td>16778486</td>
  <td>172446</td>
  <td>419591254</td>
  <td>1777573503</td>
  </tr>
  <tr>
  <td>64</td>
  <td>1461530579</td>
  <td>1124094228</td>
  <td>16769606</td>
  <td>169559</td>
  <td>419970612</td>
  <td>1810351736</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p><strong>Whiskey Lake Results</strong>:</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>N</th>
  <th>cycles</th>
  <th>branches</th>
  <th>branch-misses</th>
  <th>idq.dsb_uops</th>
  <th>lsd.uops</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>1</td>
  <td>8332553879</td>
  <td>35005847</td>
  <td>37925</td>
  <td>1799462</td>
  <td>6019</td>
  </tr>
  <tr>
  <td>2</td>
  <td>8329926329</td>
  <td>51163346</td>
  <td>34338</td>
  <td>1114352</td>
  <td>5919</td>
  </tr>
  <tr>
  <td>3</td>
  <td>8357233041</td>
  <td>67925775</td>
  <td>32270</td>
  <td>9241935</td>
  <td>5555</td>
  </tr>
  <tr>
  <td>4</td>
  <td>8379609449</td>
  <td>85364250</td>
  <td>35667</td>
  <td>18215077</td>
  <td>5712</td>
  </tr>
  <tr>
  <td>5</td>
  <td>8394301337</td>
  <td>101563554</td>
  <td>33177</td>
  <td>26392216</td>
  <td>2159</td>
  </tr>
  <tr>
  <td>6</td>
  <td>8409830612</td>
  <td>118918934</td>
  <td>35007</td>
  <td>35318763</td>
  <td>5295</td>
  </tr>
  <tr>
  <td>7</td>
  <td>8435794672</td>
  <td>135162597</td>
  <td>35592</td>
  <td>43033739</td>
  <td>4478</td>
  </tr>
  <tr>
  <td>8</td>
  <td>8445843118</td>
  <td>152636271</td>
  <td>37802</td>
  <td>52154850</td>
  <td>5629</td>
  </tr>
  <tr>
  <td>9</td>
  <td>8459141676</td>
  <td>168577876</td>
  <td>30766</td>
  <td>59245754</td>
  <td>1543</td>
  </tr>
  <tr>
  <td>10</td>
  <td>8475484632</td>
  <td>185354280</td>
  <td>30825</td>
  <td>68059212</td>
  <td>4672</td>
  </tr>
  <tr>
  <td>11</td>
  <td>8493529857</td>
  <td>202489273</td>
  <td>31703</td>
  <td>77386249</td>
  <td>5556</td>
  </tr>
  <tr>
  <td>12</td>
  <td>8509281533</td>
  <td>218912407</td>
  <td>32133</td>
  <td>84390084</td>
  <td>4399</td>
  </tr>
  <tr>
  <td>13</td>
  <td>8528605921</td>
  <td>236303681</td>
  <td>33056</td>
  <td>93995496</td>
  <td>2093</td>
  </tr>
  <tr>
  <td>14</td>
  <td>8553971099</td>
  <td>252439989</td>
  <td>572416</td>
  <td>99700289</td>
  <td>2477</td>
  </tr>
  <tr>
  <td>15</td>
  <td>8558526147</td>
  <td>269148605</td>
  <td>29912</td>
  <td>109772044</td>
  <td>6121</td>
  </tr>
  <tr>
  <td>16</td>
  <td>8576658106</td>
  <td>286414453</td>
  <td>29839</td>
  <td>118504526</td>
  <td>5850</td>
  </tr>
  <tr>
  <td>17</td>
  <td>8591545887</td>
  <td>302698593</td>
  <td>28993</td>
  <td>126409458</td>
  <td>4865</td>
  </tr>
  <tr>
  <td>18</td>
  <td>8611628234</td>
  <td>319960954</td>
  <td>32568</td>
  <td>136298306</td>
  <td>5066</td>
  </tr>
  <tr>
  <td>19</td>
  <td>8627289083</td>
  <td>336312187</td>
  <td>30094</td>
  <td>143759724</td>
  <td>6598</td>
  </tr>
  <tr>
  <td>20</td>
  <td>8644741581</td>
  <td>353730396</td>
  <td>49458</td>
  <td>152217853</td>
  <td>9275</td>
  </tr>
  <tr>
  <td>21</td>
  <td>8685908403</td>
  <td>369886284</td>
  <td>1175195</td>
  <td>161313923</td>
  <td>7958903</td>
  </tr>
  <tr>
  <td>22</td>
  <td>8694494654</td>
  <td>387336207</td>
  <td>354008</td>
  <td>169541244</td>
  <td>2553802</td>
  </tr>
  <tr>
  <td>23</td>
  <td>8702920906</td>
  <td>403389097</td>
  <td>29315</td>
  <td>176524452</td>
  <td>12932</td>
  </tr>
  <tr>
  <td>24</td>
  <td>8711458401</td>
  <td>420211718</td>
  <td>31924</td>
  <td>184984842</td>
  <td>11574</td>
  </tr>
  <tr>
  <td>25</td>
  <td>8729941722</td>
  <td>437299615</td>
  <td>32472</td>
  <td>194553843</td>
  <td>12002</td>
  </tr>
  <tr>
  <td>26</td>
  <td>8743658904</td>
  <td>453739403</td>
  <td>28809</td>
  <td>202074676</td>
  <td>13279</td>
  </tr>
  <tr>
  <td>27</td>
  <td>8763317458</td>
  <td>470902005</td>
  <td>32298</td>
  <td>211321630</td>
  <td>15377</td>
  </tr>
  <tr>
  <td>28</td>
  <td>8788189716</td>
  <td>487432842</td>
  <td>37105</td>
  <td>218972477</td>
  <td>27666</td>
  </tr>
  <tr>
  <td>29</td>
  <td>8796580152</td>
  <td>504414945</td>
  <td>36756</td>
  <td>228334744</td>
  <td>79954</td>
  </tr>
  <tr>
  <td>30</td>
  <td>8821174857</td>
  <td>520930989</td>
  <td>39550</td>
  <td>235849655</td>
  <td>140461</td>
  </tr>
  <tr>
  <td>31</td>
  <td>8818857058</td>
  <td>537611096</td>
  <td>34142</td>
  <td>648080</td>
  <td>79191</td>
  </tr>
  <tr>
  <td>32</td>
  <td>8855038758</td>
  <td>555138781</td>
  <td>37680</td>
  <td>18414880</td>
  <td>70489</td>
  </tr>
  <tr>
  <td>33</td>
  <td>8870680446</td>
  <td>571194669</td>
  <td>37541</td>
  <td>34596108</td>
  <td>131455</td>
  </tr>
  <tr>
  <td>34</td>
  <td>8888946679</td>
  <td>588222521</td>
  <td>33724</td>
  <td>52553756</td>
  <td>80009</td>
  </tr>
  <tr>
  <td>35</td>
  <td>9256640352</td>
  <td>604791887</td>
  <td>16658672</td>
  <td>132185723</td>
  <td>41881719</td>
  </tr>
  <tr>
  <td>36</td>
  <td>9189040776</td>
  <td>621918353</td>
  <td>12296238</td>
  <td>257921026</td>
  <td>235389707</td>
  </tr>
  <tr>
  <td>37</td>
  <td>8962737456</td>
  <td>638241888</td>
  <td>1086663</td>
  <td>109613368</td>
  <td>35222987</td>
  </tr>
  <tr>
  <td>38</td>
  <td>9005853511</td>
  <td>655453884</td>
  <td>2059624</td>
  <td>131945369</td>
  <td>73389550</td>
  </tr>
  <tr>
  <td>39</td>
  <td>9005576553</td>
  <td>671845678</td>
  <td>1434478</td>
  <td>143002441</td>
  <td>51959363</td>
  </tr>
  <tr>
  <td>40</td>
  <td>9284680907</td>
  <td>688991063</td>
  <td>12776341</td>
  <td>349762585</td>
  <td>347998221</td>
  </tr>
  <tr>
  <td>41</td>
  <td>9049931865</td>
  <td>705399210</td>
  <td>1778532</td>
  <td>174597773</td>
  <td>72566933</td>
  </tr>
  <tr>
  <td>42</td>
  <td>9314836359</td>
  <td>722226758</td>
  <td>12743442</td>
  <td>365270833</td>
  <td>380415682</td>
  </tr>
  <tr>
  <td>43</td>
  <td>9072200927</td>
  <td>739449289</td>
  <td>1344663</td>
  <td>205181163</td>
  <td>61284843</td>
  </tr>
  <tr>
  <td>44</td>
  <td>9346737669</td>
  <td>755766179</td>
  <td>12681859</td>
  <td>383580355</td>
  <td>409359111</td>
  </tr>
  <tr>
  <td>45</td>
  <td>9117099955</td>
  <td>773167996</td>
  <td>1801713</td>
  <td>235583664</td>
  <td>88985013</td>
  </tr>
  <tr>
  <td>46</td>
  <td>9108062783</td>
  <td>789247474</td>
  <td>860680</td>
  <td>250992592</td>
  <td>43508069</td>
  </tr>
  <tr>
  <td>47</td>
  <td>9129892784</td>
  <td>806871038</td>
  <td>984804</td>
  <td>268229102</td>
  <td>51249366</td>
  </tr>
  <tr>
  <td>48</td>
  <td>9146468279</td>
  <td>822765997</td>
  <td>1018387</td>
  <td>282312588</td>
  <td>58278399</td>
  </tr>
  <tr>
  <td>49</td>
  <td>9476835578</td>
  <td>840085058</td>
  <td>13985421</td>
  <td>241172394</td>
  <td>809315446</td>
  </tr>
  <tr>
  <td>50</td>
  <td>9495578885</td>
  <td>856579327</td>
  <td>14155046</td>
  <td>241909464</td>
  <td>847629148</td>
  </tr>
  <tr>
  <td>51</td>
  <td>9537115189</td>
  <td>873483093</td>
  <td>15057500</td>
  <td>238735335</td>
  <td>932663942</td>
  </tr>
  <tr>
  <td>52</td>
  <td>9556102594</td>
  <td>890026435</td>
  <td>15322279</td>
  <td>238194482</td>
  <td>982429654</td>
  </tr>
  <tr>
  <td>53</td>
  <td>9589094741</td>
  <td>907142375</td>
  <td>15899251</td>
  <td>234845868</td>
  <td>1052080437</td>
  </tr>
  <tr>
  <td>54</td>
  <td>9609053333</td>
  <td>923477989</td>
  <td>16049518</td>
  <td>233890599</td>
  <td>1092323040</td>
  </tr>
  <tr>
  <td>55</td>
  <td>9628950166</td>
  <td>940997348</td>
  <td>16172619</td>
  <td>235383688</td>
  <td>1131146866</td>
  </tr>
  <tr>
  <td>56</td>
  <td>9650657175</td>
  <td>957049360</td>
  <td>16445697</td>
  <td>231276680</td>
  <td>1183699383</td>
  </tr>
  <tr>
  <td>57</td>
  <td>9666446210</td>
  <td>973785857</td>
  <td>16330748</td>
  <td>233203869</td>
  <td>1205098118</td>
  </tr>
  <tr>
  <td>58</td>
  <td>9687274222</td>
  <td>990692518</td>
  <td>16523542</td>
  <td>230842647</td>
  <td>1254624242</td>
  </tr>
  <tr>
  <td>59</td>
  <td>9706652879</td>
  <td>1007946602</td>
  <td>16576268</td>
  <td>231502185</td>
  <td>1288374980</td>
  </tr>
  <tr>
  <td>60</td>
  <td>9720091630</td>
  <td>1024044005</td>
  <td>16547047</td>
  <td>230966608</td>
  <td>1321807705</td>
  </tr>
  <tr>
  <td>61</td>
  <td>9741079017</td>
  <td>1041285110</td>
  <td>16635400</td>
  <td>230873663</td>
  <td>1362929599</td>
  </tr>
  <tr>
  <td>62</td>
  <td>9761596587</td>
  <td>1057847755</td>
  <td>16683756</td>
  <td>230289842</td>
  <td>1399235989</td>
  </tr>
  <tr>
  <td>63</td>
  <td>9782104875</td>
  <td>1075055403</td>
  <td>16299138</td>
  <td>237386812</td>
  <td>1397167324</td>
  </tr>
  <tr>
  <td>64</td>
  <td>9790122724</td>
  <td>1091147494</td>
  <td>16650471</td>
  <td>229928585</td>
  <td>1463076072</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>Edit:
  2 things worth noting:</p>
  <ol>
  <li><p>If I add padding to the inner loop so it won't fit in the uop cache I don't see this behavior until ~150 iterations.</p>
  </li>
  <li><p>Adding an <code>lfence</code> on with the padding in the outer loop changes N threshold to 31.</p>
  </li>
  </ol>
  <p>edit2: <s>Benchmark which clears branch history</s> The condition was reversed. It should be <code>cmove</code> not <code>cmovne</code>. With the fixed version any iteration count sees elevated <strong>Branch Misses</strong> at the same rate as above (1.67 * 10^9). This means the <strong>LSD</strong> is not itself causing <strong>Branch Misses</strong>, but leaves open the possibility that <strong>LSD</strong> is in some way defeating the <strong>Branch Predictor</strong> (what I believe to be the case).</p>
  <pre><code>static void BENCH_ATTR
  bench(uint64_t inner_N) {
      uint64_t inner_loop_cnt, outer_loop_cnt;
      asm volatile(
          &quot;.p2align 12\n&quot;
          &quot;movl   %k[outer_N], %k[outer_loop_cnt]\n&quot;
          &quot;.p2align   6\n&quot;
          &quot;1:\n&quot;
          &quot;testl  $3, %k[outer_loop_cnt]\n&quot;
          &quot;movl   $1000, %k[inner_loop_cnt]\n&quot;
          THIS NEEDS TO BE CMOVE
          &quot;cmovne   %k[inner_N], %k[inner_loop_cnt]\n&quot;
          // Extra align surrounding inner loop so that the entire thing
          // doesn't execute out of LSD.
          &quot;.p2align   10\n&quot;
          &quot;2:\n&quot;
          &quot;decl   %k[inner_loop_cnt]\n&quot;
          &quot;jnz    2b\n&quot;
          &quot;.p2align   10\n&quot;
          &quot;decl   %k[outer_loop_cnt]\n&quot;
          &quot;jnz    1b\n&quot;
          : [ inner_loop_cnt ] &quot;=&amp;r&quot;(inner_loop_cnt),
            [ outer_loop_cnt ] &quot;=&amp;r&quot;(outer_loop_cnt)
          : [ inner_N ] &quot;ri&quot;(inner_N), [ outer_N ] &quot;i&quot;(outer_N)
          :);
  }
  </code></pre>