title: large performance drop with gcc, maybe related to inline
body: <p>I'm currently experiencing some weird effect with <code>gcc</code> (tested version: 4.8.4).</p>
  
  <p>I've got a performance oriented code, which runs pretty fast. Its speed depends for a large part on inlining many small functions.</p>
  
  <p>Since inlining across multiple <code>.c</code> files is difficult (<code>-flto</code> is not yet widely available), I've kept a lot of small functions (typically 1 to 5 lines of code each) into a common C file, into which I'm developing a codec, and its associated decoder. It's "relatively" large by my standard (about ~2000 lines, although a lot of them are just comments and blank lines), but breaking it into smaller parts opens new problems, so I would prefer to avoid that, if that is possible.</p>
  
  <p>Encoder and Decoder are related, since they are inverse operations. But from a programming perspective, they are completely separated, sharing nothing in common, except a few typedef and very low-level functions (such as reading from unaligned memory position).</p>
  
  <p>The strange effect is this one:</p>
  
  <p>I recently added a new function <code>fnew</code> to the encoder side. It's a new "entry point". It's not used nor called from anywhere within the <code>.c</code> file.</p>
  
  <p>The simple fact that it exists makes the performance of the <em>decoder</em> function <code>fdec</code> drops substantially, by more than 20%, which is way too much to be ignored.</p>
  
  <p>Now, keep in mind than encoding and decoding operations are completely separated, and share almost nothing, save some minor <code>typedef</code> (<code>u32</code>, <code>u16</code> and such) and associated operations (read/write).</p>
  
  <p>When defining the new encoding function <code>fnew</code> as <code>static</code>, performance of the decoder <code>fdec</code> increases back to normal. Since <code>fnew</code> isn't called from the <code>.c</code>, I guess it's the same as if it was not there (dead code elimination).</p>
  
  <p>If <code>static fnew</code> is now called from the encoder side, performance of <code>fdec</code> remains strong.</p>
  
  <p>But as soon as <code>fnew</code> is modified, <code>fdec</code> performance just drops substantially.</p>
  
  <p>Presuming <code>fnew</code> modifications crossed a threshold, I increased the following <code>gcc</code> parameter: <code>--param max-inline-insns-auto=60</code> (by default, its value is supposed to be 40.) And it worked : performance of <code>fdec</code> is now back to normal.</p>
  
  <p>And I guess this game will continue forever with each little modification of <code>fnew</code> or anything else similar, requiring further tweak.</p>
  
  <p>This is just plain weird. There is no logical reason for some little modification in function <code>fnew</code> to have knock-on effect on completely unrelated function <code>fdec</code>, which only relation is to be in the same file.</p>
  
  <p>The only tentative explanation I could invent so far is that maybe the simple presence of <code>fnew</code> is enough to cross some kind of <code>global file threshold</code> which would impact <code>fdec</code>. <code>fnew</code> can be made "not present" when it's: 1. not there, 2. <code>static</code> but not called from anywhere 3. <code>static</code> and small enough to be inlined. But it's just hiding the problem. Does that mean I can't add any new function?</p>
  
  <p>Really, I couldn't find any satisfying explanation anywhere on the net.</p>
  
  <p>I was curious to know if someone already experienced some equivalent side-effect, and found a solution to it.</p>
  
  <p><em>[Edit]</em></p>
  
  <p>Let's go for some more crazy test.
  Now I'm adding <em>another</em> completely useless function, just to play with. Its content is strictly exactly a copy-paste of <code>fnew</code>, but the name of the function is obviously different, so let's call it <code>wtf</code>.</p>
  
  <p>When <code>wtf</code> exists, it doesn't matter if <code>fnew</code> is static or not, nor what is the value of <code>max-inline-insns-auto</code>: performance of <code>fdec</code> is back to normal.
  Even though <code>wtf</code> is not used nor called from anywhere... :'(</p>
  
  <p><em>[Edit 2]</em> 
  there is no <code>inline</code> instruction. All functions are either normal or <code>static</code>. Inlining decision is solely within compiler's realm, which has worked fine so far.</p>
  
  <p><em>[Edit 3]</em>
  As suggested by Peter Cordes, the issue is not related to inline, but to instruction alignment. On newer Intel cpus (Sandy Bridge and later), hot loop benefit from being aligned on 32-bytes boundaries.
  Problem is, by default, <code>gcc</code> align them on 16-bytes boundaries. Which gives a 50% chance to be on proper alignment depending on length of previous code. Hence a difficult to understand issue, which "looks random".</p>
  
  <p>Not all loop are sensitive. It only matters for critical loops, and only if their length make them cross one more 32-bytes instruction segment when being less ideally aligned.</p>