title: How can I reduce the compile-time memory footprint of large templates?
body: <p>Suppose I have a class which contains a <em>large</em> number of other class declarations. Is it possible to spread the cost of these out somehow so that compile-time memory consumption doesn't grow quadratically for nested types? I'm willing to take a hit on compile time if need be, and would gladly divide this out into different translation units if that were an option.</p>
  
  <p>To try and come up with a solution to this I've written the following program that illustrates a simplified version of the kind of code that leads to these blowouts:</p>
  
  <pre><code>// Add T to the list of args of SeqWithArgs N number of times:
  template &lt;int N, typename T, typename SeqWithArgs&gt;
  struct Append;
  
  template &lt;int N, typename T, template &lt;typename...&gt; class Seq, typename... Args&gt;   
  struct Append&lt;N, T, Seq&lt;Args...&gt;&gt;                                                  
  {                                                                                  
      using type = typename Append&lt;N-1, T, Seq&lt;Args..., T&gt;&gt;::type;                   
  };                                                                                 
  
  template &lt;typename T, template&lt;typename...&gt; class Seq, typename... Args&gt;           
  struct Append&lt;0, T, Seq&lt;Args...&gt;&gt;                                                  
  {                                                                                  
      using type = Seq&lt;Args...&gt;;                                                     
  };                                                                                 
  
  static constexpr const int N = 10;                                              
  
  // Tuple containing N instances of int
  using Small = typename Append&lt;N, int, std::tuple&lt;&gt;&gt;::type;
  
  // Tuple containing N instances of Small
  using Big = typename Append&lt;N, Small, std::tuple&lt;&gt;&gt;::type;
  
  // Tuple containing N instances of Big
  using Huge = typename Append&lt;N, Big, std::tuple&lt;&gt;&gt;::type;
  
  int main()
  {                                                                               
      Huge h;                                                                     
      return 0;                                                                   
  }
  </code></pre>
  
  <p><strong>As righly pointed out by Yakk, these <code>Append</code> operations are horribly inefficient. However, in the real version of this code modifying these would require a fundamental restructure of the code.</strong></p>
  
  <p>I varied <code>N</code> from 10 to 70 and got this result compiling my program using <code>GCC 4.8.1</code>. I also ran <code>time -v make</code> to obtain peak resident memory usage. Here are the results using just the default flags:</p>
  
  <p><img src="https://i.stack.imgur.com/3jNqf.png" alt="N^3 nested tuple compile-time memory usage"></p>
  
  <p>This result seems excessive to me, not because of the shape (it is expected to be O(N^3) and seems to follow that shape), but because of the magnitude. It almost looks like <code>Small</code> is being expanded for <em>every</em> instantiation of <em>Big</em>, and <em>Big</em> in turn is being expanded for every instantiation of <em>Huge</em>. In less template-heavy code one would normally declare an generic specialisation of some type using the <code>extern</code> keyword and would therefore avoid this "nested expansion", but these are <em>types</em>, not <em>values</em>; does something similar exist for this kind of construct?</p>
  
  <p>What is the cause of this blowout in memory and what can I do to reduce this memory footprint <em>without</em> changing the type of <code>Small</code>, <code>Big</code> and <code>Huge</code>?</p>