title: Compiling C# scripts is slow
body: <p>I have an application that uses "scripts" written in C#, compiled on start-up using the C# CodeDomProvider. Loading a few dozen, even >170 scripts isn't much of a problem, I compile them and then save the assemblies into a cache folder for the next startup. For 170 scripts this initial compilation takes about 1.5 minutes.</p>
  
  <p>However, when I tried to load >1000 scripts it took well over an hour to compile them all. I added a Stopwatch and learned that every script takes a little bit longer to load than the one before, in the case of 170 files it went from ~150ms for the first one to >650ms for the last one, increasing bit by bit for every file.</p>
  
  <p>I know I could reduce my load time drastically in general by combining the scripts to one big file, but for several reasons I'd prefer it a lot if I could compile them separately :/ It makes reloading them easy and fast, I don't have to worry about recompiling the whole scripts folder when something changes, I can easily give a progress bar for the compilation progress, etc.</p>
  
  <p>Now my question is, what's the problem here? Why does the compile time per file increase over time? And can I do something about that?</p>
  
  <p><strong>Edit</strong></p>
  
  <p>I'll <em>try</em> to give more information, as request in the comments.</p>
  
  <p>As I've said, I'm compiling every script using the C# CodeDomProvider, basically the following in a loop, for every of my currently 171 script files, with each file containing one or multiple classes that I instantiate after the assembly was created:</p>
  
  <pre><code>var provider = System.CodeDom.Compiler.CodeDomProvider.CreateProvider("CSharp"); ;
  var parameters = new System.CodeDom.Compiler.CompilerParameters();
  parameters.ReferencedAssemblies.AddRange(AppDomain.CurrentDomain.GetAssemblies().Where(a =&gt; !a.IsDynamic).Select(a =&gt; a.Location).ToArray());
  parameters.GenerateExecutable = false;
  parameters.GenerateInMemory = false;
  parameters.OutputAssembly = tmpFileName + ".compiled";
  parameters.TreatWarningsAsErrors = false;
  parameters.WarningLevel = 0;
  var results = provider.CompileAssemblyFromFile(parameters, tmpFileName);
  asm = results.CompiledAssembly;
  </code></pre>
  
  <p>I get the same results using CS-Script, which does something similar.</p>
  
  <pre><code>asm = CSScript.LoadWithConfig(tmpFileName, null, debug, CSScript.GlobalSettings, "/warnaserror- /warn:0");
  </code></pre>
  
  <p>The created assembly is then saved into a cache folder. That saved assembly is used when the script file is older than the assembly, so I don't have to compile it again.</p>
  
  <p>That it gets slower and slower over time becomes obvious when you sit in front of it, watching the progress bar advance slower and slower, so I added a simple Stopwatch call around the above code, <code>StartNew, Stop, Elapsed</code>, nothing fancy. The result reflects what I saw on the progress bar, with each additional file the compile time increased. </p>
  
  <pre><code>timer.Restart();
  
  // compile as shown above
  
  timer.Stop();
  Console.WriteLine(asm.Location + ": " + timer.Elapsed);
  </code></pre>
  
  <p>A slight difference between files is to be expected of course, depending on how big they are, and how complex, but the scripts are generally all more or less the same in size and complexity, and I'm witnessing a consistent increase over time.</p>
  
  <pre><code>Temp\tmpDDD7.tmp.compiled: 00:00:00.1389177
  Temp\tmpDE74.tmp.compiled: 00:00:00.1327150
  ...
  Temp\tmpE156.tmp.compiled: 00:00:00.1719746
  Temp\tmpE213.tmp.compiled: 00:00:00.1431011
  ...
  Temp\tmpF05C.tmp.compiled: 00:00:00.1696297
  Temp\tmpF118.tmp.compiled: 00:00:00.1739564
  ...
  Temp\tmpF7D5.tmp.compiled: 00:00:00.1824292
  Temp\tmpF891.tmp.compiled: 00:00:00.1819889
  ...
  Temp\tmp29F1.tmp.compiled: 00:00:00.2912163
  Temp\tmp2B2B.tmp.compiled: 00:00:00.2909096
  ...
  Temp\tmp362F.tmp.compiled: 00:00:00.3161408
  Temp\tmp3773.tmp.compiled: 00:00:00.3170768
  ...
  Temp\tmpA4C9.tmp.compiled: 00:00:00.5457990
  Temp\tmpA6FC.tmp.compiled: 00:00:00.5460514
  </code></pre>