title: .NET heaps filled up with string object -> OutOfMemoryException
body: <p>I'm constantly (every 30-60 min) getting a System.OutOfMemoryException in my Windows Service. The service's job is to loop though 6 directories which contains data files which the service datawashes to a common XML data format. </p>
  
  <p>These 6 folders contains 5-10.000 files each, so the total number of files is about 45.000 and new files is added duing the day. There is added about 1-2000 new files a day. The files is between 4KB and 500KB.</p>
  
  <p>Each data file is washed to the common XML data format through the XElement object.</p>
  
  <p>I have used RedGates ANTS Memory Profiler on the service and the objects which are using the most memory is string (about 90.000.000 bytes) and XElement (about 51.000.000 bytes). </p>
  
  <p>In the Memory Profiler, when i trace, what is using the string object, i can see that it's mostly (93%) the XElement object which is using the string object. </p>
  
  <p>The server have 6 cpu's and 6GB of RAM, so i can't see why i'm getting the OutOfMemoryException. If i look at the Windows Service in the Processes it's MAX use of RAM have been 1.2GB.</p>
  
  <p>I have read that .NET garbage collector doesn't clear the string object because the string object is stored in a intern table. Could this be the error, if so what can i do about it?</p>
  
  <p>The code below shows how i'm looping through the files. As you can see i have also tried to take 20 files at a time. This just pushes the OutOfMemoryException a few hours, so the service will run for 4-5 hours instead of 30-60 min.</p>
  
  <p>Why do i can the OutOfMemoryException?</p>
  
  <pre><code>private static void CheckExistingImportFiles(object sender, System.Timers.ElapsedEventArgs e)
      {
          CheckTimer.Stop();
          var dir = Directory.GetFiles(RawDataDirectory.FullName, "*.*", SearchOption.AllDirectories);
  
          List&lt;ManualResetEvent&gt; doneEvents = new List&lt;ManualResetEvent&gt;();
          int i = 0;
          //int doNumberOfFiles = 20;
  
          foreach (string existingFile in Directory.GetFiles(RawDataDirectory.FullName, "*.*", SearchOption.AllDirectories))
          {
              if (existingFile.EndsWith("ignored") || existingFile.EndsWith("error") || existingFile.EndsWith("importing"))
              {
                  //if (DateTime.UtcNow.Subtract(File.GetCreationTimeUtc(existingFile)).TotalDays &gt; 5)
                  //  File.Delete(existingFile);
                  //continue;
              }
  
              StringBuilder fullFileName = new StringBuilder().Append(existingFile);
  
              if (!fullFileName.ToString().ToLower().EndsWith("error") &amp;&amp; !fullFileName.ToString().ToLower().EndsWith("ignored") &amp;&amp; !fullFileName.ToString().ToLower().EndsWith("importing"))
              {
                  File.Move(fullFileName.ToString(), fullFileName + ".importing");
                  fullFileName = fullFileName.Append(".importing");
  
                  ImportFileJob newJob = new ImportFileJob(fullFileName.ToString());
  
                  doneEvents.Add(new ManualResetEvent(false));
  
                  ThreadPool.QueueUserWorkItem(newJob.Run, doneEvents.ElementAt(i));
                  i++;
              }
  
              //if (i &gt; doNumberOfFiles)
              //{
              //    i = 0;
              //    doNumberOfFiles = 20;
              //    break;
              //}
          }
          i = 0;
          WaitHandle.WaitAll(doneEvents.ToArray());
  
          CheckTimer.Start();
      }
  </code></pre>