title: Reading a large log file in C#
body: <p>For my project, I need to extract message types from a log file. I have a 700 MB log file which contains about 4.7 million lines and I need to read each entry line by line and extract the message field. I need to find the size of message in each entry(which is the event size) and store it along with that message in a dictionary. There can be multiple messages for same event size. But I get a OutOfMemoryException when I use the below logic. </p>
  
  <pre><code>Dictionary&lt;Int32,List&lt;String&gt;&gt; dt=new Dictionary&lt;Int32,List&lt;String&gt;&gt;();
  List&lt;String&gt; entries=new List&lt;String&gt;();
  StreamReader sr=new StreamReader("Bluegene.log");
  String s;
  while((s=sr.readLine())!=null)
  {
      eventsize=s.length - 9; //size of only the message field
      entries.Add(s);
      if (!dt.ContainsKey(eventsize))
      {
          dt.Add(eventsize, entries);
      }
      else
      {
         dt.Remove(eventsize);
         dt.Add(eventsize, entries);
       }
    }
  </code></pre>
  
  <p>Will using MemoryMappedFile help?</p>