title: Optimize C# file IO
body: <p>Scenario - 150MB text file which is the exported Inbox of an old email account.  Need to parse through and pull out emails from a specific user and writes these to a new, single file.  I have code that works, its just dogged slow.</p>
  
  <p>I'm using marker strings to search for where to begin/end the copy from the original file.</p>
  
  <p>Here's the main function:</p>
  
  <pre><code> StreamReader sr = new StreamReader("c:\\Thunderbird_Inbox.txt");
          string working = string.Empty;
          string mystring = string.Empty;
          while (!sr.EndOfStream)
          {
              while ((mystring = sr.ReadLine()) != null)
              {
                  if (mystring == strBeginMarker)
                  {
                      writeLog(mystring);
  
                      //read the next line
                      working = sr.ReadLine();
  
                          while( !(working.StartsWith(strEndMarker)))
                          {
                              writeLog(working);
                              working = sr.ReadLine();
  
                          }
                    }
              }
  
          }
          this.Text = "DONE!!";
          sr.Close();
  </code></pre>
  
  <p>The function that writes the selected messages to the new file:</p>
  
  <pre><code>  public void writeLog(string sMessage)
      {
              fw = new System.IO.StreamWriter(path, true);
              fw.WriteLine(sMessage);
              fw.Flush();
              fw.Close();
      }
  </code></pre>
  
  <p>Again, this process works.  I get a good output file, it just takes a long time and I'm sure there are ways to make this faster.</p>