title: Optimizing C# Custom Xml Parser Class
body: <p>I am trying to write an XML parser to parse all courses from that a university offers given a calendar year and semester.  In particular, I am trying to get the department acronym (i.e. FIN for Finance, etc), the Course Number (i.e. Math 415, 415 would be the number), the Course Name, and the number of credit hours the course is worth.</p>
  
  <p>The file I am trying to parse can be found <a href="http://courses.illinois.edu/cisapp/explorer/schedule.xml" rel="nofollow">HERE</a></p>
  
  <p><strong>EDIT AND UPDATE</strong></p>
  
  <p>Upon reader deeper into XML parsing, and the best way to optimize it, I stumbled upon this blog <a href="http://codingrules.blogspot.com/2010/06/c-xmlreader.html" rel="nofollow">POST</a></p>
  
  <p>Assuming the results of the tests run in that article are both honest and accurate, it seems that XmlReader far outperforms both XDocument and XmlDocument, which verifies what is said in the great answers below.  Having said that, I re-coded my parser class using XmlReader along with limiting the number of readers used in a single method.</p>
  
  <p>Here is the new parser class:</p>
  
  <pre><code>        public void ParseDepartments()
          {
              // Create reader for the given calendar year and semester xml file
              using (XmlReader reader = XmlReader.Create(xmlPath)) {
                  reader.ReadToFollowing("subjects");  // Navigate to the element 'subjects'
                  while (!reader.EOF) {
                      string pth = reader.GetAttribute("href");  // Get department's xml path
                      string acro = reader.GetAttribute("id");  // Get the department's acronym
                      reader.Read();  // Read through current element, ensures we visit each element
                      if (acro != null &amp;&amp; acro != string.Empty) {  // If the acronym is valid, add it to the department list
                          deps.AddDepartment(acro, pth);
                      }
                  }
              }
          }
  
          public void ParseDepCourses()
          {
              //  Loop through all the departments, and visit there respective xml file
              foreach (KeyValuePair&lt;string, string&gt; department in deps.DepartmentPaths) {
                  try {
                      using (XmlReader reader = XmlReader.Create(department.Value)) {
                          reader.ReadToFollowing("courses");  // Navigate to the element 'courses'
                          while (!reader.EOF) {
                              string pth = reader.GetAttribute("href");
                              string num = reader.GetAttribute("id");
                              reader.Read();
                              if (num != null &amp;&amp; num != string.Empty) {
                                  string crseName = reader.Value;  //  reader.Value is the element's value, i.e. &lt;elementTag&gt;Value&lt;/elementTag&gt; 
                                  deps[department.Key].Add(new CourseObject(num, crseName, termID, pth)); // Add the course to the department's course list
                              }
                          }
                      }
                  } catch (WebException) { }  // WebException is thrown (Error 404) when there is no xml file found, or in other words, the department has no courses
              }
  
          }
  
          public void ParseCourseInformation()
          {
              Regex expr = new Regex(@"^\S(L*)\d\b|^\S(L*)\b|^\S\d\b|^\S\b", RegexOptions.IgnoreCase | RegexOptions.Singleline | RegexOptions.IgnorePatternWhitespace);  // A regular expression that will check each section and determine if it is a 'Lecture' section, at which point, that section's xml file is visited, and instructor added
              foreach (KeyValuePair&lt;string, Collection&lt;CourseObject&gt;&gt; pair in deps) {
                  foreach (CourseObject crse in pair.Value) {
                      try {
                          using (XmlReader reader = XmlReader.Create(crse.XmlPath)) {
                              reader.ReadToFollowing("creditHours");  // Get credit hours for the course
                              crse.ParseCreditHours(reader.Value);  //  Class method to parse the string and grab the correct integer values
                              reader.ReadToFollowing("sections"); //  Navigate to the element 'sections'
                              while (!reader.EOF) {
                                  string pth = reader.GetAttribute("href");
                                  string crn = reader.GetAttribute("id");
                                  reader.Read();
                                  if (crn != null &amp;&amp; crn != string.Empty) {
                                      string sction = reader.Value;
                                      if (expr.IsMatch(sction)) {  // Check if sction is a 'Lecture' section
                                          using (XmlReader reader2 = XmlReader.Create(pth)) {  //  Navigate to its xml file
                                              reader2.ReadToFollowing("instructors");  // Navigate to the element 'instructors'
                                              while (!reader2.EOF) {
                                                  string firstName = reader2.GetAttribute("firstName");
                                                  string lastName = reader2.GetAttribute("lastName");
                                                  reader2.Read();
                                                  if ((firstName != null &amp;&amp; firstName != string.Empty) &amp;&amp; (lastName != null &amp;&amp; lastName != string.Empty)) { //  Check and make sure its a valid name
                                                      string instr = firstName + ". " + lastName;  //  Concatenate into full name
                                                      crse.AddSection(pth, sction, crn, instr);  //  Add section to course
                                                  }
                                              }
                                          }
                                      }
                                  }
                              }
                          }
                      } catch (WebException) { }  // No course/section information found
                  }
              }
          }
  </code></pre>
  
  <p>Although the execution of this code takes quite some time (anywhere between 10-30 min), it is expected given the large amount of data being parsed.  Thanks to everyone who posted answers, it was much appreciated.  I hope this helps any other people who may have similar problems/questions.</p>
  
  <p>Thanks,</p>
  
  <p>David</p>