title: Speed up looping through a dictionary in VB.Net 2008
body: <p>I have a process that imports a daily file of product registrations, and adds them into our database.  Originally this process would query against the database multiple times for each record to determine how to process the data.</p>
  
  <p>In order to speed up this process and prevent any timeout issues that could appear to people trying to use a reporting site that uses the same database, I've changed the code to pull down a few of the tables into dictionaries, and then iterate across them to see if that customer/address/dealership exists, and pull the Id from the key of the dictionary if it does, or insert it into the table and then the dictionary if it doesn't.</p>
  
  <p>However, I'm currently finding this to be running <em>slower</em> than if I were querying the database multiple times for each registration.  One possible cause that I can think of is that my dictionaries are quite large (one has 8 million entries and another has 11 million).</p>
  
  <p>Here is one example of what I'm doing:</p>
  
  <pre><code>    For Each kvp As KeyValuePair(Of Int64, String) In dCust
              If kvp.Value = firstName &amp; "|" &amp; lastName &amp; "|" &amp; companyName &amp; "|" &amp; addrId &amp; "|" &amp; typeID &amp; "|" &amp; phone &amp; "|" &amp; email Then
                  custId = kvp.Key
                  Exit For
              End If
      Next
  </code></pre>
  
  <p>This dictionary has around 11 million records in it.</p>
  
  <p>An idea a coworker of mine had was to run a <code>Dictionary.ContainsValue()</code> before the loop to see if it's even there. And if it isn't skip the loop entirely.  I'd only want to try this if that runs faster than just doing the loop itself, if they take the same time I don't see a point in basically running the loop twice.</p>
  
  <p>So my questions to you are:</p>
  
  <ul>
  <li>Am I going about this in the most
  effiecient way?</li>
  <li>Would it be faster to
  run a <code>Dictionary.ContainsValue()</code> before
  attempting the loop, or will the
  system interpret them as the same
  thing, thus doubling my time?</li>
  <li>Is there anything else that I should be looking for?</li>
  </ul>