title: Efficiently synchronize streaming input with large SQL database
body: <p>I've got an efficiency problem on my hands and I'm looking for ways to solve it. Here is the situation:</p>
  
  <ul>
  <li>I have streaming row-based data coming into a system from an online source, each with unique <code>id</code>.</li>
  <li>I have an SQL database with existing row-based data, indexed by <code>id</code>.</li>
  <li>I need to update the SQL database with new data streaming in if it differs from the existing data already in the database.</li>
  </ul>
  
  <p>The obvious solution is:</p>
  
  <ol>
  <li>Read incoming row</li>
  <li>Read corresponding row in database</li>
  <li>If data differs, update the database</li>
  </ol>
  
  <p>The large number of round-trips is making this algorithm infeasibly slow.</p>
  
  <p>The alternative solution I have is to read the entire SQL database into memory, and compare with new incoming data. This eliminates the round-trips but the memory required makes this also infeasible.</p>
  
  <p>So, what alternatives do I have?</p>