title: How should I update multiple RavenDB documents from a daily feed?
body: <p>I'm using RavenDB to hold several thousand documents. The data comes from a daily xml feed which I'll update by running a C# console app. Below is the code that processes the feed to keep the database in sync with any changes. I've had quite a few problems with this so I'm wondering if I've picked the wrong strategy. Here are some important things to note.</p>
  
  <ol>
  <li>New items may have been added to the feed and existing items may
  have changed, so each time it runs I want to either add or update a
  document depending on whether or not it's new.</li>
  <li>The xml feed doesn't contain any reference to my RavenDB IDs, only its internal key for each item. So when retrieving an existing
  document to update I can only do that by examining the "SourceID"
  property on the document.</li>
  <li>I'm using "take" to only work with 500 docs at a time partly because my db is limited to 1000 docs, and partly because without
  Take() I seem to be only able to retrieve 128 docs.</li>
  <li>As it stands, this code falls over with a "can't do more than 30 updates in a session" error, I think because each time I try to
  retrieve an existing record from dbItems it actually hits the
  database again.</li>
  <li>I can fix the issue at (4) above by calling ToList() on items, but if I do that the existing item doesn't get updated when I call
  session.SaveChanges() (I'm imagining this like a disconnected
  recordset).</li>
  </ol>
  
  <p>Can anyone give me some pointers?</p>
  
  <pre><code>        public void ProcessFeed(string rawXml)
          {
              XDocument doc = XDocument.Parse(rawXml);
              var items = ExtractItemsFromFeed(doc).OrderBy(x =&gt; x.SourceId).Take(500);
              using (IDocumentSession session = _store.OpenSession())
              {
                  var dbItems = session.Query&lt;AccItem&gt;().OrderBy(x =&gt; x.SourceId).Take(500);
                  foreach (var item in items)
                  {
                      var existingRecord = dbItems.SingleOrDefault(x =&gt; x.SourceId == item.SourceId);
                      if (existingRecord == null)
                      {
                          session.Store(item);
                          _logger.Info("Saved new item {0}.", item.ShortName);
                      }
                      else
                      {
                          // update just one field for now
                          existingRecord.Village = item.Village;
                          _logger.Info("Updated item {0}.", item.ShortName);
                      }
                  }
                  session.SaveChanges();
              }            
          }
  </code></pre>