title: Implications of modeling a graph in couchdb
body: <p>I've been toying with modeling a graph structure (property graph with named relationships) in couchdb and would like to know what are the potential bottlenecks in performance that I will find.</p>
  
  <p>I'm using the following principles:</p>
  
  <ul>
  <li>Keep documents small.</li>
  <li>Try to embed as little as possible.</li>
  <li>Record all relationships between documents as a new document (a link).</li>
  </ul>
  
  <p>It seems that all these principles are in contradiction with CouchDB philosophy,</p>
  
  <p>With this principles, for example, tagging a person becomes three documents:</p>
  
  <pre><code>{ _id: '10', type: 'person', 'name': 'John Doe' }
  { _id: '20', type: 'tag', 'name': 'Important' }
  { _id: '30', type: 'link', from: 10, to: 20, name: 'tag' }
  </code></pre>
  
  <p>I have also created the following views in a <code>_design</code> document called <code>links</code>:</p>
  
  <pre><code>{
    outgoing: {
      map: function(doc) {
        if (doc.type == 'link') {
          emit([doc.from, doc.name], {_id: doc.to});
        }
      }
    },
    incoming: {
      map: function(doc) {
        if (doc.type == 'link') {
          emit([doc.to, doc.name], { _id: doc.from });
        }
      }
    }
  }
  </code></pre>
  
  <p>I can get all the links incoming or outgoing from a document with these urls:</p>
  
  <pre><code>http://host/db/_design/links/_view/incoming?startkey=["10"]&amp;endkey=["10",{}]
  http://host/db/_design/links/_view/outgoing?startkey=["10"]&amp;endkey=["10",{}]
  </code></pre>
  
  <p>I can even get all the links by name with these urls:</p>
  
  <pre><code>http://host/db/_design/links/_view/incoming?startkey=["10","tag"]&amp;endkey=["10","tag",{}]
  http://host/db/_design/links/_view/outgoing?startkey=["10","tag"]&amp;endkey=["10","tag",{}]
  </code></pre>
  
  <p>And if I include the <code>include_docs=true</code> parameter I get the documents referenced by the link; either incoming or outgoing.  So far so good. There is a graph structure and a way to query it, albeit on a node by node basis.</p>
  
  <p>Good things about this approach:</p>
  
  <ul>
  <li>It is a general way of storing all relationships. Not necessarily tags, but every relationship.</li>
  <li>You can change the tag name quickly, without changing every person tagged.</li>
  <li>You can merge persons or tags and just update the <code>link</code> documents, which should be very simple.</li>
  <li>Tagging when using replication does not change the documents being tagged or the tags themselves. Just add or delete a tiny <code>link</code> document.</li>
  <li>It would be easy to keep a history of tags for each element.</li>
  </ul>
  
  <p>Bad things, and where I need your help:</p>
  
  <ul>
  <li>Query for a list of people with their tags is not trivial.  In general, querying for a list of documents and their relationships is a very expensive operation that requires many hits.</li>
  <li>Updating the database and keeping it consistent could be a problem.  Maybe this is something that will never go away when using couch.</li>
  <li>Doing 'maintenance' on the database, like finding orphan links, could be expensive. Perhaps the database requires garbage collection?</li>
  <li>Visualizing and manipulating this graph structure is neither intuitive nor simple, and applications developed on top of it are responsible for all the graph structure management (which is a bit scary!).</li>
  </ul>
  
  <p>So back to my questions:</p>
  
  <ol>
  <li>What are the potencial bottlenecks to expect?</li>
  <li>Will this approach scale to millions of records?</li>
  <li>How to do traversing of this structure efficiently without having to do many server hits?</li>
  </ol>