title: How to design a database schema for a search engine?
body: <p>I'm writing a small search engine in C with curl, libxml2, and mysql. The basic plan is to grab pages with curl, parse them with libxml2, then iterate over the DOM and find all the links. Then traverse each of those, and repeat, all while updating a SQL database that maintains the relationship between URLs.</p>
  
  <p>My question is: how can I best represent the relationship between URLs?.</p>