title: Machine Learning on PostgreSQL
body: <p>I am interested in running machine learning algorithms directly inside of PostgreSQL as described <a href="https://asterixdb.ics.uci.edu/pub/ieee12.pdf" rel="noreferrer">here</a>.</p>
  
  <p>The basic gist of the paper is that I write my algorithm as a function which gives the nth model</p>
  
  <pre><code>def get_model(n):
     return make_step(model(n-1))
  
  def make_step(model):
     # Compute gradient and make update and return new model
  </code></pre>
  
  <p>The idea is that the database will optimize the data flow such that the query runs very quickly.</p>
  
  <p>My data is about 10^12 samples, 10^9 features, with an average of 1000 features that aren't zero per sample (highly! sparse).</p>
  
  <p>I justify my question because I found the following,</p>
  
  <ul>
  <li><a href="http://www.postgresql.org/docs/8.4/static/xtypes.html" rel="noreferrer">For defining sparse vectors</a></li>
  <li><a href="http://www.postgresql.org/docs/9.1/static/sql-createfunction.html" rel="noreferrer">Defining inner products</a></li>
  <li><a href="http://www.postgresql.org/docs/8.4/static/queries-with.html" rel="noreferrer">The recursive Query</a></li>
  <li>And I am pretty sure that PostgreSQL can be distributed onto many machines</li>
  </ul>
  
  <p>My questions are,</p>
  
  <ul>
  <li>How well will PostgreSQL handle sparse matrix vector products?</li>
  <li>Will PostgreSQL actually optimize for better data flow and processing?</li>
  <li>Are any of my assumptions or other goals not realistic?</li>
  </ul>