title: Computing text similarity against many documents
body: <p>I'm trying to compute the text similarity of a search term, <strong>A</strong>, like "How to make chickens" against a collection of other search terms. To compute similarity I'm using the cosine distance and TF-IDF to transform <strong>A</strong> into a vector. I'd like to compare the similarity of <strong>A</strong> against <em>all</em> documents at once.</p>
  
  <p>Currently, my approach involves computing the cosine similarity for <strong>A</strong> against every other document one at a time, iteratively. I have 100 documents I'm comparing against. If the result of <code>cos_sim(A, X) &gt; 0.8</code> then I break and say "cool, this is similar".</p>
  
  <p>However, I feel like this might not be a true representation of the overall similarity. Is there a way to pre-compute a vector(s) for my 100 documents at runtime, and every time I see a new search query <strong>A</strong>, I can compare against this pre-defined vector/document?</p>
  
  <p>I believe I can achieve this by simply combining all documents into one... feels rough though. What are the pros and &amp; cons, and possible solutions? Extra points for efficiency!</p>