title: How to speed up computation of cosine similarity between set of vectors
body: <p>I have a set of vectors (~30k), each of which consists of 300 elements generated by fasttext, each vector is representing the meaning of an entity, I want to calculate the similarity between all entities, so I iterate over the vectors in a nested matter, O(N^2) complexity, which is not practical in terms of time.</p>
  
  <p>Can you recommend me another approach for calculating this, or how can I parallelize it?</p>
  
  <pre><code>def calculate_similarity(v1, v2):
      """
      Calculate cosine distance between two vectors
      """
      n1 = np.linalg.norm(v1)
      n2 = np.linalg.norm(v2)
      return np.dot(v1, v2) / n1 / n2
  
  
  similarities = {}
  for ith_entity, ith_vector in vectors.items():
      for jth_entity, jth_vector in vectors.items():
          if ith_entity == jth_entity:
              continue
          if (ith_entity, jth_entity) in similarities.keys() or (jth_entity, ith_entity) in similarities.keys():
              continue
          similarities[(ith_entity, jth_entity)] = calculate_similarity(ith_vector, jth_vector)
  </code></pre>