title: Optimizing Gensim word mover's distance function for speed (wmdistance)
body: <p>I am using <code>gensim</code> <code>wmdistance</code> for calculating similarity between a reference sentence and 1000 other sentences. </p>
  
  <pre><code>    model = gensim.models.KeyedVectors.load_word2vec_format(
       'GoogleNews-vectors-negative300.bin', binary=True)
      model.init_sims(replace=True)  
  
      reference_sentence = "it is a reference sentence"
      other_sentences = [1000 sentences]
      index = 0
      for sentence in other_sentences: 
        distance [index] = model.wmdistance(refrence_sentence, other_sentences)
        index = index + 1
  </code></pre>
  
  <p>According to <code>gensim</code> <a href="https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/models/keyedvectors.py" rel="nofollow noreferrer">source code</a>, <code>model.wmdistance</code> returns the following:</p>
  
  <pre><code>emd(d1, d2, distance_matrix)
  </code></pre>
  
  <p>where </p>
  
  <pre><code>d1 =  # Compute nBOW representation of reference_setence.
  d2 =  # Compute nBOW representation of other_sentence (one by one).
  distance_matrix = see the source code as its a bit too much to paste it here.
  </code></pre>
  
  <p>This code is inefficient in two ways for my use case. </p>
  
  <p>1) For the reference sentence, it is repeatedly calculating d1 (1000 times) for the distance function <code>emd(d1, d2, distance_matrix)</code>.</p>
  
  <p>2) This distance function is called by multiple users from different points which repeat this whole process of <code>model.wmdistance(doc1, doc2)</code> for the same other_sentences and it is computationally expensive. For this 1000 comparisons, it takes around 7-8 seconds. </p>
  
  <p>Therefore, I would like to isolate the two tasks. The final calculation of distance: <code>emd(d1, d2, distance_matrix)</code> and the preparation of these inputs: d1, d2, and distance matrix. As distance matrix depends on both so at least its input preparation should be isolated from the final matrix calculation. </p>
  
  <p>My initial plan is to create three customized functions: </p>
  
  <pre><code>d1 = prepared1(reference_sentence)
  d2 = prepared2(other_sentence)
  distance_matrix inputs = prepare inputs 
  </code></pre>
  
  <p>Is it possible to do this with this <code>gensim</code> function or should I just go my own customized version? Any ideas and solutions to deal with this problem in a better way?</p>