title: Gensim Word2vec model is not converged
body: <p>I'm training a Word2vec model using Gensim Word2vec on a well-known Wikipedia dump provided by Tobias Schnabel in the following link:
  <a href="http://www.cs.cornell.edu/~schnabts/eval/index.html" rel="nofollow noreferrer">http://www.cs.cornell.edu/~schnabts/eval/index.html</a> (about 4GB).</p>
  
  <p>I would like to understand how many epochs I should run the model for training until the model will be converged.</p>
  
  <p>I added the following code:</p>
  
  <pre><code> model = Word2Vec(size=self._number_of_dimensions_in_hidden_layer,
                      window=self._window_size,
                      min_count=3,
                      max_vocab_size=self._max_vocabulary_size,
                      sg=self._use_cbow,
                      seed=model_seed,
                      compute_loss=True,
                      iter=self._epochs)
      model.build_vocab(sentences)
  
      learning_rate = 0.025
      step_size = (learning_rate - 0.001) / self._epochs
  
      for i in range(self._epochs):
          end_lr = learning_rate - step_size
          trained_word_count, raw_word_count = model.train(sentences, compute_loss=True,
                                                           start_alpha=learning_rate,
                                                           end_alpha=learning_rate,
                                                           total_examples=model.corpus_count,
                                                           epochs=1)
          loss = model.get_latest_training_loss()
          print("iter={0}, loss={1}, learning_rate={2}".format(i, loss, learning_rate))
          learning_rate  *= 0.6
  
  
      model.save(model_name_path)
  </code></pre>
  
  <p>However I cannot see the model is converging:</p>
  
  <pre><code>iter=0, loss=76893000.0, learning_rate=0.025
  iter=1, loss=74870528.0, learning_rate=0.015
  iter=2, loss=73959232.0, learning_rate=0.009
  iter=3, loss=73605400.0, 
  learning_rate=0.005399999999999999
  iter=4, loss=73224288.0, 
  learning_rate=0.0032399999999999994
  iter=5, loss=73008048.0, 
  learning_rate=0.0019439999999999995
  iter=6, loss=72935888.0, 
  learning_rate=0.0011663999999999997
  iter=7, loss=72774304.0, 
  learning_rate=0.0006998399999999999
  iter=8, loss=72642072.0, 
  learning_rate=0.0004199039999999999
  iter=9, loss=72624384.0, 
  learning_rate=0.00025194239999999993
  iter=10, loss=72700064.0, 
  learning_rate=0.00015116543999999996
  iter=11, loss=72478656.0, 
  learning_rate=9.069926399999997e-05
  iter=12, loss=72486744.0, 
  learning_rate=5.441955839999998e-05
  iter=13, loss=72282776.0, 
  learning_rate=3.2651735039999986e-05
  iter=14, loss=71841968.0, 
  learning_rate=1.9591041023999992e-05
  iter=15, loss=72119848.0, 
  learning_rate=1.1754624614399995e-05
  iter=16, loss=72054544.0, 
  learning_rate=7.0527747686399965e-06
  iter=17, loss=71958888.0, 
  learning_rate=4.2316648611839976e-06
  iter=18, loss=71933808.0, 
  learning_rate=2.5389989167103985e-06
  iter=19, loss=71739256.0, 
  learning_rate=1.523399350026239e-06
  iter=20, loss=71660288.0, 
  learning_rate=9.140396100157433e-07
  </code></pre>
  
  <p>I don't undersatnd why the loss function result is not reducing and stay quite constant around 71M. </p>